| "infinite" nested for [message #86629] |
Mon, 25 November 2013 00:13  |
Chrisss
Messages: 18
Registered: November 2013
|
Junior Member |
|
|
Hi everyone! help me, please!
my IDL code has 3 nested for (for a 256x256x60matrix).estimated time for the result is 21hours approximately, but after 12 hours my work crashed! if I decrease matrix dim., the result is ok! do you think the problem is my pc? do you have an alternative idea for my "infinite for"?
Cheers, M.C.
|
|
|
|
|
|
| Re: "infinite" nested for [message #86632 is a reply to message #86630] |
Mon, 25 November 2013 02:24  |
Chrisss
Messages: 18
Registered: November 2013
|
Junior Member |
|
|
evil? oh no!!! :-(
this is for-loop:
for i=0, dx-1 do begin
for j=0,dy-1 do begin
y=reform (Im[i,j,*],N_ELEMENTS(x)))
fitErrors=FLTARR(N_ELEMENTS(x))
T1values=FLTARR(N_ELEMENTS(x))
S0values=FLTARR(N_ELEMENTS(x))
fvalues=FLTARR(N_ELEMENTS(x))
for t=0,N_ELEMENTS(x)-1 do begin
T0=t
yCurr=y
if (t ne 0) then yCurr[0:T0]=-y[0:To]
A=mpfitfun('myfunct', x,yCurr, ERR, PARINFO=parinfo, WEIGHTS=w, YFIT=yfit, $ BESTNORM=error, /QUIET )
fitErrors[t]=error
T1values[t]=A[0]
S0values[t]=A[1]
fvalues[t]=A[2]
endfor
min=MIN(fitErrors,idmin)
T1=T1values[idmin]
f=fvalues[idmin]
S0=S0values[idmin]
yfit=myfunct(x,[S0,f,T1])
plot,x,y
oplot,x,yfit
oplot,x,abs(yfit)
wait,1
T1map[i,j]=T1
endfor
endfor
where x=t;t=t*(findgen(60)+1) dx=dy=256, Im is a 256x256x60 matrix
cheers,
MC
|
|
|
|
| Re: "infinite" nested for [message #86633 is a reply to message #86632] |
Mon, 25 November 2013 03:22 |
Kai Muehlbauer
Messages: 13
Registered: February 2011
|
Junior Member |
|
|
Am 25.11.2013 11:24, schrieb Chrisss:
> evil? oh no!!! :-(
> this is for-loop:
>
> for i=0, dx-1 do begin
> for j=0,dy-1 do begin
> y=reform (Im[i,j,*],N_ELEMENTS(x)))
> fitErrors=FLTARR(N_ELEMENTS(x))
> T1values=FLTARR(N_ELEMENTS(x))
> S0values=FLTARR(N_ELEMENTS(x))
> fvalues=FLTARR(N_ELEMENTS(x))
> for t=0,N_ELEMENTS(x)-1 do begin
> T0=t
> yCurr=y
> if (t ne 0) then yCurr[0:T0]=-y[0:To]
> A=mpfitfun('myfunct', x,yCurr, ERR, PARINFO=parinfo, WEIGHTS=w, YFIT=yfit, $ BESTNORM=error, /QUIET )
> fitErrors[t]=error
> T1values[t]=A[0]
> S0values[t]=A[1]
> fvalues[t]=A[2]
> endfor
> min=MIN(fitErrors,idmin)
> T1=T1values[idmin]
> f=fvalues[idmin]
> S0=S0values[idmin]
> yfit=myfunct(x,[S0,f,T1])
> plot,x,y
> oplot,x,yfit
> oplot,x,abs(yfit)
> wait,1
> T1map[i,j]=T1
> endfor
> endfor
>
> where x=t;t=t*(findgen(60)+1) dx=dy=256, Im is a 256x256x60 matrix
> cheers,
> MC
First thing would be to remove the "wait,1". This will cancel out 256 *
256 (65536) seconds (which is around 18 hours). Also plotting is time
consuming.
But there may be additional improvements possible by reorganising the
for loops and doing some vectorisation. But this is for the experts here.
Cheers,
Kai
|
|
|
|
|
|
| Re: "infinite" nested for [message #86635 is a reply to message #86634] |
Mon, 25 November 2013 04:12 |
Kai Muehlbauer
Messages: 13
Registered: February 2011
|
Junior Member |
|
|
Am 25.11.2013 12:53, schrieb Chrisss:
> thank you so much! I'm a new idl user and I'm new in the world of
> programmers...so I don't know a lot of things. unfortunately for my
> work is important to view plots to analyze the results! Anyway, I
> removed "wait,1" for faster work! thanks, Christina
If you want to view the plots to analyze the results, one could consider
to do calculation first and save the results to disk without plotting.
This should be very fast compared to your momentary solution.
Afterwards you could load the results and plot them one by one.
But anyway, this review is a time consuming job. If you already know
what you are looking for in the data, you could search the data
automatically. Then you can plot only those datasets you are interested in.
As you haven't said anything about the underlying problem, it is not so
easy to recommend you further things. If you are looking for eg.
outliers or the like you could plot 2d images of your results to find
interesting parts.
So if you can explain a bit about your data there may possibly be more
help to come.
Cheers,
Kai
|
|
|
|
|
|
| Re: "infinite" nested for [message #86641 is a reply to message #86638] |
Mon, 25 November 2013 07:38 |
Mats Löfdahl
Messages: 263
Registered: January 2012
|
Senior Member |
|
|
Den måndagen den 25:e november 2013 kl. 15:56:08 UTC+1 skrev Chrisss:
> Kai, thank you so much!!! ^_^
>
> you were very accurate and I thank you so much for your helpful and your informations. I was not too detailed because I did not want to bore you and take advantage of your precious time, but I really need help! so I try to explain it better:
>
> I'm looking for best exponential curve fit, and so the best values about S0, T1 and f (curve parameters) and minimum error. So, I would like to show the plot with the best fit and the min error value in output. I need nested for to display a T1 map of all values calculated within the loops. values refer to everyone of 256x256x60 pixel matrix that I need to built the T1 map
If I read your code right, your inner loop is used to change the sign of part of the array of data that you are fitting the exponential to. And then you are trying to find the version of your y array that give the best fit.
Any reason you can't remove the inner loop and just fit to abs(reform(Im[i,j,*],N_ELEMENTS(x))?
|
|
|
|
|
|
| Re: "infinite" nested for [message #86643 is a reply to message #86642] |
Mon, 25 November 2013 08:37 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Chrisss writes:
Another way you can improve this code is to only calculate unchanging
variable once, and outside the loops. For example, you have this, in
which you calculate the number of elements of X, which never changes,
six times!
for i=0, dx-1 do begin
for j=0,dy-1 do begin
y=reform (Im[i,j,*],N_ELEMENTS(x)))
fitErrors=FLTARR(N_ELEMENTS(x))
T1values=FLTARR(N_ELEMENTS(x))
S0values=FLTARR(N_ELEMENTS(x))
fvalues=FLTARR(N_ELEMENTS(x))
for t=0,N_ELEMENTS(x)-1 do begin
This can be changed to this:
num_x = N_Elements(x)
fitErrors=FLTARR(num_x)
T1values=FLTARR(num_x)
S0values=FLTARR(num_x)
fvalues=FLTARR(num_x)
for i=0, dx-1 do begin
for j=0,dy-1 do begin
y=reform (Im[i,j,*],num_x)
for t=0,num_x-1 do begin
If you need to reinitialize your arrays inside the loop, then do this,
rather than re-allocating more memory. I think this is probably why your
program crashed after 12 hours. You ran out of dynamic memory on your
machine:
fitErrors=Temporary(fitErrors)*0.0
T1values=Temporary(T1values)*0.0
S0values=Temporary(S0values)*0.0
fvalues=Temporary(fvalues)*0.0
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")
|
|
|
|
|
|
|
|
|
|
|
|
| Re: "infinite" nested for [message #86660 is a reply to message #86658] |
Tue, 26 November 2013 09:05 |
Chrisss
Messages: 18
Registered: November 2013
|
Junior Member |
|
|
I just decided to put out plots from my code or save and show only significative plots
thanks for your all considerations and instructions, they were important to me
Christina
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
