| Re: Astronomys` Sixth Neighbour Needs Help [message #35963 is a reply to message #35884] |
Fri, 25 July 2003 15:34   |
Pavel Romashkin
Messages: 166
Registered: April 1999
|
Senior Member |
|
|
Hi Bruno,
What you're discovering in the meantime is that, for large arrays
(n*10E7 points) memory allocation is slower than loops. This is what's
happening with Rob's code. The program, while great, expands the data
size by at least a factor of 8 more than necessary (counting matrices
and index expansion for SORT), hence the incredible memory consumption.
Try the following:
FUNCTION CALCULATE_6TH_NEIGHBOR, x, y
; Calculate all of the distances
n = n_elements(x)
u = 1+bytarr(n)
d6 = fltarr(n)
dx = u#x
dy = u#y
for i = 0L, n-1 do dx[0, i] = dx[*, i] - x
for i = 0L, n-1 do dy[0, i] = dy[*, i] - x
d2 = sqrt(dx^2+dy^2)
for i = 0L, n-1 do begin
temp = d2[*, i]
ind = sort(temp)
d6[i] = temp[ind[6]]
endfor
return, d6
END
Not all loops are bad in IDL.
Why are these loops faster? Memory allocation is much smaller, thus
contiguous chunks are easier available - faster; all loops are aligned
by the row and kept short - this is important. In my test, speed is
higher by a factor of 20 with identical results.
Cheers,
Pavel
astronomer wrote:
>
> AMAZING!!!!
>
> I am extremely amazed and impressed at the increase in processing
> speed that the changes made!!!!!!
> I was not expecting such a change! Amazing how inefficient my program
> was. Where was all the time going to? I mean, how come the for loops
> made the program so slow whereas now...
>
> I can`t thank you enough!
> I managed to run a 3800 star file in under 11 minutes whereas before
> it took over 24 hours. Now I am running a 9300 star file!
> For the larger file I am using a dual processor computer with 1 Gb and
> with 2400GHz each processor. The program sucks 99.8% of the CPU but
> it is running.
>
> I am also very surprised at how much memory is allocated to the IDL
> processes. What I mean is that the whole computer would slow down
> immensely, giving priority to IDL. Strange.
>
> This is indeed a good introduction to this newsgroup! and to IDL.
>
> Thanks Rob!
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
