| Re: Optimizing loops [message #92372 is a reply to message #92368] |
Thu, 03 December 2015 02:10   |
Sergey Anfinogentov
Messages: 11
Registered: September 2012
|
Junior Member |
|
|
среда, 2 декабря 2015 г., 19:04:33 UTC пользователь sam.t...@gmail.com написал:
> Hello all! I am working with a large number of satellite data files. The files are quite big, but I've never had this much trouble working with them before, even while doing similar processing.
>
> The basic program structure that's the slowest is below; it also just dominates the processing power as it enters this loop. I need to do this for each file I process.
>
> ; we want to calculate the percent of each satellite pixel
> ; covered by land
> land_perc = FLTARR(N_ELEMENTS(field1))
> FOR j = 0, N_ELEMENTS(field1)-1 DO BEGIN
> dist = (((land_lon - sat_lon[j])*COSD(land_lat))^2. $
> + (land_lat - sat_lat[j])^2.)^0.5*111.12
>
> ind14 = WHERE(dist LE 14.)
> land14 = land_mask(ind14)
> landy = WHERE(land14 EQ 0, landy_cnt)
> land_perc[j] = FLOAT(landy_cnt)/FLOAT(N_ELEMENTS(land14))*100
> ENDFOR
>
> If anyone has optimization suggestions, please let me know! Thanks :)
There is no problem with loops here because the code inside the loop is vectorised.
The code is slow because of two things
1)The computational complexity is of the order of N^2. For large N it will be slow even if coded in C or Fortran.
2)In present form, COSD function is called N times. I guess, land_lat is an array. Sum functionsFunctions like SIN, COS, ALOG are computationally very heavy and take a lot of time to be called.
3) Square root "^0.5" is computed many times inside loop. It is also very computationally heavy operation.
You can try the following steps to improve the performance
(1) If land_lat is a large array, you can improve the performance by calculating COSD only one time before the loop:
cosd_land_lat = COSD(land_lat)
Then you should use "cosd_land_lat" instead of "COSD(land_lat)" inside the loop.
(2) don't calulate square root of inside the loop. Compute dist^2 instead of dist and compare it with the modified trhreshold.
The optimised code may look like this
land_perc = FLTARR(N_ELEMENTS(field1))
cosd_land_lat = COSD(land_lat)
thr14 = (14./111.12)^2
FOR j = 0, N_ELEMENTS(field1)-1 DO BEGIN
dist2 = (((land_lon - sat_lon[j])*cosd_land_lat)^2. $
+ (land_lat - sat_lat[j])^2.)
ind14 = WHERE(dist2 LE thr14)
land14 = land_mask(ind14)
landy = WHERE(land14 EQ 0, landy_cnt)
land_perc[j] = FLOAT(landy_cnt)/FLOAT(N_ELEMENTS(land14))*100
ENDFOR
Hope, it will help
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
