comp.lang.idl-pvwave archive: archive » Re: Using "the IDL way" and it's still not fast enough

Home » Public Forums » archive » Re: Using "the IDL way" and it's still not fast enough

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Using "the IDL way" and it's still not fast enough [message #83712 is a reply to message #83710]

Wed, 27 March 2013 06:52

Brian Daniel
Messages: 80
Registered: July 2009

Member

Shooting from the hip here, but I expect performance would improve if you reorganized your array to [A,B,C,D,M*N]. The min operation should be much faster when it looks at the last dimension in the array.

On Tuesday, March 26, 2013 6:21:00 PM UTC-4, Edward Hyer wrote:
> Hello IDL wizards,
>
>
>
> I am trying to speed up a routine whose PROFILER looks like this (sorted by total time):
>
>
>
> Module Type Count Only(s) Avg.(s) Time(s) Avg.(s)
>
> REBIN (S) 2158 285.788439 0.132432 285.788439 0.132432
>
> MIN (S) 272 39.719054 0.146026 39.719054 0.146026
>
> FILE_SEARCH (S) 4 21.07632 5.26908 21.07632 5.26908
>
> REFORM (S) 2591 12.59025 0.004859 12.59025 0.004859
>
>
>
> The heart of the calculation is a
>
> MINARRAY = MIN(BIGARRAY,DIM=1), where
>
> BIGARRAY is [M*N,A,B,C,D] and so
>
> MINARRAY is [A,B,C,D].
>
> M=~10,000
>
> N=~200
>
> A,B,C,D are all <5
>
>
>
> In order to get to BIGARRAY, several steps of REBIN are required. And the result is a calculation that is too slow; it takes 6-20 seconds, depending on the particular machine we run it on. My instinct says that this is not a calculation that should be this slow, though I guess I could be wrong.
>
>
>
> Note that 1) I don't think memory is an obstacle, we have 16GB of RAM and the routine has peak usage <3 GB (I would know exactly if there was a working MEMTEST for 64bit IDL); 2) Threading is not really an option, as we intend to multiplex this process with 1 job per processor once we get it tuned.
>
>
>
> Does the collective wisdom of the newsgroup have any suggestions as to why this routine might be spending so much time REBINning, and how we might speed it up?
>
>
>
> In supplication,
>
>
>
> --Edward H.

Report message to a moderator

[Message index]

		Re: Using "the IDL way" and it's still not fast enough By: chris_torrence@NOSPAM on Wed, 27 March 2013 20:40
		Re: Using "the IDL way" and it's still not fast enough By: MarioIncandenza on Wed, 27 March 2013 15:49
		Re: Using "the IDL way" and it's still not fast enough By: MarioIncandenza on Wed, 27 March 2013 12:19
		Re: Using "the IDL way" and it's still not fast enough By: Craig Markwardt on Wed, 27 March 2013 09:56
		Re: Using "the IDL way" and it's still not fast enough By: MarioIncandenza on Wed, 27 March 2013 09:14
		Re: Using "the IDL way" and it's still not fast enough By: Jeremy Bailin on Wed, 27 March 2013 08:14
		Re: Using "the IDL way" and it's still not fast enough By: Kenneth Bowman on Wed, 27 March 2013 07:42
		Re: Using "the IDL way" and it's still not fast enough By: lecacheux.alain on Wed, 27 March 2013 07:15
		Re: Using "the IDL way" and it's still not fast enough By: Brian Daniel on Wed, 27 March 2013 06:52

Previous Topic:	Re: volume() command
Next Topic:	Regression fit and random noise

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 04:29:14 PDT 2025

Total time taken to generate the page: 1.22454 seconds