comp.lang.idl-pvwave archive: archive » Re: Using "the IDL way" and it's still not fast enough

Home » Public Forums » archive » Re: Using "the IDL way" and it's still not fast enough

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Using "the IDL way" and it's still not fast enough [message #83709 is a reply to message #83708]

Wed, 27 March 2013 07:42

Kenneth Bowman
Messages: 86
Registered: November 2006

Member

I don't think so. In IDL the first array index varies fastest (is
contiguous in memory). But it would depend on how they have chosen to
implement the MIN algorithm. Wouldn't it be nice if they provided
information about things like that in the docs?

Ken Bowman

On 2013-03-27 13:52:26 +0000, Brian J. Daniel said:

> Shooting from the hip here, but I expect performance would improve if
> you reorganized your array to [A,B,C,D,M*N]. The min operation should
> be much faster when it looks at the last dimension in the array.
> On Tuesday, March 26, 2013 6:21:00 PM UTC-4, Edward Hyer wrote:
>> Hello IDL wizards,
>>
>>
>>
>> I am trying to speed up a routine whose PROFILER looks like this
>> (sorted by total time):
>>
>>
>>
>> Module Type Count Only(s) Avg.(s) Time(s) Avg.(s)
>>
>> REBIN (S) 2158 285.788439 0.132432 285.788439 0.132432
>>
>> MIN (S) 272 39.719054 0.146026 39.719054 0.146026
>>
>> FILE_SEARCH (S) 4 21.07632 5.26908 21.07632 5.26908
>>
>> REFORM (S) 2591 12.59025 0.004859 12.59025 0.004859
>>
>>
>>
>> The heart of the calculation is a>> MINARRAY = MIN(BIGARRAY,DIM=1),
>> where>> BIGARRAY is [M*N,A,B,C,D] and so>> MINARRAY is [A,B,C,D].>>
>> M=~10,000
>>
>> N=~200
>>
>> A,B,C,D are all <5
>>
>>
>>
>> In order to get to BIGARRAY, several steps of REBIN are required. And
>> the result is a calculation that is too slow; it takes 6-20 seconds,
>> depending on the particular machine we run it on. My instinct says that
>> this is not a calculation that should be this slow, though I guess I
>> could be wrong.
>>
>>
>>
>> Note that 1) I don't think memory is an obstacle, we have 16GB of RAM
>> and the routine has peak usage <3 GB (I would know exactly if there was
>> a working MEMTEST for 64bit IDL); 2) Threading is not really an option,
>> as we intend to multiplex this process with 1 job per processor once we
>> get it tuned.
>>
>>
>>
>> Does the collective wisdom of the newsgroup have any suggestions as to
>> why this routine might be spending so much time REBINning, and how we
>> might speed it up?
>>
>>
>>
>> In supplication,
>>
>>
>>
>> --Edward H.

Report message to a moderator

[Message index]

		Re: Using "the IDL way" and it's still not fast enough By: chris_torrence@NOSPAM on Wed, 27 March 2013 20:40
		Re: Using "the IDL way" and it's still not fast enough By: MarioIncandenza on Wed, 27 March 2013 15:49
		Re: Using "the IDL way" and it's still not fast enough By: MarioIncandenza on Wed, 27 March 2013 12:19
		Re: Using "the IDL way" and it's still not fast enough By: Craig Markwardt on Wed, 27 March 2013 09:56
		Re: Using "the IDL way" and it's still not fast enough By: MarioIncandenza on Wed, 27 March 2013 09:14
		Re: Using "the IDL way" and it's still not fast enough By: Jeremy Bailin on Wed, 27 March 2013 08:14
		Re: Using "the IDL way" and it's still not fast enough By: Kenneth Bowman on Wed, 27 March 2013 07:42
		Re: Using "the IDL way" and it's still not fast enough By: lecacheux.alain on Wed, 27 March 2013 07:15
		Re: Using "the IDL way" and it's still not fast enough By: Brian Daniel on Wed, 27 March 2013 06:52

Previous Topic:	Re: volume() command
Next Topic:	Regression fit and random noise

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Thu Oct 09 22:08:42 PDT 2025

Total time taken to generate the page: 0.16024 seconds