comp.lang.idl-pvwave archive: archive » Re: Concatenating arrays

Home » Public Forums » archive » Re: Concatenating arrays - speed issues?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Concatenating arrays - speed issues? [message #76471 is a reply to message #76470]

Wed, 08 June 2011 04:55

Michael Galloy
Messages: 1114
Registered: April 2006

Senior Member

Rob <rjp23@le.ac.uk> wrote:
> On Jun 7, 5:42 pm, Craig Markwardt <craig.markwa...@gmail.com> wrote:
>> What you are doing is the "IDL way" in the sense that it's a natural
>> use of the concatenation feature of the language.
>>
>> But as you noticed, the performance degrades for lots of append
>> operations.
>>
>> The next best way is to grow the array in chunks, and then fill in the
>> chunks with available data. This forces you to keep track of the
>> number of used elements in the array, separate from the array size.
>> Once you fill the available chunk, only then do you add another
>> chunk.
>>
>> This doesn't really get rid of the problem you noticed, but it does
>> reduce the problem significantly. So, if each chunk has 1000
>> elements, then the performance degradation is 1000x less. Then you
>> can start to get fancy by growing the array with variable sized
>> chunks.
>>
>> Craig
>
>
> That might actually be quite a nice solution, it just means keeping
> track with a few more counters.
>
> I'll have a play and see how it goes :-)
>
> Cheers

My classes do the accounting for this technique for you.

Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation

Report message to a moderator

[Message index]

		Re: Concatenating arrays - speed issues? By: Michael Galloy on Thu, 09 June 2011 04:55
		Re: Concatenating arrays - speed issues? By: rjp23 on Thu, 09 June 2011 04:35
		Re: Concatenating arrays - speed issues? By: penteado on Wed, 08 June 2011 05:05
		Re: Concatenating arrays - speed issues? By: Michael Galloy on Wed, 08 June 2011 04:55
		Re: Concatenating arrays - speed issues? By: rjp23 on Wed, 08 June 2011 02:48
		Re: Concatenating arrays - speed issues? By: rjp23 on Wed, 08 June 2011 02:46
		Re: Concatenating arrays - speed issues? By: penteado on Tue, 07 June 2011 11:20
		Re: Concatenating arrays - speed issues? By: ben.bighair on Tue, 07 June 2011 10:24
		Re: Concatenating arrays - speed issues? By: Craig Markwardt on Tue, 07 June 2011 09:42

Previous Topic:	Re: openr or openw with COMPRESS flag broken for large data files
Next Topic:	labeling is not visible in postscript output

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Tue Dec 02 07:47:55 PST 2025

Total time taken to generate the page: 1.60310 seconds