comp.lang.idl-pvwave archive: archive » array concatenation and optimization

Home » Public Forums » archive » array concatenation and optimization

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: array concatenation and optimization [message #26855 is a reply to message #26793]

Thu, 27 September 2001 15:25

Mark Hadfield
Messages: 783
Registered: May 1995

Senior Member

From: "Craig Markwardt" <craigmnet@cow.physics.wisc.edu>
> ...
> My pet favorite is to read the file line by line, but grow the array
> in chunks. I usually grow it by powers of two until a certain limit.
> Example (not tested),

I built essentially the same logic into my MGH_Vector class, see

http://katipo.niwa.cri.nz/~hadfield/gust/software/idl/mgh_ve ctor__define.pro

The data are stored in a pointer array which is initialised with spare
capacity. Elements can be added one at a time; every time the capacity of
the array is reached it is extended (which means it is replaced by a larger
one). After some trial & error I set the initial size to 1000 & the resizing
algorithm to

new_size = round(1.5*old_size) > (new_size+1000)

The advantage of doing this inside an object, of course, is that all the
details can be hidden and forgotten about.

Performance is acceptable: creating the object, adding 10^6 items (5-char
strings), retrieving them all and then destroying the object takes 20 s
(Pentium III 800). This compares with about 3 s to do the same operations
with a plain string array, using the same logic to extend the array when
necessary. Either way, the time varies more-or-less linearly with the number
of items to be processed.

The timing code is in

http://katipo.niwa.cri.nz/~hadfield/gust/software/idl/mgh_ex ample_container.
pro

---
Mark Hadfield
m.hadfield@niwa.cri.nz http://katipo.niwa.cri.nz/~hadfield
National Institute for Water and Atmospheric Research

--
Posted from clam.niwa.cri.nz [202.36.29.1]
via Mailgate.ORG Server - http://www.Mailgate.ORG

Report message to a moderator

[Message index]

		array concatenation and optimization By: Sean Raffuse on Wed, 26 September 2001 11:18
		Re: array concatenation and optimization By: Stein Vidar Hagfors H[1] on Fri, 28 September 2001 06:56
		Self-sizing arrays (was array concat and opt) By: ngls on Thu, 04 October 2001 07:22
		Re: array concatenation and optimization By: Mark Hadfield on Thu, 27 September 2001 15:25

Previous Topic:	Declaration of variables in IDL
Next Topic:	Re: TOTAL(): was Declaration of variables in IDL

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 14:55:59 PDT 2025

Total time taken to generate the page: 0.00464 seconds