| Re: array concatenation and optimization [message #26848 is a reply to message #26793] |
Fri, 28 September 2001 06:56   |
![Stein Vidar Hagfors H[1] is currently offline](theme/default/images/xoffline.png.pagespeed.ic.XRkd1fkXye.png "Stein Vidar Hagfors H[1] is currently offline")
Stein Vidar Hagfors H[1]
Messages: 56
Registered: February 2000
|
Member |
|
|
"Mark Hadfield" <m.hadfield@niwa.cri.nz> writes:
> From: "Craig Markwardt" <craigmnet@cow.physics.wisc.edu>
>> ...
>> My pet favorite is to read the file line by line, but grow the array
>> in chunks. I usually grow it by powers of two until a certain limit.
>> Example (not tested),
>
> I built essentially the same logic into my MGH_Vector class, see
>
> http://katipo.niwa.cri.nz/~hadfield/gust/software/idl/mgh_ve ctor__define.pro
>
> The data are stored in a pointer array which is initialised with spare
> capacity. Elements can be added one at a time; every time the capacity of
> the array is reached it is extended (which means it is replaced by a larger
> one). After some trial & error I set the initial size to 1000 & the resizing
> algorithm to
> new_size = round(1.5*old_size) > (new_size+1000)
>
> The advantage of doing this inside an object, of course, is that all the
> details can be hidden and forgotten about.
>
> Performance is acceptable: creating the object, adding 10^6 items (5-char
> strings), retrieving them all and then destroying the object takes 20 s
> (Pentium III 800). This compares with about 3 s to do the same operations
> with a plain string array, using the same logic to extend the array when
> necessary. Either way, the time varies more-or-less linearly with the number
> of items to be processed.
If you're taking the trouble of hiding it all inside an object, why
not go further to use e.g. lists, dropping the need for replacing
anything until possibly after as part of a "reconstitution" operation
(would need the user program to signal when building is finished - or
possibly trigger it automatically when a first read access is made ?)
--
------------------------------------------------------------ --------------
Stein Vidar Hagfors Haugan
ESA SOHO SOC/European Space Agency Science Operations Coordinator for SOHO
NASA Goddard Space Flight Center, Email: shaugan@esa.nascom.nasa.gov
Mail Code 682.3, Bld. 26, Room G-1, Tel.: 1-301-286-9028/240-354-6066
Greenbelt, Maryland 20771, USA. Fax: 1-301-286-0264
------------------------------------------------------------ --------------
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
