comp.lang.idl-pvwave archive: archive » memory allocation for structure arrays

Home » Public Forums » archive » memory allocation for structure arrays

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

memory allocation for structure arrays [message #11586]

Wed, 29 April 1998 00:00

Ian Sprod
Messages: 5
Registered: November 1997

Junior Member

Hi,

I am trying to read a pretty large data file (40Mb) into IDL. The file
is 925,801 records, each 44 bytes long. I can describe the 44 bytes as a
data structure and then replicate this to make a structure array (albeit
a very large one). Then, in theory, reading in the data is a breeze.

The problem is that IDL runs out of memory trying to read in the file.
It seems that each line of the structure array is somehow requiring MORE
than 44 bytes of memory. Poking around with top and free shows that it
seems to be using ~312 bytes for each line instead. At this rate I can
only read in the first ~225,000 lines of the file.

Does anyone know exactly how IDL allocates memory for structures?

Should I be using an associative array to do this?

I am running IDL 5.0.2 on a Linux box with 128Mb of physical RAM and
twice that of swap space.

Thanks for any help or advice,

Ian

--

Ian E. Sprod
CIRES ian@ngdc.noaa.gov
NOAA/NGDC E/GC1 http://swat.ngdc.noaa.gov/~ian
325 Broadway 303-497-6284 (voice)
Boulder, CO 80303 303-497-6513 (fax)

Report message to a moderator

[Message index]

		memory allocation for structure arrays By: Ian Sprod on Wed, 29 April 1998 00:00
		Re: memory allocation for structure arrays By: mallors on Sat, 02 May 1998 00:00
		Re: memory allocation for structure arrays By: Ian Sprod on Fri, 01 May 1998 00:00
		Re: memory allocation for structure arrays By: korpela on Thu, 30 April 1998 00:00
		Re: memory allocation By: Essa Yacoub on Tue, 27 July 1999 00:00
		Re: memory allocation By: Liam Gumley on Tue, 27 July 1999 00:00
		Re: memory allocation By: Craig Markwardt on Tue, 27 July 1999 00:00
		Re: memory allocation By: davidf on Mon, 26 July 1999 00:00
		Re: memory allocation By: Essa Yacoub on Mon, 26 July 1999 00:00
		Re: memory allocation By: davidf on Mon, 26 July 1999 00:00

Previous Topic:	password-entry widget
Next Topic:	Re: idlwave mode for emacs/xemacs?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:09:48 PDT 2025

Total time taken to generate the page: 0.00643 seconds