comp.lang.idl-pvwave archive: archive » Input Files

Home » Public Forums » archive » Input Files - Size Limit?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Input Files - Size Limit? [message #9046 is a reply to message #8977]

Wed, 21 May 1997 00:00

sritcey
Messages: 5
Registered: March 1997

Junior Member

Mark Harvey:
: Can IDL handle extremely large files?

David Fanning:
> Large files are usually not a problem with IDL, as long
> as you don't read all the data into memory at once. Use
> the associated variable method to read just that portion
> of the data you need at any one time

Christian Marquardt:
: But: assoc is called with an [...]
: 'array_structure'. Thus, to assoc() my 120MB array stored
: in 'foo.dat', I'd use something like

: openr, unit, 'foo.dat', /get_lun
: a = assoc(unit, fltarr(very_large_number_of_elements))
: ...
: free_lun, unit

: Now, the IDL interpreter will first create the fltarr(...),
: and then pass the very large array over to the assoc function.
: The problem: if the fltarr(...) is larger than the (virtual)
: memory of my machine, IDL will not assoc anything but stop
: with a warning saying that there is not enough memory (I know -
: I tried...).

David has already replied, agreeing that it's a problem, but not
mentioning the following workaround:

When I first read David's message I assumed that he meant using
something like

a = assoc (unit, fltarr (a_not_impossibly_large_number), $
offset = get_me_where_I_want_to_be)

And now 'walk' through the file, repeatedly re-assoc-ing with larger and
larger offsets. Now I agree that doing this 'manually' is a pain, but it
is workable if the format of the data file is such that it make sense to
take 'chunks' like this.

[PVWave, not IDL: does it matter?]

Report message to a moderator

[Message index]

		Input Files - Size Limit? By: Mark Harvey on Mon, 19 May 1997 00:00
		Re: Input Files - Size Limit? By: davidf on Sun, 18 May 1997 00:00
		Re: Input Files - Size Limit? By: pit on Fri, 23 May 1997 00:00
		Re: Input Files - Size Limit? By: haimov on Wed, 21 May 1997 00:00
		Re: Input Files - Size Limit? By: sritcey on Wed, 21 May 1997 00:00
		Re: Input Files - Size Limit? By: marq on Wed, 21 May 1997 00:00

Previous Topic:	Converting from IDL to mpeg or quicktime?
Next Topic:	PV_WAVE software company

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 03:01:31 PDT 2025

Total time taken to generate the page: 0.39961 seconds