comp.lang.idl-pvwave archive: archive » Re: Reading files with unknown amount of data

Home » Public Forums » archive » Re: Reading files with unknown amount of data

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Reading files with unknown amount of data [message #1357 is a reply to message #1355]

Thu, 04 November 1993 04:44

zawodny
Messages: 121
Registered: August 1992

Senior Member

In article <2bao2r$qi9@mailgzrz.TU-Berlin.DE> brau1231@camillo.fb12.tu-berlin.de writes:
> In article <28fbv1$qg6@reznor.larc.nasa.gov>, zawodny@arbd0.larc.nasa.gov (Dr Joseph M Zawodny) writes:
> |> >Robert Davis (rdavis@aerospace.aero.org) wrote:
> |> >I am trying to read a data file containing an unknown amount of data into
> |> >arrays in IDL. I know the format of the data in the file, but not the number
> |> >of pieces of data in the file. Currently, I read the file twice; once to
> |> >determine the amount of data in the file and then a second time to actually
> |> >read the data into an array (now that I know the size of array needed).
> |> >Is there a better way to do this (without having to read the file twice)?
> |>
> |> Use the EOF function in IDL. Try something like this:
> |>
> |>
>
>
> That's good, but you can also add entry's to a array.
> IDL let grow the array automatically, too.
>
>
>
> array=0. ; to initialize a variable called array
>
>
> openr,lun,filename, /get_lun ; no interest of 'lun's name
> point_lun,lun,0 ; but 'lun' know what I mean
> ; (logical unit number)
>
> while not eof(lun) do begin
>
> ;I use mostly
> readf,lun,a ; reading the entry of 'lun'
>
> array=[array,a] ; idl set the variable 'array' (if existing)
> ; to the right dimensions / append one entry
> ; so-called 'a'
> endwhile
> close,lun
> free_lun,lun
>
>
> ;cut the first entry of the array (array=0.)
>
> array=(1:*)
>
> from now on, the array 'array' has all entries and the right dimension.
>
> --
> Rainer Brauckhoff TU-Berlin Germany
>
> E-Mail: brau1231@camillo.fb12.tu-berlin.de

If I understand the way IDL uses memory, this "growing" of the array could be
a real memory hog. As I understand it, IDL will go off and try to find a
contiguous piece of memory for the new array. The areas where the old ARRAYs
will not be big enough for the new version of ARRAY. So if the final size of
ARRAY is 10000 points, then this method will consume (n*(n+1)/2) or 50 million
points worth of space (200 MB if we are talking floating point numbers) in the
process. Obviously this is a worse case scenario and is dependant upon what
else you may be doing in your program. In my mind it is clearly more efficient
to make an ARRAY which is way too big (like [file size in bytes]/4 elements).

--
Joseph M. Zawodny (KO4LW) NASA Langley Research Center
Internet: zawodny@arbd0.larc.nasa.gov MS-475, Hampton VA, 23681-0001
Packet: ko4lw@n4hog.va.usa

Report message to a moderator

[Message index]

		Re: Reading files with unknown amount of data By: pendleton on Thu, 04 November 1993 09:46
		Re: Reading files with unknown amount of data By: alans on Thu, 04 November 1993 08:09
		Re: Reading files with unknown amount of data By: zawodny on Thu, 04 November 1993 04:44
		Re: Reading files with unknown amount of data By: brau1231 on Thu, 04 November 1993 03:12
		Re: Reading files with unknown amount of data By: sterne on Thu, 30 September 1993 16:38
		Re: Reading files with unknown amount of data By: zawodny on Thu, 30 September 1993 12:27
		Re: Reading files with unknown amount of data By: pat on Thu, 30 September 1993 10:32
		Re: Reading files with unknown amount of data By: jim on Thu, 30 September 1993 10:47
		Re: Reading files with unknown amount of dat By: oet on Fri, 05 November 1993 00:28

Previous Topic:	Re: Calling External programm
Next Topic:	Re: Reading files with unknown amount of dat

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 06:26:20 PDT 2025

Total time taken to generate the page: 6.73365 seconds