comp.lang.idl-pvwave archive: archive » READ, adn get data into an array from LARGE SIZE FILES

Home » Public Forums » archive » READ, adn get data into an array from LARGE SIZE FILES

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: READ, adn get data into an array from LARGE SIZE FILES [message #91394 is a reply to message #91380]

Wed, 08 July 2015 11:12

Craig Markwardt
Messages: 1869
Registered: November 1996

Senior Member

On Tuesday, July 7, 2015 at 12:43:49 PM UTC-4, luc...@gmail.com wrote:
> Hello All
> I have a big problem
> I need to open ,read and extract some useful data from big files, I have about 20 files ranging from 189MB to 22GB in size.
> There is a header (first 4 lines or so )
>
> Not all the lines are the same size, want to re-structure the file into an array with only a few data point from each line.
> Like from the first data line I just want the 1000
> From the following line I want to 212
> From the following line I want the 0.80000E+01

This is a parsing problem.

You will need to come up with a READF statement that can read each different kind of line from the file. I can't tell from your example, but if there are 3 different kinds of line, then you need three different READF statements. You will need to learn about the FORMAT keyword.

Then you will need to put those statements into a loop. If you know you will always get one line of format type 1, one line of format type 2, and 5000 lines of type 3, then you will need to put two READF statements according to those first types followed by a loop that reads the third type 5000 types. And so on. This is how we parse files.

CM

Report message to a moderator

[Message index]

		READ, adn get data into an array from LARGE SIZE FILES By: lucesmm on Tue, 07 July 2015 09:43
		Re: READ, adn get data into an array from LARGE SIZE FILES By: Jeremy Bailin on Tue, 07 July 2015 19:12
		Re: READ, adn get data into an array from LARGE SIZE FILES By: Craig Markwardt on Wed, 08 July 2015 11:12

Previous Topic:	READF: Input conversion error.
Next Topic:	Plotting one point per loop

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Jul 03 06:22:34 PDT 2026

Total time taken to generate the page: 0.80053 seconds