| Re: ASCII read problem [message #44457] |
Fri, 17 June 2005 09:39 |
David Streutker
Messages: 34
Registered: June 2005
|
Member |
|
|
Can't you use SKIP_LUN with the /LINES keyword? (Available in version
5.6 and up.) It's kind of slow, so it may be more efficient just to
read in the entire file.
-David
|
|
|
|
| Re: ASCII read problem [message #44461 is a reply to message #44457] |
Fri, 17 June 2005 09:01  |
Michael Wallace
Messages: 409
Registered: December 2003
|
Senior Member |
|
|
> I have some large ASCII files I need to extract data from. The files
> are organized such that each line represent one dataelement, and each
> line does not have a constant length or size.
>
> Is it any elegant method were I can just access a set of specific lines
> without having to read through the whole file?
> I wish /LINES was a keyword to point_lun
Maybe someone else has a good idea, but I think you have to read through
the entire file. In order to know where a line begins you have to look
for the newline character (or carriage return/line feed). If the
newline character shows up at different places and you can't predict
where it will fall, you have to read through all the characters to know
where it is.
You should be able to speed things up if it's taking too long to read
your text files. I don't know how IDL works behind the scenes with
respect to file I/O. Someone correct me if I am wrong or if IDL already
does this for you. In other languages, it's possible to read in your
data as blocks rather than as lines or characters at a time. By
reading blocks into a buffer, you can greatly reduce the total number of
I/O operations. And I/O is usually the bottleneck in programs like this.
-Mike
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
