Home »
Public Forums »
archive »
Re: reading long files
| Re: reading long files [message #1445 is a reply to message #1442] |
Sat, 06 November 1993 01:51  |
rdc
Messages: 1
Registered: November 1993
|
Junior Member |
|
|
A previous post mentions a two pass method of reading the file first
determining the necessary size of the array and then reading into it. For
people running on unix systems spawning a 'wc' will accomplish the first
pass faster than reading it with idl. Our wc returns # lines, # words, and
# characters in the file so the array dimensions can be calculated by your
proceedure. (its more efficient to use a 1d array and REFORM it to 2d
after reading it) One drawback (other than portability) is when some
entries completely fill the available width of their column. This causes
two entries to run together and a free format read will lose its place.
But this situation can be flagged when #cols = #words / #lines doesn't
come out integer. This situation has to be handled as a special case,
otherwise a proceedure to do this can be kludged up to be pretty general.
Another aproach I have used a few times is to grow the array a record at a
time into a stattic array. This avoids both the wasteful use of memory and
also greatly speeds things up since the array isn't repeatedly being
recopied. Of course you have to predimension the array to be at least as
large as the file being read and you can have nightmares thinking about
ways you can overflowing it. Don't know if the options I used still work
with version 3.x, it was about 2.2 when I tried this. I dropped it in
favor of the safer two pass method.
--
Richard Clark
rclark@lpl.arizona.edu (or the address in the header)
For the 10,000th time: it's in Astronomical Algorithms by Jean Meeus!
|
|
|
|
Current Time: Sat Oct 11 12:55:42 PDT 2025
Total time taken to generate the page: 1.04259 seconds
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
