| Re: Read large ascii file quickly [message #74265] |
Sun, 09 January 2011 12:08  |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <MPG.27914d7ad0fe57dd989914@news.giganews.com>,
David Fanning <news@dfanning.com> wrote:
> No argument from me here. But "slower" doesn't mean
> minutes slower for well-formatted ASCII files. Possibly
> some number of seconds slower. But, as someone pointed
> out, I don't have a lot of experience with extremely
> large ASCII files, as people with big data sets generally
> use binary formats.
In my experience ASCII files larger than a few 10's of MB
quickly become tedious for interactive work (and for batch
work when you have many of them). Gigabyte sized files
... forget it.
We find it to be worthwhile to convert virtually all ASCII files
to netCDF.
> Well, binary files force good documentation practices because
> without it you are utterly and totally without hope. :-)
If only *that* were true! Fortunately I very rarely see a
plain binary file anymore.
Ken
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
