| Re: ASCII data with 5000 columns [message #14598] |
Thu, 11 March 1999 00:00 |
Vapuser
Messages: 63
Registered: November 1998
|
Member |
|
|
"R.Bauer" <R.Bauer@fz-juelich.de> writes:
> Hi,
>
> how to read an ASCII datafile with 5000 data coloumns.
> The length of each line is about 56400 characters or longer.
>
> reads, readf don't work because the max stringlenth is exceeded.
>
> The only way I know to read is readu.
> I separate the numbers in the byte array convert them to string and at
> last to
> float.
>
> It's a bit unbelievable that's idl is able to handle more than 2 Giga
> Byte but
> only if they are organized by lines and a few columns.
>
>
>
> Any further ideas?
Um... I'm a bit confused. Is this file coming from some external
source over which you have no control? If not, then why would you
write out an ascii file with so many columns? To me this defeats the
purpose of writing ascii, which I would characterise as ease of viewing your
data with a simple 'cat' or 'more' or 'type' or whatever ascii file
viewer you want. I'd consider writing binary, since you're not
getting that particular benefit this particular type of data file.
Can you actually view these files in some ascii viewer? I jerry
rigged one up with a line length of 64K and even venerable 'vi'
complained about the line length!
If it comes from some outside source over which you have no control,
then forget everything I just wrote. ;->
Now that I think of it, it must be, since I can't think of a way to
write a line this long out in IDL without converting it to a byte
array and using writeu.
This must be some implied do-loop in fortran, right? If so, I'd get
them to change the fortran source to write a more user friendly file.
whd
|
|
|
|
| Re: ASCII data with 5000 columns [message #14604 is a reply to message #14598] |
Thu, 11 March 1999 00:00  |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
R.Bauer wrote:
>
> Hi,
>
> how to read an ASCII datafile with 5000 data coloumns.
> The length of each line is about 56400 characters or longer.
>
> reads, readf don't work because the max stringlenth is exceeded.
>
> The only way I know to read is readu.
> I separate the numbers in the byte array convert them to string and at
> last to
> float.
>
> It's a bit unbelievable that's idl is able to handle more than 2 Giga
> Byte but only if they are organized by rows and a few columns.
>
> R.Bauer
Hallo Reimar,
maybe you can use string arrays and the format statement? Here's an
example:
a=replicate('*',62000L)
b=string(a,format='(32000A1)')
will give you a two-element string array.
Even better of course to have a formatted file where you can assign the
format statement to individual tags (e.g. 5000A8)
try: c=strarr(5000)
readf,ilun,c,format='(5000A8)'
Hoffe, das hilft,
Viele Gruesse,
Martin.
--
------------------------------------------------------------ -------
Dr. Martin Schultz
Department for Engineering&Applied Sciences, Harvard University
109 Pierce Hall, 29 Oxford St., Cambridge, MA-02138, USA
phone: (617)-496-8318
fax : (617)-495-4551
e-mail: mgs@io.harvard.edu
Internet-homepage: http://www-as.harvard.edu/people/staff/mgs/
------------------------------------------------------------ -------
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
