comp.lang.idl-pvwave archive: archive » Re: Read/format mixed int/flt/strng ascii files

Home » Public Forums » archive » Re: Read/format mixed int/flt/strng ascii files

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Read/format mixed int/flt/strng ascii files [message #31786]

Tue, 20 August 2002 08:08

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

david (knapp@ucsu.colorado.nospam.edu) writes:

> So maybe the ultimate answer is that everyone has been doing this "by hand"
> every time?

JD has probably written some kind of "chaotic string" theory
to deal with this. But, yes, the rest of us do it
"by hand". :-(

Cheers,

David
--
David W. Fanning, Ph.D.
Fanning Software Consulting, Inc.
Phone: 970-221-0438, E-mail: david@dfanning.com
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Toll-Free IDL Book Orders: 1-888-461-0155

Report message to a moderator

Re: Read/format mixed int/flt/strng ascii files [message #31787 is a reply to message #31786]

Tue, 20 August 2002 07:55

david[3]
Messages: 4
Registered: August 2002

Junior Member

Rick Towler wrote:

> "david" <knapp@ucsu.colorado.nospam.edu> wrote
>
>> I need to operate on large tab delimited ascii files that contain a very
>> wide mix of int, flt, string etc
>
> Can you elaborate on the format a little? When you say a wide mix, does
> that mean that any field could contain any data type or is there at least
> consistency within each file?

Yes, each file is self consistent, but file to file I have differing numbers of
fields
and numbers of rows. Type of field in a given position can change from a number
to
a character or int to flt etc...

> Do the string fields contain spaces?

I think I'm free of spaces (have underscores in)

>
> I have dealt with this by reading each line in the file as a string, parsing
> it with STRSPLIT and dealing with the parts but even this approach has
> limits.

So maybe the ultimate answer is that everyone has been doing this "by hand"
every time?

--
David Knapp
Atmospheric Odd Nitrogen Project
Atmospheric Chemistry Division
National Center for Atmospheric Research
Box 3000, Boulder, CO 80307

Report message to a moderator

Re: Read/format mixed int/flt/strng ascii files [message #31793 is a reply to message #31787]

Tue, 20 August 2002 07:07

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

david wrote:

>
> I need to operate on large tab delimited ascii files that contain a very
> wide mix of int, flt, string etc
> and am having a tough time finding a non-complicated way of reading it
> all and manipulating it.
> I've looked at creating my own template, but I have a variety of file
> formats so I'd need to spend
> a lot of time creating each template.
>
> Presumably this has been done extensively.
>
> Does anyone know of a user defined procedure that already does this with
> more automation and less
> hand-coding of the final structure?
>
> Any tips appreciated :)
>
> -David

Dear David

may be I have a routine for this but I am not sure.
Can you show some lines please.

regards

Reimar

--
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
http://www.fz-juelich.de/icg/icg-i/
============================================================ ======
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg-i/idl_icglib/idl_lib_intro. html

Report message to a moderator

Re: Read/format mixed int/flt/strng ascii files [message #31800 is a reply to message #31793]

Mon, 19 August 2002 14:48

Rick Towler
Messages: 821
Registered: August 1998

Senior Member

"david" <knapp@ucsu.colorado.nospam.edu> wrote

> I need to operate on large tab delimited ascii files that contain a very
> wide mix of int, flt, string etc

Can you elaborate on the format a little? When you say a wide mix, does
that mean that any field could contain any data type or is there at least
consistency within each file? Do the string fields contain spaces?

I have dealt with this by reading each line in the file as a string, parsing
it with STRSPLIT and dealing with the parts but even this approach has
limits.

-Rick

Report message to a moderator

Re: Read/format mixed int/flt/strng ascii files [message #31801 is a reply to message #31800]

Mon, 19 August 2002 14:01

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

david (knapp@ucsu.colorado.nospam.edu) writes:

> Does anyone know of a user defined procedure that already does this with
> more automation and less
> hand-coding of the final structure?
>
> Any tips appreciated :)

Get one of your graduate students to do this.
It is always drudgery and takes a long time,
especially if you have strings embedded in your
lines. Someone of your stature shouldn't have to
be doing this!

Cheers,

David

P.S. You could hire it out to an expert, but
they would charge you an arm and a leg because
it is drudgery for them, too. :-)

--
David W. Fanning, Ph.D.
Fanning Software Consulting, Inc.
Phone: 970-221-0438, E-mail: david@dfanning.com
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Toll-Free IDL Book Orders: 1-888-461-0155

Report message to a moderator

Re: Read/format mixed int/flt/strng ascii files [message #31853 is a reply to message #31787]

Wed, 21 August 2002 09:08

Pavel A. Romashkin
Messages: 531
Registered: November 2000

Senior Member

I am afraid it is laborious to do this. Although it is indeed very
simple. Number of lines does not matter, it is the varying field types
that do. If you had the same field types it would be very simple but
varying the type will require line-by-line reading, which is slow.
What's the source of the files? Is there a chance they can be organized
better to start with?
Like others said, give us an example, and the worst case example if possible.
Good luck,
Pavel

Report message to a moderator

Previous Topic:	Obtaining image of screen
Next Topic:	Reading Wav files

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:39:24 PDT 2025

Total time taken to generate the page: 0.00421 seconds