comp.lang.idl-pvwave archive: archive » Re: Read ASCII File with tab seperator

Home » Public Forums » archive » Re: Read ASCII File with tab seperator

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Read ASCII File with tab seperator [message #75369]

Fri, 04 March 2011 06:16

noahh.schwartz
Messages: 10
Registered: February 2008

Junior Member

Thanks everyone! You've been most helpful!
I can now read my data properly. Yes, it is a bizarre data format, but
what can I do ;-P
Cheers,
Noah

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75372 is a reply to message #75369]

Thu, 03 March 2011 08:03

Mark Piper
Messages: 198
Registered: December 2009

Senior Member

I'm a huge fan of READCOL (along with RDFLOAT and READFMT) because its syntax models the file -- I see columns in the text file, so I list variables which give back arrays containing those columns.

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75382 is a reply to message #75372]

Wed, 02 March 2011 11:09

Paul Van Delst[1]
Messages: 1157
Registered: April 2002

Senior Member

Paulo Penteado wrote:
> On Mar 2, 2:20 pm, David Fanning <n...@idlcoyote.com> wrote:
>> Who in the world thinks up these bizarre data formats!?
>
> I always wondered that. People deliberately trying to make things bad?
>
> Maybe it is a psychology experiment, to measure how much suffering
> others will put up with.

I reckon it's due to three things:
- a general human tendency to (want to) assume symmetry (if it's easy to write the format then it *must* be easy to
read, right?)
- the lack of appreciation for the importance of tacit knowledge (the "works-for-me!" syndrome :o)
- people are time-crunched (writing generic software is, generally, not a quickly accomplished task).

Or it's a psychology experiment.

Whichev.

:o)

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75385 is a reply to message #75382]

Wed, 02 March 2011 10:27

penteado
Messages: 866
Registered: February 2018

Senior Member
Administrator

On Mar 2, 2:20 pm, David Fanning <n...@idlcoyote.com> wrote:
> Who in the world thinks up these bizarre data formats!?

I always wondered that. People deliberately trying to make things bad?

Maybe it is a psychology experiment, to measure how much suffering
others will put up with.

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75386 is a reply to message #75385]

Wed, 02 March 2011 10:19

wlandsman
Messages: 743
Registered: June 2000

Senior Member

Another way to read it is with readcol.pro
( http://idlastro.gsfc.nasa.gov/ftp/pro/misc/readcol.pro ) --Wayne

IDL> readcol,'test.dat',c1,c2,c3,c4,delim=string(9b),/preserve_nu ll
IDL> print,c1,c2,c3,c4
1.00000 2.00000 3.00000 4.00000
0.00000 0.00000 12.0000 10.0000
0.00000 10.0000 12.0000 10.0000
0.00000 10.0000 0.00000 0.00000

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75387 is a reply to message #75386]

Wed, 02 March 2011 10:12

Carsten Lechte
Messages: 124
Registered: August 2006

Senior Member

Hi,

after firmly kicking the person responsible for this abomination ;)
you can try this little sed command on the data to generate non-broken files:
(all needs to go into one line)

sed <asdf.tab -e 's/\t\t/\t0\t/g' -e 's/\t\t/\t0\t/g' -e 's/\t\t/\t0\t/g' -e 's/\t$/\t0/g'

asdf.tab:
1 2 3 4
2 3
3 5 5

result:
1 2 3 4
2 0 0 3
3 5 5 0

chl

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75388 is a reply to message #75387]

Wed, 02 March 2011 09:50

Michael Galloy
Messages: 1114
Registered: April 2006

Senior Member

On 3/2/11 10:30 AM, Michael Galloy wrote:
> On 3/2/11 10:01 AM, naoh wrote:
>> Hi everyone,
>>
>> I'm trying to read an ascii file with tabs as seperator. The catch is
>> that when the value equals zero, it is not actually written into the
>> file. Example 1 gives the file as it should look like if zeros where
>> present. Example 2 gives the actual file ('0' 'TAB' have been replaced
>> by 'TAB' only).
>> I've used the following code for example 1 but can't figure out how to
>> read example 2 without messing the data up.
>> data = fltarr(4,4)
>> readf, lun, data
>>
>> Thank very much for your help!
>>
>> EXAMPLE 1:
>> 1 0 0 0
>> 2 0 10 10
>> 3 0 12 12
>> 4 0 10 10
>>
>> EXAMPLE 2:
>> 1
>> 2 10 10
>> 3 12 12
>> 4 10 10
>
> [507]> d = read_ascii('test.txt', missing_value=0, delimiter=string(9B))
> [508]> help, d
> ** Structure <15b14b8>, 1 tags, length=64, data length=64, refs=1:
> FIELD1 FLOAT Array[4, 4]
> [509]> print, d.field1
> 1.00000 0.00000 0.00000 0.00000
> 2.00000 0.00000 10.0000 10.0000
> 3.00000 0.00000 12.0000 12.0000
> 4.00000 0.00000 10.0000 10.0000
>
> Mike

By the way, here is another (slightly more verbose) way of doing this
that gives a bit more control over data types, etc:

[501]> t = { version: 1., datastart:0L, delimiter:9B, missingvalue:0L,
commentsymbol:'', fieldcount:4L, fieldtypes:lonarr(4) + 3L,
fieldnames:'col' + strtrim(sindgen(4), 2), fieldlocations:lindgen(4),
fieldgroups:lindgen(4) }
[502]> data = read_ascii('test.txt', template=t)
[504]> help, data
** Structure <201e848>, 4 tags, length=64, data length=64, refs=1:
COL0 LONG Array[4]
COL1 LONG Array[4]
COL2 LONG Array[4]
COL3 LONG Array[4]
[505]> print, data.col0
1 2 3 4
[506]> print, data.col1
0 0 0 0
[507]> print, data.col2
0 10 12 10
[508]> print, data.col3
0 10 12 10

You could also read into a single big array:

[515]> t = { version: 1., datastart:0L, delimiter:9B, missingvalue:0L,
commentsymbol:'', fieldcount:4L, fieldtypes:lonarr(4) + 3L,
fieldnames:'col' + strtrim(sindgen(4), 2), fieldlocations:lindgen(4),
fieldgroups:lonarr(4) }
[516]> data = read_ascii('test.txt', template=t)
[517]> help, data
** Structure <1597048>, 1 tags, length=64, data length=64, refs=1:
COL0 LONG Array[4, 4]
[518]> print, data.col0
1 0 0 0
2 0 10 10
3 0 12 12
4 0 10 10

Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75389 is a reply to message #75388]

Wed, 02 March 2011 09:41

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

Michael Galloy writes:

> [507]> d = read_ascii('test.txt', missing_value=0, delimiter=string(9B))
> [508]> help, d
> ** Structure <15b14b8>, 1 tags, length=64, data length=64, refs=1:
> FIELD1 FLOAT Array[4, 4]
> [509]> print, d.field1
> 1.00000 0.00000 0.00000 0.00000
> 2.00000 0.00000 10.0000 10.0000
> 3.00000 0.00000 12.0000 12.0000
> 4.00000 0.00000 10.0000 10.0000

This is the first time in my life I've ever had
cause to think READ_ASCII might actually be useful. ;-)

Cheers,

David

--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75390 is a reply to message #75389]

Wed, 02 March 2011 09:30

Michael Galloy
Messages: 1114
Registered: April 2006

Senior Member

On 3/2/11 10:01 AM, naoh wrote:
> Hi everyone,
>
> I'm trying to read an ascii file with tabs as seperator. The catch is
> that when the value equals zero, it is not actually written into the
> file. Example 1 gives the file as it should look like if zeros where
> present. Example 2 gives the actual file ('0' 'TAB' have been replaced
> by 'TAB' only).
> I've used the following code for example 1 but can't figure out how to
> read example 2 without messing the data up.
> data = fltarr(4,4)
> readf, lun, data
>
> Thank very much for your help!
>
> EXAMPLE 1:
> 1 0 0 0
> 2 0 10 10
> 3 0 12 12
> 4 0 10 10
>
> EXAMPLE 2:
> 1
> 2 10 10
> 3 12 12
> 4 10 10

[507]> d = read_ascii('test.txt', missing_value=0, delimiter=string(9B))
[508]> help, d
** Structure <15b14b8>, 1 tags, length=64, data length=64, refs=1:
FIELD1 FLOAT Array[4, 4]
[509]> print, d.field1
1.00000 0.00000 0.00000 0.00000
2.00000 0.00000 10.0000 10.0000
3.00000 0.00000 12.0000 12.0000
4.00000 0.00000 10.0000 10.0000

Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation

Report message to a moderator

Re: Read ASCII File with tab seperator [message #75391 is a reply to message #75390]

Wed, 02 March 2011 09:20

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

naoh writes:

> I'm trying to read an ascii file with tabs as seperator. The catch is
> that when the value equals zero, it is not actually written into the
> file. Example 1 gives the file as it should look like if zeros where
> present. Example 2 gives the actual file ('0' 'TAB' have been replaced
> by 'TAB' only).
> I've used the following code for example 1 but can't figure out how to
> read example 2 without messing the data up.
> data = fltarr(4,4)
> readf, lun, data
>
> Thank very much for your help!
>
> EXAMPLE 1:
> 1 0 0 0
> 2 0 10 10
> 3 0 12 12
> 4 0 10 10
>
> EXAMPLE 2:
> 1
> 2 10 10
> 3 12 12
> 4 10 10

Who in the world thinks up these bizarre data formats!?
That would be a fun job. :-)

Cheers,

David

P.S. I don't know what to tell you, except that
what you're trying ain't gonna work. :-)

--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

Previous Topic:	Anonymous Math Functions in IDL like Matlab
Next Topic:	Re: IDL Astron Library now uses Coyote Graphics

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 16:00:29 PDT 2025

Total time taken to generate the page: 0.00638 seconds