| Read ASCII [message #10591] |
Thu, 08 January 1998 00:00  |
DMottershead
Messages: 17
Registered: January 1998
|
Junior Member |
|
|
I am trying to read in some ASCII data in a column form with a format
defined in my main procedure like this:
fileData = {WBD_File, $
yr:0, $
mn:0, $
day:0, $
hour:0, $
minute:0, $
value:0.0}
info={obj:obj, $
filedata:filedata}
ptr=ptr_new(info,/no_copy)
I am opening and reading the data in my event handler like this:
'open': begin
filename='d:\idl\mhl\syd9712.wbd'
WIDGET_CONTROL, (*ptr).infowbd.fileText, SET_VALUE = filename
x=35136
arr=fltarr(6,x)
hdr=strarr(5)
count=0
;make a the same structure as fileData
a=(*ptr).fileData
data=replicate(a,x)
openr, lun, filename,/get_lun
;read header in
readf,lun,hdr
WHILE NOT EOF(lun) DO BEGIN
READF, lun, a
data(count)=a
count=count+1
endwhile
data=data(0:count-1)
; tell user abour file header
WIDGET_CONTROL, (*ptr).infowbd.fileHeader, SET_VALUE = hdr
; get the data values
My problem exists when I try to "copy" the "data" back to the fileData
structure.
How do I make the fileData structure the same size as the "data". I have
tried to replicate (*ptr).fileData without any success. Also, how do I
copy the values from "data" to "(*ptr).fileData"? In advance, thanks for
your help.
--
Regards
David
************************************************************ *******
David Mottershead Phone: +61 2 9949 0234
Manly Hydraulics Laboratory Fax: +61 2 9948 6185
110b King St, Manly Vale, 2093 email: dmottershead@mhl.nsw.gov.au
SYDNEY, AUSTRALIA WWW: http://www.mhl.nsw.gov.au
************************************************************ *******
|
|
|
|
| Re: Read ASCII [message #10593 is a reply to message #10591] |
Wed, 07 January 1998 00:00  |
davidf
Messages: 2866
Registered: September 1996
|
Senior Member |
|
|
David Mottershead (DMottershead@mhl.nsw.gov.au) writes
from down under:
> I am trying to read in some ASCII data in a column form with a format
> defined in my main procedure like this:
>
> fileData = {WBD_File, $
> yr:0, $
> mn:0, $
> day:0, $
> hour:0, $
> minute:0, $
> value:0.0}
[much code clipped]
I've read this post over quite a few times, and I am still
not *totally* sure I understand what you are trying to do.
It looks to me like you have column data in the form of
6 columns by 35136 rows. Going on that assumption, I would
say you are trying to do too much all at once. Take it a
little bit slower.
I might try it like this. I would make my info structure,
which is where I store *all* of the information I need to
make my program work, with a field for the data template
and another field for the data itself. It might be defined
like this:
fileData = {WBD_File, yr:0, mn:0, day:0, hour:0, $
minute:0, value:0.0}
info = { template:fileData, $ ; The file template.
header:StrArr(5), $ ; Five line data file header.
rows:35136L, $ ; Number of rows in the data files.
data:Ptr_New() } ; A null pointer, for now.
And just from this one code example, I'm guessing this program
could use a little modularity. I would probably define the
Open File button like this, so I could work with a smaller
event handler:
openfileID = Widget_Button(menubase, Value='Open File...', $
Event_Pro='Open_File_Button_Event')
So, then, my event handler might look something like this:
PRO Open_File_Button_Event, event
; Get the data file name from the user.
filename = Dialog_Pickfile(/Read, Filter='*.dat')
IF filename EQ '' THEN RETURN
; Go read the file. First, get the info structure.
Widget_Control, event.top, Get_UValue=info, /No_Copy
; Create an array of structures. Read the data.
dataArray = Replicate(info.template, info.rows)
OpenR, lun, filename, /Get_Lun
Readf, lun, info.header, dataArray
Free_Lun, lun
; Make the data vectors.
yr = dataArray(*).(0)
mn = dataArray(*).(1)
day = dataArray(*).(2)
hour = dataArray(*).(3)
minute = dataArray(*).(4)
value = dataArray(*).(5)
; Store data as structure in the pointer location.
info.data = Ptr_New({yr:yr, mn:mn, day:day, hour:hour, $
minute:minute, value:value}, /No_Copy)
Widget_Control, event.top, Set_UValue=info, /No_Copy
END
Now, any module that needs to do something with the data
can access it like this:
Plot, (*info.data).day, (*info.data).value
Hope that gives you some ideas.
Cheers,
David
P.S. You might also look at the two new functions ASCII_TEMPLATE
and READ_ASCII. They are made specifically to read this kind
of column data.
-----------------------------------------------------------
David Fanning, Ph.D.
Fanning Software Consulting
E-Mail: davidf@dfanning.com
Phone: 970-221-0438
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
|
|
| Re: read ascii [message #63626 is a reply to message #10591] |
Mon, 17 November 2008 04:32 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
julia.walterspiel@gmail.com schrieb:
> hi
>
> i've been fiddling about this problem the whole morning and it seems I
> can't get it done properly:
>
> I'm trying to read in a huge (!) ascii-file ('test.dat') where the
> first column is a string (name of station), followed by 2 columns with
> integer values (second row date in the format yymmddhourminute, third
> row integer numbers between 0 and 10).
>
> example 'test.dat':
> SMA 200001010010 0
> SMA 200001010020 0
> SMA 200001010030 1
> SMA 200001010040 0
> SMA 200001010050 3
> SMA 200001010100 4
> SMA 200001010110 5
> SMA 200001010120 0
> SMA 200001010130 0
> SMA 200001010140 0
>
> no header.
>
> I don't know how many rows I got, but I'm sure it's a LOT, since excel
> crashes when trying to copy the file :)
>
> I'm using David's Code like this:
>
> OPENR, lun, '/filepath/test.dat', / GET_LUN
>
> station = strarr (1000000)
> time = fltarr (1000000)
> sunshine_duration = fltarr(1000000)
>
> s = '???' --> not sure what to insert here
> t = 0.0
> sd = 0.0
> count = 0
>
> WHILE (NOT EOF(lun)) DO BEGIN
> READF, lun, s, t, sd
> station(count) = s
> time (count) = t
> sunshine_duration(count) = sd
> count = count+1
> ENDWHILE
>
> station = station(0:count-1)
> time = time(0:count-1)
> sunshine_duration = sunshine_duration(0:count-1)
>
> FREE_LUN, lun
>
> then I get the error
> "READF: Input conversion error. Unit: 101"
> I tried to google this error but couldnt find any useful
> information...
>
> I figure, I don't understand 100% what I'm doing here, that's why I
> don't see where I make the mistake...
> furthermore, I'm not sure how the file is delimited (tab or white
> space) if this is crucial to anything..?
>
> any help appreciated!
> cheers,
> juls
hi
you can read the file once as byte data file and then do a conversion to
the types you want for each column. That tool can be brought to
perfection if you involve pointers ;)
cheers
Reimar
e.g.
file = 'test.dat'
lines = file_lines(file)
struct = replicate(create_struct('var1', bytarr(5),$
'var2', bytarr(13),$
'var3', bytarr(11)), lines)
openu,lun,file,/get_lun
readu,lun,struct
free_lun,lun
; example usage of the data
print, strtrim(struct.var1, 2)
print, long64(strtrim(struct.var2, 2))
print, long(strtrim(struct.var3, 2))
; or
result = create_struct('var1', strtrim(struct.var1, 2),$
'var2', long64(strtrim(struct.var2, 2)),$
'var3', long(strtrim(struct.var3, 2)))
print, result.var1
end
|
|
|
|
|
|
|
|
| Re: read ascii [message #63754 is a reply to message #10591] |
Mon, 17 November 2008 08:09 |
julia.walterspiel
Messages: 35
Registered: July 2008
|
Member |
|
|
On 17 Nov., 16:43, julia.waltersp...@gmail.com wrote:
> > I see three posts
>
> i didn't see my post after 10min so I figured something went wrong and
> added it again. I am not a very patient person.. :) (which makes
> programming my absolute favourite :))
>
> good, besides the posting-deleting-issue here's another simple
> question which is obviously so simple that I cannot find a decent
> answer on the internet:
>
> I put my 3 arrays (Station, Time, Sunshine_duration) in one structure
> because I'm trying to avoid having to index the sunshine_duration
> array (the data runs from 01 jan 2000 to 31 okt 2008 and I want to be
> able to plot selected months).
>
> Am I right in my assumption, that every value of the
> "sunshine_duration" has automatically assigned the right date ("time")
> when it was measured? Means, if I want to let's say plot data from
> April 2007 i can do some magic like
>
> index = where(structure.time EQ 2007)
>
> and this gives me the corresponding Sunshine_duration-values? (I
> really hope it does, otherwise I would not understand the sense of
> building structures...)
>
> ... last question for today! promise
answer to myself: yes it does...I love structures!
good night and thanks to all of you
|
|
|
|
| Re: read ascii [message #63761 is a reply to message #10591] |
Mon, 17 November 2008 07:43 |
julia.walterspiel
Messages: 35
Registered: July 2008
|
Member |
|
|
> I see three posts
i didn't see my post after 10min so I figured something went wrong and
added it again. I am not a very patient person.. :) (which makes
programming my absolute favourite :))
good, besides the posting-deleting-issue here's another simple
question which is obviously so simple that I cannot find a decent
answer on the internet:
I put my 3 arrays (Station, Time, Sunshine_duration) in one structure
because I'm trying to avoid having to index the sunshine_duration
array (the data runs from 01 jan 2000 to 31 okt 2008 and I want to be
able to plot selected months).
Am I right in my assumption, that every value of the
"sunshine_duration" has automatically assigned the right date ("time")
when it was measured? Means, if I want to let's say plot data from
April 2007 i can do some magic like
index = where(structure.time EQ 2007)
and this gives me the corresponding Sunshine_duration-values? (I
really hope it does, otherwise I would not understand the sense of
building structures...)
... last question for today! promise
|
|
|
|
| Re: read ascii [message #63764 is a reply to message #10591] |
Mon, 17 November 2008 07:18 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Brian Larsen schrieb:
> yikes, bytes, bits, structures, and splitting oh my. That certainly
> works but... yikes
>
yeah
can one make a speed comparision and the other question can format be
used with structures too?
cheers
Reimar
> This can be done with a cleaver call to format and a type conversion
> that is (in my opinion) the best and fastest way. Play with the type
> conversion as you want I didn't include it here.
>
> lines=file_lines('test.dat')
> dat = strarr(3, lines)
> openr, lun, 'test.dat', /get_lun
> readf, lun, dat, format = '(a4,a16,a)'
> free_lun, lun
> IDL> print, dat[0,*]
> SMA
> SMA
> SMA
> SMA
> SMA
> SMA
> SMA
> SMA
> SMA
> SMA
> IDL> print, dat[1,*]
> 200001010010
> 200001010020
> 200001010030
> 200001010040
> 200001010050
> 200001010100
> 200001010110
> 200001010120
> 200001010130
> 200001010140
> IDL> print, dat[2,*]
> 0
> 0
> 1
> 0
> 3
> 4
> 5
> 0
> 0
> 0
>
>
> Cheers,
>
> Brian
>
> ------------------------------------------------------------ --------------
> Brian Larsen
> Boston University
> Center for Space Physics
> http://people.bu.edu/balarsen/Home/IDL
>
>
>
|
|
|
|
| Re: read ascii [message #63765 is a reply to message #10591] |
Mon, 17 November 2008 07:24 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Reimar Bauer writes:
> julia.walterspiel@gmail.com schrieb:
>> yeah sorry David, i got confused with the TOTAL routine.. i realized
>> what bulls*** i had asked and I tried to delete my last post right
>> after I uploaded it but I assume you were quicker :))
>
>
> I see three posts
It's a user interface issue. They need to get that DELETE
button further away from the SEND button. :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: read ascii [message #63766 is a reply to message #10591] |
Mon, 17 November 2008 07:13 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
julia.walterspiel@gmail.com schrieb:
> yeah sorry David, i got confused with the TOTAL routine.. i realized
> what bulls*** i had asked and I tried to delete my last post right
> after I uploaded it but I assume you were quicker :))
I see three posts
Reimar
|
|
|
|
| Re: read ascii [message #63767 is a reply to message #10591] |
Mon, 17 November 2008 07:14 |
Brian Larsen
Messages: 270
Registered: June 2006
|
Senior Member |
|
|
yikes, bytes, bits, structures, and splitting oh my. That certainly
works but... yikes
This can be done with a cleaver call to format and a type conversion
that is (in my opinion) the best and fastest way. Play with the type
conversion as you want I didn't include it here.
lines=file_lines('test.dat')
dat = strarr(3, lines)
openr, lun, 'test.dat', /get_lun
readf, lun, dat, format = '(a4,a16,a)'
free_lun, lun
IDL> print, dat[0,*]
SMA
SMA
SMA
SMA
SMA
SMA
SMA
SMA
SMA
SMA
IDL> print, dat[1,*]
200001010010
200001010020
200001010030
200001010040
200001010050
200001010100
200001010110
200001010120
200001010130
200001010140
IDL> print, dat[2,*]
0
0
1
0
3
4
5
0
0
0
Cheers,
Brian
------------------------------------------------------------ --------------
Brian Larsen
Boston University
Center for Space Physics
http://people.bu.edu/balarsen/Home/IDL
|
|
|
|
| Re: read ascii [message #63768 is a reply to message #10591] |
Mon, 17 November 2008 07:13 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
julia.walterspiel@gmail.com writes:
> yeah sorry David, i got confused with the TOTAL routine.. i realized
> what bulls*** i had asked and I tried to delete my last post right
> after I uploaded it but I assume you were quicker :))
Yeah, at 6:00 AM I'm all over everything. ;-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: read ascii [message #63769 is a reply to message #63622] |
Mon, 17 November 2008 07:11 |
julia.walterspiel
Messages: 35
Registered: July 2008
|
Member |
|
|
yeah sorry David, i got confused with the TOTAL routine.. i realized
what bulls*** i had asked and I tried to delete my last post right
after I uploaded it but I assume you were quicker :))
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
