comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: Re: Reading and Plotting big txt. File

Re: Reading and Plotting big txt. File
type: item
description: On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > I'm trying to read and plot (surface) a very big text (.txt) file > (1020, 1024) with a 5 line string Header in IDL. My file looks like a > circle made of numbers!!!. That means in some lines and colums there > are no numbers only blanks!!!for example my file contains integers > between rows 633 and 390 and between columns 650 and 406.At the left > side of the file, there are the numbers of rows (1023,1022,1021,....0) > my code should not read, but it does. And I also notice, that my code > don't begin to read where the data starts!!By running the code I have > the following error message: READF: End of file encountered. Unit: 1. > Can someone help me? > This is how my code looks like > pro readfile, filename > > ; file=strupcase(filename) > rows=file_lines(file) > ;open the file and read the five line header. > openr,1,file > header=strarr(5) > readf,1,header > ; Find the number of columns in the file > cols=fix(strmid(header(3),14,4)) > ; Number of rows of the data > rows_data=rows-n_elements(header) > > ;Create a big array to hold the data > data=intarr (cols, rows_data) > ; All blanks should be replaced by zero > data[where(data eq ' ')]=0 > ; A small array to read a line > s=intarr(cols) > n=0 > while (~ eof(1) and (n lt rows_data -1 )) do begin > ; Read a line of data > readf,1,s > ; Store it in data > data[*,n]=s > n=n+1 > end > data=data[*,0:n-1] > > CLOSE,1 > Shade_surf, data > end > > thanks > > incognito I'm suspicious of the line converting blanks to zeros before you've even read them. I don't think the blanks will come out the way you're expecting, anyway. I'd suggest you write a program to correctly read your first line of data before you go for the whole thing. Greg
creator: greg.addr
date: 2007-08-01T10:25:59-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > > > >> I'm trying to read and plot (surface) a very big text (.txt) file >> (1020, 1024) with a 5 line string Header in IDL. My file looks like a >> circle made of numbers!!!. That means in some lines and colums there >> are no numbers only blanks!!!for example my file contains integers >> between rows 633 and 390 and between columns 650 and 406.At the left >> side of the file, there are the numbers of rows (1023,1022,1021,....0) >> my code should not read, but it does. And I also notice, that my code >> don't begin to read where the data starts!!By running the code I have >> the following error message: READF: End of file encountered. Unit: 1. >> Can someone help me? >> This is how my code looks like >> pro readfile, filename > >> ; file=strupcase(filename) >> rows=file_lines(file) >> ;open the file and read the five line header. >> openr,1,file >> header=strarr(5) >> readf,1,header >> ; Find the number of columns in the file >> cols=fix(strmid(header(3),14,4)) >> ; Number of rows of the data >> rows_data=rows-n_elements(header) > >> ;Create a big array to hold the data >> data=intarr (cols, rows_data) >> ; All blanks should be replaced by zero >> data[where(data eq ' ')]=0 >> ; A small array to read a line >> s=intarr(cols) >> n=0 >> while (~ eof(1) and (n lt rows_data -1 )) do begin >> ; Read a line of data >> readf,1,s >> ; Store it in data >> data[*,n]=s >> n=n+1 >> end >> data=data[*,0:n-1] > >> CLOSE,1 >> Shade_surf, data >> end > >> thanks > >> incognito > > I'm suspicious of the line converting blanks to zeros before you've > even read them. I don't think the blanks will come out the way you're > expecting, anyway. I'd suggest you write a program to correctly read > your first line of data before you go for the whole thing. > > Greg For starters, I'm not sure why you are converting blanks to zeroes there at all. As far as I can tell, you ha…
creator: Conor
date: 2007-08-01T12:44:44-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 1, 5:33 am, "incognito.me" <incognito...@gmx.de> wrote: > I'm trying to read and plot (surface) a very big text (.txt) file > (1020, 1024) with a 5 line string Header in IDL. My file looks like a > circle made of numbers!!!. That means in some lines and colums there > are no numbers only blanks!!!for example my file contains integers > between rows 633 and 390 and between columns 650 and 406.At the left > side of the file, there are the numbers of rows (1023,1022,1021,....0) > my code should not read, but it does. And I also notice, that my code > don't begin to read where the data starts!!By running the code I have > the following error message: READF: End of file encountered. Unit: 1. > Can someone help me? > This is how my code looks like > pro readfile, filename > > ; file=strupcase(filename) > rows=file_lines(file) > ;open the file and read the five line header. > openr,1,file > header=strarr(5) > readf,1,header > ; Find the number of columns in the file > cols=fix(strmid(header(3),14,4)) > ; Number of rows of the data > rows_data=rows-n_elements(header) > > ;Create a big array to hold the data > data=intarr (cols, rows_data) > ; All blanks should be replaced by zero > data[where(data eq ' ')]=0 > ; A small array to read a line > s=intarr(cols) > n=0 > while (~ eof(1) and (n lt rows_data -1 )) do begin > ; Read a line of data > readf,1,s > ; Store it in data > data[*,n]=s > n=n+1 > end > data=data[*,0:n-1] > > CLOSE,1 > Shade_surf, data > end > > thanks > > incognito Really, I would second Peter's suggestion. You should find some way to pre-process the file, specifically, so that there is the same number of columns in each row. If you replace all the blank columns with zero columns, then IDL will no longer have trouble reading your file. I assume that is what you were trying to do with the line 'data[where(data eq ' ')]=0', except that you hadn't read any data yet (and it wouldn't have worked anyway). For instance, if you had: 24 85 36 42 32 16 and you r…
creator: Conor
date: 2007-08-01T12:58:21-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > > > > > >> On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>> I'm trying to read and plot (surface) a very big text (.txt) file >>> (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>> circle made of numbers!!!. That means in some lines and colums there >>> are no numbers only blanks!!!for example my file contains integers >>> between rows 633 and 390 and between columns 650 and 406.At the left >>> side of the file, there are the numbers of rows (1023,1022,1021,....0) >>> my code should not read, but it does. And I also notice, that my code >>> don't begin to read where the data starts!!By running the code I have >>> the following error message: READF: End of file encountered. Unit: 1. >>> Can someone help me? >>> This is how my code looks like >>> pro readfile, filename > >>> ; file=strupcase(filename) >>> rows=file_lines(file) >>> ;open the file and read the five line header. >>> openr,1,file >>> header=strarr(5) >>> readf,1,header >>> ; Find the number of columns in the file >>> cols=fix(strmid(header(3),14,4)) >>> ; Number of rows of the data >>> rows_data=rows-n_elements(header) > >>> ;Create a big array to hold the data >>> data=intarr (cols, rows_data) >>> ; All blanks should be replaced by zero >>> data[where(data eq ' ')]=0 >>> ; A small array to read a line >>> s=intarr(cols) >>> n=0 >>> while (~ eof(1) and (n lt rows_data -1 )) do begin >>> ; Read a line of data >>> readf,1,s >>> ; Store it in data >>> data[*,n]=s >>> n=n+1 >>> end >>> data=data[*,0:n-1] > >>> CLOSE,1 >>> Shade_surf, data >>> end > >>> thanks > >>> incognito > >> I'm suspicious of the line converting blanks to zeros before you've >> even read them. I don't think the blanks will come out the way you're >> expecting, anyway. I'd suggest you write a program to correctly read >> your first line of data before you go for the whole thing. > >> Greg >…
creator: incognito.me
date: 2007-08-01T14:49:48-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 1, 10:49 am, "incognito.me" <incognito...@gmx.de> wrote: > On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > > > >> On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > >>> On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> I'm trying to read and plot (surface) a very big text (.txt) file >>>> (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>>> circle made of numbers!!!. That means in some lines and colums there >>>> are no numbers only blanks!!!for example my file contains integers >>>> between rows 633 and 390 and between columns 650 and 406.At the left >>>> side of the file, there are the numbers of rows (1023,1022,1021,....0) >>>> my code should not read, but it does. And I also notice, that my code >>>> don't begin to read where the data starts!!By running the code I have >>>> the following error message: READF: End of file encountered. Unit: 1. >>>> Can someone help me? >>>> This is how my code looks like >>>> pro readfile, filename > >>>> ; file=strupcase(filename) >>>> rows=file_lines(file) >>>> ;open the file and read the five line header. >>>> openr,1,file >>>> header=strarr(5) >>>> readf,1,header >>>> ; Find the number of columns in the file >>>> cols=fix(strmid(header(3),14,4)) >>>> ; Number of rows of the data >>>> rows_data=rows-n_elements(header) > >>>> ;Create a big array to hold the data >>>> data=intarr (cols, rows_data) >>>> ; All blanks should be replaced by zero >>>> data[where(data eq ' ')]=0 >>>> ; A small array to read a line >>>> s=intarr(cols) >>>> n=0 >>>> while (~ eof(1) and (n lt rows_data -1 )) do begin >>>> ; Read a line of data >>>> readf,1,s >>>> ; Store it in data >>>> data[*,n]=s >>>> n=n+1 >>>> end >>>> data=data[*,0:n-1] > >>>> CLOSE,1 >>>> Shade_surf, data >>>> end > >>>> thanks > >>>> incognito > >>> I'm suspicious of the line converting blanks to zeros before you've >>> even read them. I don't think the blanks will come out the way you're >>> expecting, anyway. I'd suggest …
creator: Conor
date: 2007-08-01T16:15:22-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 1 Aug., 18:15, Conor <cmanc...@gmail.com> wrote: > On Aug 1, 10:49 am, "incognito.me" <incognito...@gmx.de> wrote: > > > > > >> On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > >>> On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > >>>> On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> > I'm trying to read and plot (surface) a very big text (.txt) file >>>> > (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>>> > circle made of numbers!!!. That means in some lines and colums there >>>> > are no numbers only blanks!!!for example my file contains integers >>>> > between rows 633 and 390 and between columns 650 and 406.At the left >>>> > side of the file, there are the numbers of rows (1023,1022,1021,....0) >>>> > my code should not read, but it does. And I also notice, that my code >>>> > don't begin to read where the data starts!!By running the code I have >>>> > the following error message: READF: End of file encountered. Unit: 1. >>>> > Can someone help me? >>>> > This is how my code looks like >>>> > pro readfile, filename > >>>> > ; file=strupcase(filename) >>>> > rows=file_lines(file) >>>> > ;open the file and read the five line header. >>>> > openr,1,file >>>> > header=strarr(5) >>>> > readf,1,header >>>> > ; Find the number of columns in the file >>>> > cols=fix(strmid(header(3),14,4)) >>>> > ; Number of rows of the data >>>> > rows_data=rows-n_elements(header) > >>>> > ;Create a big array to hold the data >>>> > data=intarr (cols, rows_data) >>>> > ; All blanks should be replaced by zero >>>> > data[where(data eq ' ')]=0 >>>> > ; A small array to read a line >>>> > s=intarr(cols) >>>> > n=0 >>>> > while (~ eof(1) and (n lt rows_data -1 )) do begin >>>> > ; Read a line of data >>>> > readf,1,s >>>> > ; Store it in data >>>> > data[*,n]=s >>>> > n=n+1 >>>> > end >>>> > data=data[*,0:n-1] > >>>> > CLOSE,1 >>>> > Shade_surf, data >>>> > end > >>>> > thanks > >>>> > incognito > >>>> I'm suspicious of the line co…
creator: incognito.me
date: 2007-08-01T16:31:19-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 1, 12:31 pm, "incognito.me" <incognito...@gmx.de> wrote: > On 1 Aug., 18:15, Conor <cmanc...@gmail.com> wrote: > > > >> On Aug 1, 10:49 am, "incognito.me" <incognito...@gmx.de> wrote: > >>> On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > >>>> On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > >>>> > On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> > > I'm trying to read and plot (surface) a very big text (.txt) file >>>> > > (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>>> > > circle made of numbers!!!. That means in some lines and colums there >>>> > > are no numbers only blanks!!!for example my file contains integers >>>> > > between rows 633 and 390 and between columns 650 and 406.At the left >>>> > > side of the file, there are the numbers of rows (1023,1022,1021,....0) >>>> > > my code should not read, but it does. And I also notice, that my code >>>> > > don't begin to read where the data starts!!By running the code I have >>>> > > the following error message: READF: End of file encountered. Unit: 1. >>>> > > Can someone help me? >>>> > > This is how my code looks like >>>> > > pro readfile, filename > >>>> > > ; file=strupcase(filename) >>>> > > rows=file_lines(file) >>>> > > ;open the file and read the five line header. >>>> > > openr,1,file >>>> > > header=strarr(5) >>>> > > readf,1,header >>>> > > ; Find the number of columns in the file >>>> > > cols=fix(strmid(header(3),14,4)) >>>> > > ; Number of rows of the data >>>> > > rows_data=rows-n_elements(header) > >>>> > > ;Create a big array to hold the data >>>> > > data=intarr (cols, rows_data) >>>> > > ; All blanks should be replaced by zero >>>> > > data[where(data eq ' ')]=0 >>>> > > ; A small array to read a line >>>> > > s=intarr(cols) >>>> > > n=0 >>>> > > while (~ eof(1) and (n lt rows_data -1 )) do begin >>>> > > ; Read a line of data >>>> > > readf,1,s >>>> > > ; Store it in data >>>> > > data[*,n]=s >>>> > > n=n+1 >>>> > > end >>>> > >…
creator: Conor
date: 2007-08-01T16:43:09-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 1 Aug., 18:15, Conor <cmanc...@gmail.com> wrote: > On Aug 1, 10:49 am, "incognito.me" <incognito...@gmx.de> wrote: > > > > > >> On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > >>> On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > >>>> On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> > I'm trying to read and plot (surface) a very big text (.txt) file >>>> > (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>>> > circle made of numbers!!!. That means in some lines and colums there >>>> > are no numbers only blanks!!!for example my file contains integers >>>> > between rows 633 and 390 and between columns 650 and 406.At the left >>>> > side of the file, there are the numbers of rows (1023,1022,1021,....0) >>>> > my code should not read, but it does. And I also notice, that my code >>>> > don't begin to read where the data starts!!By running the code I have >>>> > the following error message: READF: End of file encountered. Unit: 1. >>>> > Can someone help me? >>>> > This is how my code looks like >>>> > pro readfile, filename > >>>> > ; file=strupcase(filename) >>>> > rows=file_lines(file) >>>> > ;open the file and read the five line header. >>>> > openr,1,file >>>> > header=strarr(5) >>>> > readf,1,header >>>> > ; Find the number of columns in the file >>>> > cols=fix(strmid(header(3),14,4)) >>>> > ; Number of rows of the data >>>> > rows_data=rows-n_elements(header) > >>>> > ;Create a big array to hold the data >>>> > data=intarr (cols, rows_data) >>>> > ; All blanks should be replaced by zero >>>> > data[where(data eq ' ')]=0 >>>> > ; A small array to read a line >>>> > s=intarr(cols) >>>> > n=0 >>>> > while (~ eof(1) and (n lt rows_data -1 )) do begin >>>> > ; Read a line of data >>>> > readf,1,s >>>> > ; Store it in data >>>> > data[*,n]=s >>>> > n=n+1 >>>> > end >>>> > data=data[*,0:n-1] > >>>> > CLOSE,1 >>>> > Shade_surf, data >>>> > end > >>>> > thanks > >>>> > incognito > >>>> I'm suspicious of the line co…
creator: incognito.me
date: 2007-08-02T08:55:42-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 2, 4:55 am, "incognito.me" <incognito...@gmx.de> wrote: > On 1 Aug., 18:15, Conor <cmanc...@gmail.com> wrote: > > > >> On Aug 1, 10:49 am, "incognito.me" <incognito...@gmx.de> wrote: > >>> On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > >>>> On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > >>>> > On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> > > I'm trying to read and plot (surface) a very big text (.txt) file >>>> > > (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>>> > > circle made of numbers!!!. That means in some lines and colums there >>>> > > are no numbers only blanks!!!for example my file contains integers >>>> > > between rows 633 and 390 and between columns 650 and 406.At the left >>>> > > side of the file, there are the numbers of rows (1023,1022,1021,....0) >>>> > > my code should not read, but it does. And I also notice, that my code >>>> > > don't begin to read where the data starts!!By running the code I have >>>> > > the following error message: READF: End of file encountered. Unit: 1. >>>> > > Can someone help me? >>>> > > This is how my code looks like >>>> > > pro readfile, filename > >>>> > > ; file=strupcase(filename) >>>> > > rows=file_lines(file) >>>> > > ;open the file and read the five line header. >>>> > > openr,1,file >>>> > > header=strarr(5) >>>> > > readf,1,header >>>> > > ; Find the number of columns in the file >>>> > > cols=fix(strmid(header(3),14,4)) >>>> > > ; Number of rows of the data >>>> > > rows_data=rows-n_elements(header) > >>>> > > ;Create a big array to hold the data >>>> > > data=intarr (cols, rows_data) >>>> > > ; All blanks should be replaced by zero >>>> > > data[where(data eq ' ')]=0 >>>> > > ; A small array to read a line >>>> > > s=intarr(cols) >>>> > > n=0 >>>> > > while (~ eof(1) and (n lt rows_data -1 )) do begin >>>> > > ; Read a line of data >>>> > > readf,1,s >>>> > > ; Store it in data >>>> > > data[*,n]=s >>>> > > n=n+1 >>>> > > end >>>> > > …
creator: Conor
date: 2007-08-02T12:55:03-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 2 Aug., 14:55, Conor <cmanc...@gmail.com> wrote: > On Aug 2, 4:55 am, "incognito.me" <incognito...@gmx.de> wrote: > > > > > >> On 1 Aug., 18:15, Conor <cmanc...@gmail.com> wrote: > >>> On Aug 1, 10:49 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> On 1 Aug., 14:44, Conor <cmanc...@gmail.com> wrote: > >>>> > On Aug 1, 6:25 am, greg.a...@googlemail.com wrote: > >>>> > > On Aug 1, 11:33 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> > > > I'm trying to read and plot (surface) a very big text (.txt) file >>>> > > > (1020, 1024) with a 5 line string Header in IDL. My file looks like a >>>> > > > circle made of numbers!!!. That means in some lines and colums there >>>> > > > are no numbers only blanks!!!for example my file contains integers >>>> > > > between rows 633 and 390 and between columns 650 and 406.At the left >>>> > > > side of the file, there are the numbers of rows (1023,1022,1021,....0) >>>> > > > my code should not read, but it does. And I also notice, that my code >>>> > > > don't begin to read where the data starts!!By running the code I have >>>> > > > the following error message: READF: End of file encountered. Unit: 1. >>>> > > > Can someone help me? >>>> > > > This is how my code looks like >>>> > > > pro readfile, filename > >>>> > > > ; file=strupcase(filename) >>>> > > > rows=file_lines(file) >>>> > > > ;open the file and read the five line header. >>>> > > > openr,1,file >>>> > > > header=strarr(5) >>>> > > > readf,1,header >>>> > > > ; Find the number of columns in the file >>>> > > > cols=fix(strmid(header(3),14,4)) >>>> > > > ; Number of rows of the data >>>> > > > rows_data=rows-n_elements(header) > >>>> > > > ;Create a big array to hold the data >>>> > > > data=intarr (cols, rows_data) >>>> > > > ; All blanks should be replaced by zero >>>> > > > data[where(data eq ' ')]=0 >>>> > > > ; A small array to read a line >>>> > > > s=intarr(cols) >>>> > > > n=0 >>>> > > > while (~ eof(1) and (n lt rows_data -1 )) do begin >>>> > > …
creator: incognito.me
date: 2007-08-02T15:18:58-00:00

Re: Reading and Plotting big txt. File
type: item
description: The problem is your format statement. What's going on is that with a format, IDL doesn't actually read columns. It is more of directions where to find the data. In your case, you aren't telling it where the spaces are, so it assumes that everything is a data column. If you specify 10(a4), it is really reading: aaaabbbbccccddddeeeffffgggghhhhiiiijjjj where aaaa = column1, bbbb = column2, etc... You need to give it the appropriate number of spaces, otherwise the data get's all messed up. For example, apply the above "filter" to the data below (from your file) 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 -1446 -1372 -1322 The first four columns ' 7 ' are assigned to the first column in your data array. The second four columns ' ' go to the second column in your data array, etc.. In the end you get: data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] (or something along those lines, anyway) What you need to do is actually specify where the spaces are: format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' I don't think that's quite it, but it probably needs to be something along those lines. I can't quite get it to work myself, unfortunately. I wish someone better informed about formats would join in the conversation here...
creator: Conor
date: 2007-08-02T16:55:18-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 2, 12:55 pm, Conor <cmanc...@gmail.com> wrote: > The problem is your format statement. What's going on is that with a > format, IDL doesn't actually read columns. It is more of directions > where to find the data. In your case, you aren't telling it where the > spaces are, so it assumes that everything is a data column. If you > specify 10(a4), it is really reading: > > aaaabbbbccccddddeeeffffgggghhhhiiiijjjj > > where aaaa = column1, bbbb = column2, etc... > > You need to give it the appropriate number of spaces, otherwise the > data get's all messed up. For example, apply the above "filter" to > the data below (from your file) > > 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 > -1446 -1372 -1322 > > The first four columns ' 7 ' are assigned to the first column in your > data array. The second four columns ' ' go to the second column in > your data array, etc.. In the end you get: > > data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] > > (or something along those lines, anyway) > > What you need to do is actually specify where the spaces are: > > format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' > > I don't think that's quite it, but it probably needs to be something > along those lines. I can't quite get it to work myself, > unfortunately. I wish someone better informed about formats would > join in the conversation here... Okay, here's a solution. I didn't want to have to go here, because it is possibly the worst way to solve this problem, but since I can't figure out the formats and no one else has any suggestions, we'll just do it the "bad" way. It's bad because it is not a general solution (this will only work this one sort of file), it's worse because it is really slow, and it is even worse because neither of us is going to figure out what is wrong with what we've been trying. Oh well. The plan is to manually parse the file. Rather than relying on format statements, I wrote a program that reads the file in line by line and parses it according t…
creator: Conor
date: 2007-08-02T17:27:05-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 2 Aug., 19:27, Conor <cmanc...@gmail.com> wrote: > On Aug 2, 12:55 pm, Conor <cmanc...@gmail.com> wrote: > > > > > >> The problem is your format statement. What's going on is that with a >> format, IDL doesn't actually read columns. It is more of directions >> where to find the data. In your case, you aren't telling it where the >> spaces are, so it assumes that everything is a data column. If you >> specify 10(a4), it is really reading: > >> aaaabbbbccccddddeeeffffgggghhhhiiiijjjj > >> where aaaa = column1, bbbb = column2, etc... > >> You need to give it the appropriate number of spaces, otherwise the >> data get's all messed up. For example, apply the above "filter" to >> the data below (from your file) > >> 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 >> -1446 -1372 -1322 > >> The first four columns ' 7 ' are assigned to the first column in your >> data array. The second four columns ' ' go to the second column in >> your data array, etc.. In the end you get: > >> data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] > >> (or something along those lines, anyway) > >> What you need to do is actually specify where the spaces are: > >> format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' > >> I don't think that's quite it, but it probably needs to be something >> along those lines. I can't quite get it to work myself, >> unfortunately. I wish someone better informed about formats would >> join in the conversation here... > > Okay, here's a solution. I didn't want to have to go here, because it > is possibly the worst way to solve this problem, but since I can't > figure out the formats and no one else has any suggestions, we'll just > do it the "bad" way. It's bad because it is not a general solution > (this will only work this one sort of file), it's worse because it is > really slow, and it is even worse because neither of us is going to > figure out what is wrong with what we've been trying. Oh well. The > plan is to manually parse the file. Rather than rely…
creator: incognito.me
date: 2007-08-03T11:43:15-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 3, 7:43 am, "incognito.me" <incognito...@gmx.de> wrote: > On 2 Aug., 19:27, Conor <cmanc...@gmail.com> wrote: > > > >> On Aug 2, 12:55 pm, Conor <cmanc...@gmail.com> wrote: > >>> The problem is your format statement. What's going on is that with a >>> format, IDL doesn't actually read columns. It is more of directions >>> where to find the data. In your case, you aren't telling it where the >>> spaces are, so it assumes that everything is a data column. If you >>> specify 10(a4), it is really reading: > >>> aaaabbbbccccddddeeeffffgggghhhhiiiijjjj > >>> where aaaa = column1, bbbb = column2, etc... > >>> You need to give it the appropriate number of spaces, otherwise the >>> data get's all messed up. For example, apply the above "filter" to >>> the data below (from your file) > >>> 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 >>> -1446 -1372 -1322 > >>> The first four columns ' 7 ' are assigned to the first column in your >>> data array. The second four columns ' ' go to the second column in >>> your data array, etc.. In the end you get: > >>> data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] > >>> (or something along those lines, anyway) > >>> What you need to do is actually specify where the spaces are: > >>> format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' > >>> I don't think that's quite it, but it probably needs to be something >>> along those lines. I can't quite get it to work myself, >>> unfortunately. I wish someone better informed about formats would >>> join in the conversation here... > >> Okay, here's a solution. I didn't want to have to go here, because it >> is possibly the worst way to solve this problem, but since I can't >> figure out the formats and no one else has any suggestions, we'll just >> do it the "bad" way. It's bad because it is not a general solution >> (this will only work this one sort of file), it's worse because it is >> really slow, and it is even worse because neither of us is going to >> figure out what is wrong wi…
creator: Conor
date: 2007-08-03T12:31:18-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 3 Aug., 14:31, Conor <cmanc...@gmail.com> wrote: > On Aug 3, 7:43 am, "incognito.me" <incognito...@gmx.de> wrote: > > > > > >> On 2 Aug., 19:27, Conor <cmanc...@gmail.com> wrote: > >>> On Aug 2, 12:55 pm, Conor <cmanc...@gmail.com> wrote: > >>>> The problem is your format statement. What's going on is that with a >>>> format, IDL doesn't actually read columns. It is more of directions >>>> where to find the data. In your case, you aren't telling it where the >>>> spaces are, so it assumes that everything is a data column. If you >>>> specify 10(a4), it is really reading: > >>>> aaaabbbbccccddddeeeffffgggghhhhiiiijjjj > >>>> where aaaa = column1, bbbb = column2, etc... > >>>> You need to give it the appropriate number of spaces, otherwise the >>>> data get's all messed up. For example, apply the above "filter" to >>>> the data below (from your file) > >>>> 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 >>>> -1446 -1372 -1322 > >>>> The first four columns ' 7 ' are assigned to the first column in your >>>> data array. The second four columns ' ' go to the second column in >>>> your data array, etc.. In the end you get: > >>>> data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] > >>>> (or something along those lines, anyway) > >>>> What you need to do is actually specify where the spaces are: > >>>> format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' > >>>> I don't think that's quite it, but it probably needs to be something >>>> along those lines. I can't quite get it to work myself, >>>> unfortunately. I wish someone better informed about formats would >>>> join in the conversation here... > >>> Okay, here's a solution. I didn't want to have to go here, because it >>> is possibly the worst way to solve this problem, but since I can't >>> figure out the formats and no one else has any suggestions, we'll just >>> do it the "bad" way. It's bad because it is not a general solution >>> (this will only work this one sort of file), it's worse because it is >>> really …
creator: incognito.me
date: 2007-08-03T14:15:51-00:00

Re: Reading and Plotting big txt. File
type: item
description: On Aug 3, 10:15 am, "incognito.me" <incognito...@gmx.de> wrote: > On 3 Aug., 14:31, Conor <cmanc...@gmail.com> wrote: > > > >> On Aug 3, 7:43 am, "incognito.me" <incognito...@gmx.de> wrote: > >>> On 2 Aug., 19:27, Conor <cmanc...@gmail.com> wrote: > >>>> On Aug 2, 12:55 pm, Conor <cmanc...@gmail.com> wrote: > >>>> > The problem is your format statement. What's going on is that with a >>>> > format, IDL doesn't actually read columns. It is more of directions >>>> > where to find the data. In your case, you aren't telling it where the >>>> > spaces are, so it assumes that everything is a data column. If you >>>> > specify 10(a4), it is really reading: > >>>> > aaaabbbbccccddddeeeffffgggghhhhiiiijjjj > >>>> > where aaaa = column1, bbbb = column2, etc... > >>>> > You need to give it the appropriate number of spaces, otherwise the >>>> > data get's all messed up. For example, apply the above "filter" to >>>> > the data below (from your file) > >>>> > 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 >>>> > -1446 -1372 -1322 > >>>> > The first four columns ' 7 ' are assigned to the first column in your >>>> > data array. The second four columns ' ' go to the second column in >>>> > your data array, etc.. In the end you get: > >>>> > data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] > >>>> > (or something along those lines, anyway) > >>>> > What you need to do is actually specify where the spaces are: > >>>> > format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' > >>>> > I don't think that's quite it, but it probably needs to be something >>>> > along those lines. I can't quite get it to work myself, >>>> > unfortunately. I wish someone better informed about formats would >>>> > join in the conversation here... > >>>> Okay, here's a solution. I didn't want to have to go here, because it >>>> is possibly the worst way to solve this problem, but since I can't >>>> figure out the formats and no one else has any suggestions, we'll just >>>> do it the "bad" way. It's bad becau…
creator: Conor
date: 2007-08-03T14:20:38-00:00

Re: Reading and Plotting big txt. File
type: item
description: On 3 Aug., 16:20, Conor <cmanc...@gmail.com> wrote: > On Aug 3, 10:15 am, "incognito.me" <incognito...@gmx.de> wrote: > >> On 3 Aug., 14:31, Conor <cmanc...@gmail.com> wrote: > >>> On Aug 3, 7:43 am, "incognito.me" <incognito...@gmx.de> wrote: > >>>> On 2 Aug., 19:27, Conor <cmanc...@gmail.com> wrote: > >>>> > On Aug 2, 12:55 pm, Conor <cmanc...@gmail.com> wrote: > >>>> > > The problem is your format statement. What's going on is that with a >>>> > > format, IDL doesn't actually read columns. It is more of directions >>>> > > where to find the data. In your case, you aren't telling it where the >>>> > > spaces are, so it assumes that everything is a data column. If you >>>> > > specify 10(a4), it is really reading: > >>>> > > aaaabbbbccccddddeeeffffgggghhhhiiiijjjj > >>>> > > where aaaa = column1, bbbb = column2, etc... > >>>> > > You need to give it the appropriate number of spaces, otherwise the >>>> > > data get's all messed up. For example, apply the above "filter" to >>>> > > the data below (from your file) > >>>> > > 7 -1848 -1792 -1718 -1678 -1638 -1576 -1517 >>>> > > -1446 -1372 -1322 > >>>> > > The first four columns ' 7 ' are assigned to the first column in your >>>> > > data array. The second four columns ' ' go to the second column in >>>> > > your data array, etc.. In the end you get: > >>>> > > data = [ 7 ',' ',' -1','848 ',' -17','92 ',' -17','18 ',' -16'] > >>>> > > (or something along those lines, anyway) > >>>> > > What you need to do is actually specify where the spaces are: > >>>> > > format = '(a2, 7x, a4, 2x, a4, 7( 3x, a4 ) )' > >>>> > > I don't think that's quite it, but it probably needs to be something >>>> > > along those lines. I can't quite get it to work myself, >>>> > > unfortunately. I wish someone better informed about formats would >>>> > > join in the conversation here... > >>>> > Okay, here's a solution. I didn't want to have to go here, because it >>>> > is possibly the worst way to solve this problem, but since I can't >>>> > fi…
creator: incognito.me
date: 2007-08-03T15:35:26-00:00