| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65172] |
Tue, 17 February 2009 08:07 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Spon writes:
> on that note, did you ever figure out how to make IDL 7.* give your
> exactly four spaces for every tab key press?
No, IDL 7 acts just like my youngest son. Totally
unpredictable, with a mind of its own. Sometimes it's
3, sometimes 4, sometimes it lines up with other things
in the file. I have no idea what it is doing. :-(
Cheers,
David
--
David Fanning, Ph.D.
Coyote's Guide to IDL Programming (www.dfanning.com)
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65175 is a reply to message #65172] |
Tue, 17 February 2009 07:26  |
Spon
Messages: 178
Registered: September 2007
|
Senior Member |
|
|
On Feb 16, 8:06 pm, David Fanning <n...@dfanning.com> wrote:
> Spon writes:
>> you need to stick this line:
>> data =3D fltarr(10,nstar,nfiles)
>
>> outside your FOR loop, so you don't keep re-defining it (full of
>> zeroes) at every iteration.
>
> Stefan, it is hard for me to know if the code
> you posted is the actual code you use, but if it
> is, I remind you that many of these kinds of errors
> can be fixed, or at least spotted a hell of a lot
> easier, if the code has a bit of "style" to it.
> In this case, I am talking about indenting the
> contents of loops.
>
> Some people think style is a matter of an anal
> personality run amuck. And I suppose it sometimes is.
> But it is also a way of organizing your thoughts on
> paper (or computer screen, as the case may be), and
> as such it can be an enormous help writing something
> as literal as instructions to a computer.
>
> Just thought I would mention it. :-)
>
> Cheers,
>
> David
>
> P.S. You can find many styles by reading the code
> of any IDL library. In fact, you will sometimes be able
> to date the code by the style in current use. *Which* style
> to choose is almost irrelevant. Any style used consistently
> will work for the purpose.
>
> --
> David Fanning, Ph.D.
> Coyote's Guide to IDL Programming (www.dfanning.com)
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
David,
on that note, did you ever figure out how to make IDL 7.* give your
exactly four spaces for every tab key press?
Cheers,
Chris
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65179 is a reply to message #65175] |
Mon, 16 February 2009 16:13 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
On Feb 16, 8:06 pm, David Fanning <n...@dfanning.com> wrote:
> Spon writes:
>> you need to stick this line:
>> data =3D fltarr(10,nstar,nfiles)
>
>> outside your FOR loop, so you don't keep re-defining it (full of
>> zeroes) at every iteration.
>
> Stefan, it is hard for me to know if the code
> you posted is the actual code you use, but if it
> is, I remind you that many of these kinds of errors
> can be fixed, or at least spotted a hell of a lot
> easier, if the code has a bit of "style" to it.
> In this case, I am talking about indenting the
> contents of loops.
>
> Some people think style is a matter of an anal
> personality run amuck. And I suppose it sometimes is.
> But it is also a way of organizing your thoughts on
> paper (or computer screen, as the case may be), and
> as such it can be an enormous help writing something
> as literal as instructions to a computer.
>
> Just thought I would mention it. :-)
>
> Cheers,
>
> David
>
> P.S. You can find many styles by reading the code
> of any IDL library. In fact, you will sometimes be able
> to date the code by the style in current use. *Which* style
> to choose is almost irrelevant. Any style used consistently
> will work for the purpose.
>
> --
> David Fanning, Ph.D.
> Coyote's Guide to IDL Programming (www.dfanning.com)
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Thanks David, duly noted. I've been trying to improve my formatting
(and making my programs more modular) - indenting the loop contents is
a good tip, I have adapted the code and it's definitely nicer to read.
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65185 is a reply to message #65179] |
Mon, 16 February 2009 12:06 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Spon writes:
> you need to stick this line:
> data =3D fltarr(10,nstar,nfiles)
>
> outside your FOR loop, so you don't keep re-defining it (full of
> zeroes) at every iteration.
Stefan, it is hard for me to know if the code
you posted is the actual code you use, but if it
is, I remind you that many of these kinds of errors
can be fixed, or at least spotted a hell of a lot
easier, if the code has a bit of "style" to it.
In this case, I am talking about indenting the
contents of loops.
Some people think style is a matter of an anal
personality run amuck. And I suppose it sometimes is.
But it is also a way of organizing your thoughts on
paper (or computer screen, as the case may be), and
as such it can be an enormous help writing something
as literal as instructions to a computer.
Just thought I would mention it. :-)
Cheers,
David
P.S. You can find many styles by reading the code
of any IDL library. In fact, you will sometimes be able
to date the code by the style in current use. *Which* style
to choose is almost irrelevant. Any style used consistently
will work for the purpose.
--
David Fanning, Ph.D.
Coyote's Guide to IDL Programming (www.dfanning.com)
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65187 is a reply to message #65185] |
Mon, 16 February 2009 09:22 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
On Feb 16, 4:59 pm, Spon <christoph.b...@gmail.com> wrote:
> On Feb 16, 4:46 pm, polystethylene <polystethyl...@hotmail.com> wrote:
>
>
>
>> I've run into a strange problem with this:
>
>> FOR i=0,nfiles-1 do begin
>> infile = string(filefolder)+string(i+1,format='("/",i3.3,".dump")')
>> OPENR,lun,infile,/GET_LUN
>> rows = FILE_LINES(infile)
>> data = fltarr(10,nstar,nfiles)
>> framearray = fltarr(10,rows)
>> READF,lun,format ='(I2,1x,I2,1x,I2,2x,F0,2x,I0,5F0.3)',framearray
>> close,lun
>> free_lun,lun
>> data[0,0,i] = framearray
>> print,i
>> print,data[*,*,i]
>> ENDFOR
>> print,data
>> RETURN
>> END
>
>> I've stuck a print statement within the loop to check that each 2-d
>> array is being written to the 3-d array with each pass of the loop,
>> and it seemed to be working fine.
>> However, when I came to pass the variable 'data' out to another
>> procedure, I noticed it was (mainly) full of zeros. A quick print
>> statement outside the loop within the above procedure revealed that I
>> had zeros throughout the entireity of 'data', except for the final
>> value of i.
>
>> I.e. there were only values for data[*,*,124], where 124 is the last
>> value of the loop index.
>
>> Has anyone come across such behaviour before?
>
> Hi Stefan,
>
> you need to stick this line:
> data = fltarr(10,nstar,nfiles)
>
> outside your FOR loop, so you don't keep re-defining it (full of
> zeroes) at every iteration.
>
> Regards,
> Chris
Ahha of course!
Good spot. Thanks guys
I love this place :D
I have a feeling it'll be some time before I'll be able to contribute
anything back to it though =/
|
|
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65190 is a reply to message #65189] |
Mon, 16 February 2009 09:03 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
polystethylene writes:
> I've run into a strange problem with this:
>
> FOR i=0,nfiles-1 do begin
> infile = string(filefolder)+string(i+1,format='("/",i3.3,".dump")')
> OPENR,lun,infile,/GET_LUN
> rows = FILE_LINES(infile)
> data = fltarr(10,nstar,nfiles)
> framearray = fltarr(10,rows)
> READF,lun,format ='(I2,1x,I2,1x,I2,2x,F0,2x,I0,5F0.3)',framearray
> close,lun
> free_lun,lun
> data[0,0,i] = framearray
> print,i
> print,data[*,*,i]
> ENDFOR
> print,data
> RETURN
> END
>
> I've stuck a print statement within the loop to check that each 2-d
> array is being written to the 3-d array with each pass of the loop,
> and it seemed to be working fine.
> However, when I came to pass the variable 'data' out to another
> procedure, I noticed it was (mainly) full of zeros. A quick print
> statement outside the loop within the above procedure revealed that I
> had zeros throughout the entireity of 'data', except for the final
> value of i.
>
> I.e. there were only values for data[*,*,124], where 124 is the last
> value of the loop index.
>
> Has anyone come across such behaviour before?
Humm. Well, now that you mention it, maybe I tried
that array[0,0,I] = something trick before and it didn't
work. I was afraid of this, but I didn't have time (nor
do I today) to test it. :-(
I think you are going to have to explicitly figure out
how big the thing on the right-hand side is, and set
your subscripts on the left-hand side to reflect what you
are trying to store there.
I guess the other choice would be to have a 2D "frame" that you
stuffed each right-hand side into, using the trick above (which
*will* work). Then stuff that "frame" into your 3D array.
Sort of ugly, I guess.
Cheers,
David
--
David Fanning, Ph.D.
Coyote's Guide to IDL Programming (www.dfanning.com)
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65191 is a reply to message #65190] |
Mon, 16 February 2009 08:59 |
Spon
Messages: 178
Registered: September 2007
|
Senior Member |
|
|
On Feb 16, 4:46 pm, polystethylene <polystethyl...@hotmail.com> wrote:
> I've run into a strange problem with this:
>
> FOR i=0,nfiles-1 do begin
> infile = string(filefolder)+string(i+1,format='("/",i3.3,".dump")')
> OPENR,lun,infile,/GET_LUN
> rows = FILE_LINES(infile)
> data = fltarr(10,nstar,nfiles)
> framearray = fltarr(10,rows)
> READF,lun,format ='(I2,1x,I2,1x,I2,2x,F0,2x,I0,5F0.3)',framearray
> close,lun
> free_lun,lun
> data[0,0,i] = framearray
> print,i
> print,data[*,*,i]
> ENDFOR
> print,data
> RETURN
> END
>
> I've stuck a print statement within the loop to check that each 2-d
> array is being written to the 3-d array with each pass of the loop,
> and it seemed to be working fine.
> However, when I came to pass the variable 'data' out to another
> procedure, I noticed it was (mainly) full of zeros. A quick print
> statement outside the loop within the above procedure revealed that I
> had zeros throughout the entireity of 'data', except for the final
> value of i.
>
> I.e. there were only values for data[*,*,124], where 124 is the last
> value of the loop index.
>
> Has anyone come across such behaviour before?
Hi Stefan,
you need to stick this line:
data = fltarr(10,nstar,nfiles)
outside your FOR loop, so you don't keep re-defining it (full of
zeroes) at every iteration.
Regards,
Chris
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65192 is a reply to message #65191] |
Mon, 16 February 2009 08:46 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
I've run into a strange problem with this:
FOR i=0,nfiles-1 do begin
infile = string(filefolder)+string(i+1,format='("/",i3.3,".dump")')
OPENR,lun,infile,/GET_LUN
rows = FILE_LINES(infile)
data = fltarr(10,nstar,nfiles)
framearray = fltarr(10,rows)
READF,lun,format ='(I2,1x,I2,1x,I2,2x,F0,2x,I0,5F0.3)',framearray
close,lun
free_lun,lun
data[0,0,i] = framearray
print,i
print,data[*,*,i]
ENDFOR
print,data
RETURN
END
I've stuck a print statement within the loop to check that each 2-d
array is being written to the 3-d array with each pass of the loop,
and it seemed to be working fine.
However, when I came to pass the variable 'data' out to another
procedure, I noticed it was (mainly) full of zeros. A quick print
statement outside the loop within the above procedure revealed that I
had zeros throughout the entireity of 'data', except for the final
value of i.
I.e. there were only values for data[*,*,124], where 124 is the last
value of the loop index.
Has anyone come across such behaviour before?
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65199 is a reply to message #65192] |
Mon, 16 February 2009 06:48 |
matsu770610
Messages: 6
Registered: August 2008
|
Junior Member |
|
|
> I'm still in the research phase, so I am not sure
> I have a more efficient method yet. I just know that
> when I am noodling around with something now, I am
> as likely to fire up R as I am IDL. Maybe it just
> has a cleaner syntax for noodling. So far, I haven't
> worked with large data sets in R, but when I do, I
> expect it's pass-everything-by-value mentality will
> throw me back into the arms of IDL. :-)
Some time earlier, I was thinking about switching from IDL to R,
because R has a lot of statistics packages that can be used in my
work. However, the memory problem of R when I analize large data send
me home. :-)
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65200 is a reply to message #65199] |
Mon, 16 February 2009 06:17 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
matsu770610@gmail.com writes:
> i am using IDL and R right now, and the way of interaction between
> this two platform is writing the intermediate data to a file or
> reading something from the file. It's so boring and slow. Your
> strategy of supplementing IDl with R is the SPAWN method i think,
> isn't it?
> Hope you give me a more efficient method.
I'm still in the research phase, so I am not sure
I have a more efficient method yet. I just know that
when I am noodling around with something now, I am
as likely to fire up R as I am IDL. Maybe it just
has a cleaner syntax for noodling. So far, I haven't
worked with large data sets in R, but when I do, I
expect it's pass-everything-by-value mentality will
throw me back into the arms of IDL. :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65203 is a reply to message #65200] |
Sun, 15 February 2009 22:34 |
matsu770610
Messages: 6
Registered: August 2008
|
Junior Member |
|
|
On 2月16日, 上午4时04分, David Fanning <n...@dfanning.com> wrote:
> polystethylene writes:
>> p.s. When's the 3rd edition of your book due, is it imminent enough
>> that I should hold out on buying the 2nd edition?
>
> The short answer is "no". You should buy the 2nd edition.
> It has everything you need. :-)
>
> Cheers,
>
> David
>
> P.S. The only change in the 3rd edition will be a new
> way of programming in IDL. That is, by supplementing IDL
> with R, ImageMagick, and proj4. ;-)
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Hi, David,
i am using IDL and R right now, and the way of interaction between
this two platform is writing the intermediate data to a file or
reading something from the file. It's so boring and slow. Your
strategy of supplementing IDl with R is the SPAWN method i think,
isn't it?
Hope you give me a more efficient method. Thanks. :)
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65205 is a reply to message #65203] |
Sun, 15 February 2009 12:11 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
On Feb 15, 8:04 pm, David Fanning <n...@dfanning.com> wrote:
> polystethylene writes:
>> p.s. When's the 3rd edition of your book due, is it imminent enough
>> that I should hold out on buying the 2nd edition?
>
> The short answer is "no". You should buy the 2nd edition.
> It has everything you need. :-)
>
> Cheers,
>
> David
>
> P.S. The only change in the 3rd edition will be a new
> way of programming in IDL. That is, by supplementing IDL
> with R, ImageMagick, and proj4. ;-)
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Ok, will do. Your problem solving and helpfulness on this group is the
best possible advert for your book.
Thanks again
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65206 is a reply to message #65205] |
Sun, 15 February 2009 12:04 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
polystethylene writes:
> p.s. When's the 3rd edition of your book due, is it imminent enough
> that I should hold out on buying the 2nd edition?
The short answer is "no". You should buy the 2nd edition.
It has everything you need. :-)
Cheers,
David
P.S. The only change in the 3rd edition will be a new
way of programming in IDL. That is, by supplementing IDL
with R, ImageMagick, and proj4. ;-)
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65207 is a reply to message #65206] |
Sun, 15 February 2009 11:44 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
On Feb 15, 7:35 pm, polystethylene <polystethyl...@hotmail.com> wrote:
> On Feb 15, 7:05 pm, David Fanning <n...@dfanning.com> wrote:
>
>
>
>> polystethylene writes:
>>> Thanks for the swift response!
>
>>> I've just tried that, and it fails with the message:
>
>>> % Attempt to subscript DATA with I is out of range.
>
>>> a quick print of framearray and data reveal that 'framearray' contains
>>> the values of the 125th file, but 'data' is full of zeros.
>
>>> Not quite sure what it's complaint means...
>
>> Whoops! Didn't notice your loop variables before.
>> IDL is a zero indexed language. You loops will
>> have to go from 0 to n-1, not from 1 to n.
>
>> I think this might be what the error is about.
>> I think I=125 is out of bounds of your last array
>> dimension, which should go from 0 to 124.
>
>> Cheers,
>
>> David
>> --
>> David Fanning, Ph.D.
>> Fanning Software Consulting, Inc.
>> Coyote's Guide to IDL Programming:http://www.dfanning.com/
>> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
>
> Ah yes of course. I know that as well, that's just me being stupid.
> Actually, looking at it, I know why I set the loop from 1 to 125: I
> make use of the loop index to open each file.
>
> If I change the loop counter to 0,124, it tries to open file 000.dump,
> which doesn't exist. Is there anyway to specify i+1 in my second line?
>
> Thanks David
Nevermind, i was stupidly trying to put the +1 on the format specifier
(getting mixed up between i for integer and i the index)
Thanks for all your help David, it appears to be working now.
p.s. When's the 3rd edition of your book due, is it imminent enough
that I should hold out on buying the 2nd edition?
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65208 is a reply to message #65207] |
Sun, 15 February 2009 11:35 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
On Feb 15, 7:05 pm, David Fanning <n...@dfanning.com> wrote:
> polystethylene writes:
>> Thanks for the swift response!
>
>> I've just tried that, and it fails with the message:
>
>> % Attempt to subscript DATA with I is out of range.
>
>> a quick print of framearray and data reveal that 'framearray' contains
>> the values of the 125th file, but 'data' is full of zeros.
>
>> Not quite sure what it's complaint means...
>
> Whoops! Didn't notice your loop variables before.
> IDL is a zero indexed language. You loops will
> have to go from 0 to n-1, not from 1 to n.
>
> I think this might be what the error is about.
> I think I=125 is out of bounds of your last array
> dimension, which should go from 0 to 124.
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Ah yes of course. I know that as well, that's just me being stupid.
Actually, looking at it, I know why I set the loop from 1 to 125: I
make use of the loop index to open each file.
If I change the loop counter to 0,124, it tries to open file 000.dump,
which doesn't exist. Is there anyway to specify i+1 in my second line?
Thanks David
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65209 is a reply to message #65208] |
Sun, 15 February 2009 11:05 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
polystethylene writes:
> Thanks for the swift response!
>
> I've just tried that, and it fails with the message:
>
> % Attempt to subscript DATA with I is out of range.
>
> a quick print of framearray and data reveal that 'framearray' contains
> the values of the 125th file, but 'data' is full of zeros.
>
> Not quite sure what it's complaint means...
Whoops! Didn't notice your loop variables before.
IDL is a zero indexed language. You loops will
have to go from 0 to n-1, not from 1 to n.
I think this might be what the error is about.
I think I=125 is out of bounds of your last array
dimension, which should go from 0 to 124.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Reading multiple ASCII files in as 2d arrays and putting them into a 3d array [message #65210 is a reply to message #65209] |
Sun, 15 February 2009 10:39 |
polystethylene
Messages: 28
Registered: February 2009
|
Junior Member |
|
|
On Feb 15, 6:27 pm, David Fanning <n...@dfanning.com> wrote:
> polystethylene writes:
>> I've tried two approaches. First of all, I've tried reading each file
>> into the array 'framearray', and then subsequently (within the loop)
>> placing the 2-d array (framearray) into the 3-d array (data), with
>> something like this:
>
>> data[*,*,i] = framearray
>
>> but this complains that array is the wrong size. (I assume its
>> possible to put a smaller array into a larger array without IDL
>> complaining - I figured it would fill the rest of the space with
>> zeros?).
>
> Well, sorta. :-)
>
> The * indicates a data range, (eg, 0:sizeOfX-1). Your
> data is, occasionally, smaller than that. And the array
> on the left hand size will have to equal the size of the
> array on the right hand side, or IDL will complain like
> this.
>
> There is a trick, though. If you specified only the
> *first* value of the data range, IDL will try to fit
> your array into the larger "space" on the left hand side.
> Try this:
>
> data[0,0,I] = framearray
>
> I think there is a good chance you will get away with that. :-)
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Hi David,
Thanks for the swift response!
I've just tried that, and it fails with the message:
% Attempt to subscript DATA with I is out of range.
a quick print of framearray and data reveal that 'framearray' contains
the values of the 125th file, but 'data' is full of zeros.
Not quite sure what it's complaint means...
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
