| IDL code to read MSU and HIRS in Level1b [message #40572] |
Wed, 18 August 2004 09:41  |
Yamout
Messages: 8
Registered: August 2004
|
Junior Member |
|
|
Hi there,
Is there any kind human being who can help an illiterate student to read
a few MSU and HIRS data of NOAA 1991. I have got them from
www.saa.noaa.gov and they have some differences with LAC and GAC.
I have ENVI with full version of IDL installed in PC with Windows NT.
Any answer to my mail address would be highly appreciated.
Regards
Hossein
~~~~~~~
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40612 is a reply to message #40572] |
Sun, 22 August 2004 15:00  |
Mark Hadfield
Messages: 783
Registered: May 1995
|
Senior Member |
|
|
Yamout wrote:
> Dear Mark,
>
> thank you for your response.
It was nothing, really.
I have been away (slaving in the olive grove) since midday Friday NZ
time, so I have not had the pleasure of participating in this thread
since then.
> I have not ever touched TOVS data after 1991. It is not my problem to
> read those data. My problem is to read the TOVS data before 1995.
OK, but you posted information about the *difference* between the data
formats before and after 1 Jan 1995, so I naturally assumed this was
relevant in some way.
> I have ordered and downloaded one of the TOVS sensors called MSU and
> tired to read this data as is described in POD User's Guide. After many
> weeks of errors, I discovered that those data are in a "little bit"
> different format. But the problem now is that the differences are not
> the same as the notes from the automatic help mail from SAA.
Ah. So you have written IDL code which should read the data, but it doesn't.
Do you actually know that your routine will read the data as described
in the User's Guide? It might take some effort to verify this, but it
would be nice to be able to start from some code that successfully reads
*something*.
I suggest you get some coffee (or other legal drug of choice), the
User's Guide and whatever "help" info you can, open up your code in the
IDL editor, add some breakpoints, and work your way through the data,
critically examining every number produced by every READ statement.
Look on the bright side: when you've finished you can post your code
here and achieve instant fame and fortune. ... On second thoughts,
forget about the fortune.
--
Mark Hadfield "Ka puwaha te tai nei, Hoea tatou"
m.hadfield@niwa.co.nz
National Institute for Water and Atmospheric Research (NIWA)
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40620 is a reply to message #40572] |
Fri, 20 August 2004 10:24 |
Yamout
Messages: 8
Registered: August 2004
|
Junior Member |
|
|
Dear Stockwell,
thank you very much for your posting.
I am illiterate PC user who has been programing since Comodore 64 with
audio tapes. My first PC code was in FORTRAN in an Olivetti 8084 (I
think so). Since then I have discovered how stupid I am, as I write some
codes or architecture some more stupid excutables (in some cases even
with GUI) in C# or more convenient in C++. IDL with all of its
advantages (or perhaps also disadvantages) is a complete new world for
me. I am trying to understand it since IDL 5.1 and have always discover
more and more about the sense of its use.
It would be a nominal experience for such IDL users like me to discover
more about the IDL society and their publicity.
Hope I have not caused a wasted weekend for you.
Yours
Hossein
~~~~~~~
R.G. Stockwell wrote:
> "Yamout" <hossein@uni-bonn.de> wrote in message news:cg55g6$u3q$2@f1node01.rhrz.uni-bonn.de...
>
>> Dear David,
>>
>> thank you for your reply. It doesn't matter at all. What I have found
>> beyond the worst is the robot in SAA, which compiles the "Help Desk"
>> mails. What a World would be then, if a help seeker only met robots.
>> I hope one day one time i could get the human one.
>> Goo luck for you also :-)
>> Hossein
>> ~~~~
>
>
> I agree, the robots in comp.lang.idl-pvwave are much nicer.
> R. David Fanning is an excellent algorithm.
>
> That reminds me of a story, back in grad school [ERROR: STORY #432134 file not found].
>
>
> Regards,
> R. Stockwell
>
> P.S. I take it that you are still unable to read the data, is that right?
> I'll take a look at it on the weekend and try to whip up a read routine.
>
>
>
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40621 is a reply to message #40572] |
Fri, 20 August 2004 09:58 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
R.G. Stockwell writes:
> The part I am really impressed with is the very low number of
> "crush kill destroy" occurences. R. Stockwell has about a CKD incident
> every 10 spam emails, which has resulted in multiple meatware failures
> here at CoRA.
We programmed in an almost unlimited tolerance for
dumb questions by having the robot learn from his
own experience. But what we can't seem to get right
is the irritation factor when people want to get
something for nothing. It's almost like there is
a short circuit there. We are still investigating...
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40622 is a reply to message #40572] |
Fri, 20 August 2004 09:50 |
R.G. Stockwell
Messages: 363
Registered: July 1999
|
Senior Member |
|
|
"David Fanning" <davidf@dfanning.com> wrote in message news:MPG.1b8fd5be96a430e3989847@news.frii.com...
> R.G. Stockwell writes:
>
>> I agree, the robots in comp.lang.idl-pvwave are much nicer.
>> R. David Fanning is an excellent algorithm.
>
> The sarcasm chip was a real doozy to get working correctly.
> It took months and months! And even then we had a error
> rate that was over 38.4%, which was just barely under the
> the acceptable rate of 40% sanctioned by the Robot Builders
> of American. In the end, we had to put a capacitor in that
> would just make him go brain dead every 10th post or so.
> This gave him enough acceptable posts to keep him under
> the limit.
>
> Cheers,
>
> David
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming: http://www.dfanning.com/
>
The part I am really impressed with is the very low number of
"crush kill destroy" occurences. R. Stockwell has about a CKD incident
every 10 spam emails, which has resulted in multiple meatware failures
here at CoRA.
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40623 is a reply to message #40572] |
Fri, 20 August 2004 09:33 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
R.G. Stockwell writes:
> I agree, the robots in comp.lang.idl-pvwave are much nicer.
> R. David Fanning is an excellent algorithm.
The sarcasm chip was a real doozy to get working correctly.
It took months and months! And even then we had a error
rate that was over 38.4%, which was just barely under the
the acceptable rate of 40% sanctioned by the Robot Builders
of American. In the end, we had to put a capacitor in that
would just make him go brain dead every 10th post or so.
This gave him enough acceptable posts to keep him under
the limit.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40625 is a reply to message #40572] |
Fri, 20 August 2004 09:25 |
R.G. Stockwell
Messages: 363
Registered: July 1999
|
Senior Member |
|
|
"Yamout" <hossein@uni-bonn.de> wrote in message news:cg55g6$u3q$2@f1node01.rhrz.uni-bonn.de...
> Dear David,
>
> thank you for your reply. It doesn't matter at all. What I have found
> beyond the worst is the robot in SAA, which compiles the "Help Desk"
> mails. What a World would be then, if a help seeker only met robots.
> I hope one day one time i could get the human one.
> Goo luck for you also :-)
> Hossein
> ~~~~
I agree, the robots in comp.lang.idl-pvwave are much nicer.
R. David Fanning is an excellent algorithm.
That reminds me of a story, back in grad school [ERROR: STORY #432134 file not found].
Regards,
R. Stockwell
P.S. I take it that you are still unable to read the data, is that right?
I'll take a look at it on the weekend and try to whip up a read routine.
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40627 is a reply to message #40572] |
Fri, 20 August 2004 08:31 |
Yamout
Messages: 8
Registered: August 2004
|
Junior Member |
|
|
Dear David,
thank you for your reply. It doesn't matter at all. What I have found
beyond the worst is the robot in SAA, which compiles the "Help Desk"
mails. What a World would be then, if a help seeker only met robots.
I hope one day one time i could get the human one.
Goo luck for you also :-)
Hossein
~~~~
David Fanning wrote:
> Dear Yamout,
>
> You write:
>
>
>> After many
>> weeks of errors, I discovered that those data are in a "little bit"
>> different format. But the problem now is that the differences are not
>> the same as the notes from the automatic help mail from SAA.
>
>
> I am sorry if I have seemed to belittle your experience.
> That was not my intention. Rather, I wanted to commiserate
> with it. The weeks and weeks of frustration you have
> experienced with satellite data is, in my experience,
> par for the course. I've never known it to be anything
> BUT frustrating. You can count on the directions being wrong.
>
> I assure you there is someone, somewhere who has read
> this data. It is your job to find him. He almost certainly
> does NOT work on the help desk. Probably he is a low-level
> flunky working for somebody on the committee who designed
> this godawful data format. He probably *used* to be somebody,
> but had an innovative idea once and got moved to the closet
> office next to the boiler room for his pains. His name will
> be the fourth or fifth on the list you get from the cousin
> of the guy on the help desk of people that "might be able
> to help, but probably not".
>
> He probably wrote the program that reads the data in some
> obscure language (you hope fervently wasn't COBOL), but
> even that will help you. What you need is a simple file, a
> program that reads the file, and a list of what the data looks
> like. Then you start randomly poking around until things
> start to make sense to you. It usually takes a while.
>
> In the words of Barry Lopez, in my favorite story "Directions",
> from his _Desert Notes_ book, it will be like this:
>
> "You will need three of four days to follow it out. The
> last part will be on foot. Prepare for this. Prepare for
> the impact of nothing. Get on a regimen of tea and biscuits
> and dried fruit. On the third or fourth day, when you are
> ready to quit, you will know you are on your way. When
> your throat is so thick with dust that you cannot breathe
> you will be almost halfway there. When the soles of your
> feet go numb with the burning and you cannot walk you will
> know you have made no wrong turns. When you can no longer
> laugh at all it is only a little further. Push on.
>
> "It will not be as easy as it sounds. ... [But] you will
> always know this: others have made it. The man you gave
> you the map has been here."
>
> Good luck! :-)
>
> Cheers,
>
> David
>
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40628 is a reply to message #40572] |
Fri, 20 August 2004 08:17 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Dear Yamout,
You write:
> After many
> weeks of errors, I discovered that those data are in a "little bit"
> different format. But the problem now is that the differences are not
> the same as the notes from the automatic help mail from SAA.
I am sorry if I have seemed to belittle your experience.
That was not my intention. Rather, I wanted to commiserate
with it. The weeks and weeks of frustration you have
experienced with satellite data is, in my experience,
par for the course. I've never known it to be anything
BUT frustrating. You can count on the directions being wrong.
I assure you there is someone, somewhere who has read
this data. It is your job to find him. He almost certainly
does NOT work on the help desk. Probably he is a low-level
flunky working for somebody on the committee who designed
this godawful data format. He probably *used* to be somebody,
but had an innovative idea once and got moved to the closet
office next to the boiler room for his pains. His name will
be the fourth or fifth on the list you get from the cousin
of the guy on the help desk of people that "might be able
to help, but probably not".
He probably wrote the program that reads the data in some
obscure language (you hope fervently wasn't COBOL), but
even that will help you. What you need is a simple file, a
program that reads the file, and a list of what the data looks
like. Then you start randomly poking around until things
start to make sense to you. It usually takes a while.
In the words of Barry Lopez, in my favorite story "Directions",
from his _Desert Notes_ book, it will be like this:
"You will need three of four days to follow it out. The
last part will be on foot. Prepare for this. Prepare for
the impact of nothing. Get on a regimen of tea and biscuits
and dried fruit. On the third or fourth day, when you are
ready to quit, you will know you are on your way. When
your throat is so thick with dust that you cannot breathe
you will be almost halfway there. When the soles of your
feet go numb with the burning and you cannot walk you will
know you have made no wrong turns. When you can no longer
laugh at all it is only a little further. Push on.
"It will not be as easy as it sounds. ... [But] you will
always know this: others have made it. The man you gave
you the map has been here."
Good luck! :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40629 is a reply to message #40572] |
Fri, 20 August 2004 08:17 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Dear Yamout,
You write:
> After many
> weeks of errors, I discovered that those data are in a "little bit"
> different format. But the problem now is that the differences are not
> the same as the notes from the automatic help mail from SAA.
I am sorry if I have seemed to belittle your experience.
That was not my intention. Rather, I wanted to commiserate
with it. The weeks and weeks of frustration you have
experienced with satellite data is, in my experience,
par for the course. I've never known it to be anything
BUT frustrating. You can count on the directions being wrong.
I assure you there is someone, somewhere who has read
this data. It is your job to find him. He almost certainly
does NOT work on the help desk. Probably he is a low-level
flunky working for somebody on the committee who designed
this godawful data format. He probably *used* to be somebody,
but had an innovative idea once and got moved to the closet
office next to the boiler room for his pains. His name will
be the fourth or fifth on the list you get from the cousin
of the guy on the help desk of people that "might be able
to help, but probably not".
He probably wrote the program that reads the data in some
obscure language (you hope ferventlyy wasn't COBOL), but
even that will help you. What you need is a simple file, a
program that reads the file, and a list of what the data looks
like. Then you start randomly poking around until things
start to make sense to you. It usually takes a while.
In the words of Barry Lopez, in my favorite story "Directions",
from his _Desert Notes_ book, it will be like this:
"You will need three of four days to follow it out. The
last part will be on foot. Prepare for this. Prepare for
the impact of nothing. Get on a regimen of tea and biscuits
and dried fruit. On the third or fourth day, when you are
ready to quit, you will know you are on your way. When
your throat is so thick with dust that you cannot breathe
you will be almost halfway there. When the soles of your
feet go numb with the burning and you cannot walk you will
know you have made no wrong turns. When you can no longer
laugh at all it is only a little further. Push on.
"It will not be as easy as it sounds. ... [But] you will
always know this: others have made it. The man you gave
you the map has been here."
Good luck! :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: IDL code to read MSU and HIRS in Level1b [message #40630 is a reply to message #40572] |
Fri, 20 August 2004 07:44 |
Yamout
Messages: 8
Registered: August 2004
|
Junior Member |
|
|
Dear Makr,
thank you for your response.
I have not ever touched TOVS data after 1991. It is not my problem to
read those data. My problem is to read the TOVS data before 1995.
I have ordered and downloaded one of the TOVS sensors called MSU and
tired to read this data as is described in POD User's Guide. After many
weeks of errors, I discovered that those data are in a "little bit"
different format. But the problem now is that the differences are not
the same as the notes from the automatic help mail from SAA. The
following is the last so called help which could get from the robot there:
---------------------------------
In response to your SAA Help Desk inquiry
E-Mail : Dear Hossein Yamout,
The only difference between the data from before and after January
1, 1995 is the record length. For data from prior to 1995, there were
additional spare bytes at the end of each data record, effectively
increasing the overall record length. In particular, for the line item
you referenced, L5409101, which was for MSU data, you will want to look
at section 4.3 of the POD Guide
http://www2.ncdc.noaa.gov/docs/podug/html/c4/sec4-3.htm
If you look at table 4.3.2.1-1. titled Format of MSU data, you will
see that pre-1995 data had spare bytes in byte locations 401 to 440
while the data starting in 1995 had spare bytes in byte locations 401 to
437. Otherwise, the formats are identical. If you are interested in the
other TOVS data types, they too had a record length change starting in
1995. The appropriate section of the POD Guide spells out the record
length change for these data types as well. For SSU data, see
http://www2.ncdc.noaa.gov/docs/podug/html/c4/sec4-2.htm
and for HIRS data see
http://www2.ncdc.noaa.gov/docs/podug/html/c4/sec4-1.htm
I hope this clarifies this matter for you.
Sincerely,
CLASS Help Desk
------------------------------------
The robot in SAA help desk have not realized that there is not only one
different between the post and past level1b formats of TOVS data, but
there are many others.
By the way, after so many weeks of mailing and now posting, I have
received no one (among them many meteorologists in NOAA) which reply me
with the word "sorry" somewhere in their mail. Apparently there are not
so many people (even high experts in NOAA) who have ever used these
data. The next question which would one make is, why they operate and
produce so many satellite data which no one can / have used.
If you would like I can make a long list of meteorologists and satellite
experts which I have sent a mail and have got "sorry" answer or no
answer at all.
I the field of remote sensing I have been met many times some statements
in which one claims that has done this or that but nobody can follow how
it has been done or if it has ever been done as it is claimed. If it is
calibration or correction of a satellite image or just some remotely
sensed maps or other earth surface products.
So I suppose would enough for the time being. If you are still
interested in this matter, just let me know.
Yours
Hossein
~~~~~~~
Mark Hadfield wrote:
> Yamout wrote:
>
>> Dear Mark,
>>
>> thank you for your reply. As I said, I have ENVI with full version of
>> IDL unfortunately in PC (windows) platform. As you have written there
>> are some differences between the before and after 1995 formats. I have
>> asked SAA people and they can only send me such "helps" which I knew
>> already.
>
>
> So are you saying that you *can* read the post-1995 format? If so, how?
> Is there any possibility of your modifying the reader for the post-1995
> file so that it will cope with the difference in record length?
>
> Please be patient if the answers to these questions seem obvious to you,
> but I don't know much about ENVI. Please realise when you post questions
> to a news group that you need to give the readers enough information for
> them to help you.
>
> ---
> Mark Hadfield "Ka puwaha te tai nei, Hoea tatou"
> m.hadfield@niwa.co.nz
> National Institute for Water and Atmospheric Research (NIWA)
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
