| Re: NCEP Reanalysis Temperature Data Problem [message #56492] |
Thu, 25 October 2007 11:27  |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <MPG.218a8245507fa0a9989cea@news.frii.com>,
David Fanning <david@dfanning.com> wrote:
> David Fanning writes:
>
>> I even considered a byteorder problem (although how
>> this would happen in NetCDF files is beyond me), but
>> that just make the problem worst.
>
> I'm so flummoxed by the whole situation, I am
> speaking in a faux-Chinese accent now. :-(
>
> Cheers,
>
> David
The time is expressed as hours since 0001-01-01 00:00:00.
I have a rather large library to read these files if you are
interested. Of course, it uses my own date and time routines. ;-)
Ken Bowman
|
|
|
|
|
|
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56501 is a reply to message #56492] |
Thu, 25 October 2007 11:42  |
ben.bighair
Messages: 221
Registered: April 2007
|
Senior Member |
|
|
On Oct 25, 2:33 pm, David Fanning <da...@dfanning.com> wrote:
> Folks,
>
> OK, just so we are all on the same page here. The time data
> in this NetCDF file is in hours since 1 January 0001. (Strange,
> but true.) BUT, here is the thing, you are not *suppose*
> to worry about that. What you are suppose to worry about
> is the time *difference*.
>
> So, in this data set, if I subtract time[1]-time[0]
> I should have the number of hours between the two
> measurements. If I divide by 24, I should get the number
> of days:
>
> IDL> Print, (time[1]-time[0]) / 24
> 31
>
> In fact, if I do this:
>
> IDL> Print, (Shift(time,1) - time) / 24
>
> I get a comforting series of 31s and 30s, with the occational
> 28 thrown in there to give me hope!
>
> So, all appears to be well in the Universe again. :-)
>
> Thanks for everyone's help.
>
All this help was, ehem, timely?
Cheers,
Ben
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56502 is a reply to message #56492] |
Thu, 25 October 2007 11:37 |
R.G.Stockwell
Messages: 163
Registered: October 2004
|
Senior Member |
|
|
"David Fanning" <david@dfanning.com> wrote in message
news:MPG.218a8f2f68d912f6989cee@news.frii.com...
> Folks,
>
> OK, just so we are all on the same page here. The time data
> in this NetCDF file is in hours since 1 January 0001. (Strange,
> but true.) BUT, here is the thing, you are not *suppose*
> to worry about that. What you are suppose to worry about
> is the time *difference*.
I'd guess it is just Julian Day * 24 (+ offset). So let the caldat julday
routines do all the headaching about it.
So you can use it as the absolute date (as in that blurb of code
i posted), as well as just looking at differences.
Cheers,
bob
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56503 is a reply to message #56492] |
Thu, 25 October 2007 11:35 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
R.G. Stockwell writes:
> hour = 17067072d
>
> day = hour/24
>
> julday = julday(1,1,1,0,0,0)+day
>
>
> CALDAT, julday, Month , Day , Year , Hour , Minute , Second
>
> print, Month , Day , Year , Hour , Minute , Second
>
>>
> 1 1 1948 0 0 4.0233135e-005
>
>
> Seems bang on.
I'll write this up, so the next poor schmuck who has to
read a new type of NetCDF file doesn't waste a whole morning!
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56504 is a reply to message #56492] |
Thu, 25 October 2007 11:33 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Folks,
OK, just so we are all on the same page here. The time data
in this NetCDF file is in hours since 1 January 0001. (Strange,
but true.) BUT, here is the thing, you are not *suppose*
to worry about that. What you are suppose to worry about
is the time *difference*.
So, in this data set, if I subtract time[1]-time[0]
I should have the number of hours between the two
measurements. If I divide by 24, I should get the number
of days:
IDL> Print, (time[1]-time[0]) / 24
31
In fact, if I do this:
IDL> Print, (Shift(time,1) - time) / 24
I get a comforting series of 31s and 30s, with the occational
28 thrown in there to give me hope!
So, all appears to be well in the Universe again. :-)
Thanks for everyone's help.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56505 is a reply to message #56492] |
Thu, 25 October 2007 11:23 |
R.G.Stockwell
Messages: 163
Registered: October 2004
|
Senior Member |
|
|
"David Fanning" <david@dfanning.com> wrote in message
news:MPG.218a87139cfe6277989ced@news.frii.com...
> R.G. Stockwell writes:
>
>> print, 17067072d/24/365
>>
>> 1948.2959
>>
>> print, 17590104d/24/365
>>
>> 2008.0027
>>
>> (i'd guess number of hours since 1/1/1 0:0:0 or something)
>
> Ahhhh. Still a strange unit, como no?
>
> And do we add or subtract the month Julius
> Ceasar declared didn't exist to get the final
> value? :-(
hour = 17067072d
day = hour/24
julday = julday(1,1,1,0,0,0)+day
CALDAT, julday, Month , Day , Year , Hour , Minute , Second
print, Month , Day , Year , Hour , Minute , Second
>
1 1 1948 0 0 4.0233135e-005
Seems bang on.
Cheers,
bob
|
|
|
|
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56509 is a reply to message #56507] |
Thu, 25 October 2007 10:58 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
R.G. Stockwell writes:
> print, 17067072d/24/365
>
> 1948.2959
>
> print, 17590104d/24/365
>
> 2008.0027
>
> (i'd guess number of hours since 1/1/1 0:0:0 or something)
Ahhhh. Still a strange unit, como no?
And do we add or subtract the month Julius
Ceasar declared didn't exist to get the final
value? :-(
Thanks,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56511 is a reply to message #56510] |
Thu, 25 October 2007 10:48 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
ben.bighair writes:
> While it is hard to figure out the benchmark date - the difference in
> the two might be close to the number of hours between 1948 and 2007.
> Perhaps it's wishful thinking, but if you know the final date you
> might be able to use the first record as a bench mark and figure
> subsequent dates from there.
>
> IDL> t1=17590104 & t0 = 17067072
> IDL> print, t1-t0
> 523032
> IDL> print,(2007-1948)*365.25*24.
> 517194.
Yeah, I've been down that road, too. But, again,
the numbers just don't ever add up. Which leads
me to think it is also a dead end. Seconds, years, ...
Milliseconds comes closest, but for gridded data
and a monthly average!? Even govenment scientists
aren't that... Well, never mind. I almost is one on
this project. :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56512 is a reply to message #56511] |
Thu, 25 October 2007 10:43 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
David Fanning writes:
> P.S. This MIGHT make sense if it is the number of seconds
> since the demise of the dinosaurs, but this seems a dubious
> time unit.
I could, of course, just make up my own damn time
variables. Which, I'm coming to believe strongly, is
what my predecessor did. But that does seem a little
like cheating to me.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56513 is a reply to message #56512] |
Thu, 25 October 2007 10:38 |
ben.bighair
Messages: 221
Registered: April 2007
|
Senior Member |
|
|
On Oct 25, 12:56 pm, David Fanning <da...@dfanning.com> wrote:
> Folks,
>
> I am looking at some monthly NCEP Temperature Data, stored in a
> NetCDF file:
>
> http://www.cdc.noaa.gov/dtat.ncep.reanalysis.pressure.html
>
> I have no problem reading the data file, but the time data,
> which is suppose to be monthly from 1 Jan 1948 to present,
> has a min and max values of 17067072 and 17590104. These looks like
> Julian numbers to me, except they are off by several orders
> of magnitude from anything that makes sense to me!!
>
> IDL> Caldat, 17067072, m, d, y
> IDL> Print, m, d, y
> 12 15 42015
>
> These numbers are even several orders of magnitude bigger
> than the number of SECONDS since 1948. :-(
>
> Does anyone have any experience with this data set and
> have some idea of how I can get these values into something
> that makes sense?
>
Hi David,
While it is hard to figure out the benchmark date - the difference in
the two might be close to the number of hours between 1948 and 2007.
Perhaps it's wishful thinking, but if you know the final date you
might be able to use the first record as a bench mark and figure
subsequent dates from there.
IDL> t1=17590104 & t0 = 17067072
IDL> print, t1-t0
523032
IDL> print,(2007-1948)*365.25*24.
517194.
Cheers,
Ben
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56514 is a reply to message #56513] |
Thu, 25 October 2007 10:37 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
David Fanning writes:
> I even considered a byteorder problem (although how
> this would happen in NetCDF files is beyond me), but
> that just make the problem worst.
I'm so flummoxed by the whole situation, I am
speaking in a faux-Chinese accent now. :-(
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56515 is a reply to message #56514] |
Thu, 25 October 2007 10:32 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Conor writes:
> Have you considered that it might already be in date format? It looks
> like it could be in YYYY-MM-DD format. You would get:
>
> 70, 72, 1706 and 01, 04, 1795
>
> If you then allow months and days to wrap you would get actual dates
> of:
>
> December 20, 1710 and
> January 4, 1795
>
> That sounds pretty reasonable to me... It's obviously an old data
> set. Where was the US government taking temperature data in the
> 1700's???
Yes, I considered that, but the data is suppose to be
monthly from 1 Jan 1948, and there are the correct
*number* of months in the time data, just not the
(as far as I can tell) correct values. :-(
So values in the 1700s does me no good either.
I even considered a byteorder problem (although how
this would happen in NetCDF files is beyond me), but
that just make the problem worst.
Cheers,
David
P.S. This MIGHT make sense if it is the number of seconds
since the demise of the dinosaurs, but this seems a dubious
time unit.
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
|
|
|
|
| Re: NCEP Reanalysis Temperature Data Problem [message #56516 is a reply to message #56515] |
Thu, 25 October 2007 10:23 |
Conor
Messages: 138
Registered: February 2007
|
Senior Member |
|
|
Have you considered that it might already be in date format? It looks
like it could be in YYYY-MM-DD format. You would get:
70, 72, 1706 and 01, 04, 1795
If you then allow months and days to wrap you would get actual dates
of:
December 20, 1710 and
January 4, 1795
That sounds pretty reasonable to me... It's obviously an old data
set. Where was the US government taking temperature data in the
1700's???
On Oct 25, 12:56 pm, David Fanning <da...@dfanning.com> wrote:
> Folks,
>
> I am looking at some monthly NCEP Temperature Data, stored in a
> NetCDF file:
>
> http://www.cdc.noaa.gov/dtat.ncep.reanalysis.pressure.html
>
> I have no problem reading the data file, but the time data,
> which is suppose to be monthly from 1 Jan 1948 to present,
> has a min and max values of 17067072 and 17590104. These looks like
> Julian numbers to me, except they are off by several orders
> of magnitude from anything that makes sense to me!!
>
> IDL> Caldat, 17067072, m, d, y
> IDL> Print, m, d, y
> 12 15 42015
>
> These numbers are even several orders of magnitude bigger
> than the number of SECONDS since 1948. :-(
>
> Does anyone have any experience with this data set and
> have some idea of how I can get these values into something
> that makes sense?
>
> Thanks,
>
> David
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
