|
|
|
|
| Re: GRIB data question [message #78323 is a reply to message #78322] |
Fri, 11 November 2011 05:55  |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <MPG.292607687621692a9898de@news.giganews.com>,
David Fanning <news@dfanning.com> wrote:
> The problem, as you suggest, is easier to manage with global
> data sets. It is a LOT harder when the spatial extent of the
> data is limited. When you are in the middle of the ocean, as
> we appear to be with this data set, the bets as to which way
> is up get iffy, at best.
You don't want to get me started on the inadequacy of metadata
in publicly available data sets.
I would, someday, like to see a data set where the metadata
actually provides all of the necessary information about
a dataset.
Ken
|
|
|
|
| Re: GRIB data question [message #78330 is a reply to message #78323] |
Thu, 10 November 2011 15:08 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Kenneth P. Bowman writes:
> A quick and simple sanity check is to plot a global map of
> surface temperature or air temperature at the lowest level
> (typically 1000 hPa). The continents stand out quite clearly.
Sigh...
OK, I'm an old man (well, a hell of a lot older than I was
when I started worrying about map projections about three
weeks ago for a new book I am writing!), and I've been around
the block a couple of times, so problems with data don't
particularly surprise me. But, in the example Mark presents,
the only "clue" to how to apply the map projection information
in the file to the extremely limited-spatial-extent data that
is in the file (nothing like something where continents are
going to stand out!) is the word "Alaska" in his description
of the data set. One would presume that you might find an outline
of Alaska somewhere near the image data!
I have every confidence that Mark has displayed the data
correctly. My only concern is how he knew to display it in
that particular way. Relying on the word "Alaska" seems
particularly fraught with unpleasant possibilities. :-(
This is an issue that I would like to address in the book,
because I used to run into it over and over again when I
was working at NSIDC. How do you know whether or not
your data is upside down!? It seems a simple thing, but
it really is not. And it is MOST embarrassing to have this
pointed out to you in the middle of an important presentation
reporting your results!
The problem, as you suggest, is easier to manage with global
data sets. It is a LOT harder when the spatial extent of the
data is limited. When you are in the middle of the ocean, as
we appear to be with this data set, the bets as to which way
is up get iffy, at best.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: GRIB data question [message #78331 is a reply to message #78330] |
Thu, 10 November 2011 14:40 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <MPG.2925d11397c59c409898dc@news.giganews.com>,
David Fanning <news@dfanning.com> wrote:
> Kenneth P. Bowman writes:
>
>> So when I get GRIB files, the first thing I do is put the
>> data into right-handed coordinate systems in netCDF files.
>> This requires some work, but then everything is consistent,
>> making life vastly easier for me and those I work with.
>
> Yeah, OK, I agree with this. But, how do you do it? How
> do you know what coordinate system the GRIB file creator
> had in mind when he or she created the file? How do you
> know what variables you have to change to get them into
> your right-handed coordinate system?
>
> Cheers,
>
> David
A quick and simple sanity check is to plot a global map of
surface temperature or air temperature at the lowest level
(typically 1000 hPa). The continents stand out quite clearly.
You can also compare something like 500 hPa geopotential height
with a trusted source, like maps on the weather service web site.
Ken
|
|
|
|
| Re: GRIB data question [message #78332 is a reply to message #78331] |
Thu, 10 November 2011 11:25 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
David Fanning writes:
> Yeah, OK, I agree with this. But, how do you do it? How
> do you know what coordinate system the GRIB file creator
> had in mind when he or she created the file? How do you
> know what variables you have to change to get them into
> your right-handed coordinate system?
Incidentally, I think everyone agrees on a right-hand
coordinate system (maybe some of the planetary guys do
it differently on Mars or something). That's really not
the issue.
The issue is how do you start counting columns and rows.
The map projection people use a right-handed coordinate
system. It's just that they start counting from the
upper-left corner. Positive columns go to the right,
and positive rows go down (toward the origin of their
right-handed coordinate system, if you like).
My question is this: Is there a convention for how
and where one starts to count columns and rows for
a GRIB 2D array?
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: GRIB data question [message #78333 is a reply to message #78332] |
Thu, 10 November 2011 11:16 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Kenneth P. Bowman writes:
> So when I get GRIB files, the first thing I do is put the
> data into right-handed coordinate systems in netCDF files.
> This requires some work, but then everything is consistent,
> making life vastly easier for me and those I work with.
Yeah, OK, I agree with this. But, how do you do it? How
do you know what coordinate system the GRIB file creator
had in mind when he or she created the file? How do you
know what variables you have to change to get them into
your right-handed coordinate system?
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: GRIB data question [message #78334 is a reply to message #78333] |
Thu, 10 November 2011 10:56 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <MPG.2925b646c20b58a9898db@news.giganews.com>,
David Fanning <news@dfanning.com> wrote:
> I found this piece of documentation. If you read the
> section entitled The Solution - Part 1, toward the
> end of the section, you find that the GRIB data has
> a grid that goes from 90 to -90 in lat and 0 to 360
> in lon, and he indicates the (1,1) point is at 90
> deg N and 0 deg E. This indicates to me that the
> convention is the upper-left convention, which is
> (honestly) what I would expect.
>
> http://www.iges.org/grads/gadoc/grib2.html
This doesn't really have anything to do with image data. When
global atmospheric models (particularly spectral transform codes)
were first being developed in the 60's and 70's, the designers had
to make choices about storage conventions. Some of the original
models developed at the time used the convention of ordering data
from the north pole to the south pole. This may be in part because
some models had both one- and two-hemisphere versions.
Starting at the north pole makes indices consistent in the
northern hemisphere (where most models were developed).
Models generally evolve from earlier codes, and changing
conventions in the code is likely to be painful and introduce
difficulty to track bugs, so it is not surprising that
spectral transform models continue to this day to start
the latitude indexing at the north pole.
Grid point models are often different.
Similarly, some models (and some data sets) go from (-180, 180)
in longitude, while others go from (0, 360). Different
developers made, and continue to make, different choices.
I think the choice was unfortunate, because I prefer to
have everything in a right-handed coordinate system where a
variable f(x, y, z) is stored as f[i,j,k], with i increasing
in longitude (0 to 360), j increasing in latitude
(-90, 90), and k increasing in altitude (0, top) or decreasing
in pressure (1000, 0).
So when I get GRIB files, the first thing I do is put the
data into right-handed coordinate systems in netCDF files.
This requires some work, but then everything is consistent,
making life vastly easier for me and those I work with.
Ken
|
|
|
|
| Re: GRIB data question [message #78335 is a reply to message #78334] |
Thu, 10 November 2011 09:22 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Fabzou writes:
>
> On 11/10/2011 05:41 PM, David Fanning wrote:
>> Normally, if you work with map projection software at all,
>> you think of the first row and first column as being
>> located in the upper-left corner of the image data.
>> This is how ENVI, and GeoTiff files, and any netCDF
>> file I have ever worked with think.
>
> Some ncdf files are not like this, e.g. TRMM files, or output files from
> the community model WRF (which uses meteorological data in GRIB format
> as input).
>
> I don't know about GRIB files in general, though.
I found this piece of documentation. If you read the
section entitled The Solution - Part 1, toward the
end of the section, you find that the GRIB data has
a grid that goes from 90 to -90 in lat and 0 to 360
in lon, and he indicates the (1,1) point is at 90
deg N and 0 deg E. This indicates to me that the
convention is the upper-left convention, which is
(honestly) what I would expect.
http://www.iges.org/grads/gadoc/grib2.html
I see in the GRIB documentation itself that you are
encouraged to write the latitude and longitude of
not just the first grid cell, but the last grid cell,
too, which might remove this ambiguity. But, the data
that Mark provided (which he got from a NOAA source)
doesn't seem to follow this sensible convention,
either. :-(
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: GRIB data question [message #78336 is a reply to message #78335] |
Thu, 10 November 2011 08:54 |
Fabzou
Messages: 76
Registered: November 2010
|
Member |
|
|
On 11/10/2011 05:41 PM, David Fanning wrote:
> Normally, if you work with map projection software at all,
> you think of the first row and first column as being
> located in the upper-left corner of the image data.
> This is how ENVI, and GeoTiff files, and any netCDF
> file I have ever worked with think.
Some ncdf files are not like this, e.g. TRMM files, or output files from
the community model WRF (which uses meteorological data in GRIB format
as input).
I don't know about GRIB files in general, though.
Fab
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
