|
|
|
|
|
|
| Re: will RSI support HDF5? [message #27452 is a reply to message #26527] |
Tue, 23 October 2001 07:44   |
Liam E. Gumley
Messages: 378
Registered: January 2000
|
Senior Member |
|
|
H C Pumphrey wrote:
[stuff deleted]
> Now, NASA's EOS program is (a) a Large IDL customer, and (b) is actively
> switching from HDF4 to HDF5. Do I have to go on?
I have been involved with the NASA EOS project for about 10 years, and I
can tell you there is a rather large existing code base (in IDL, Matlab,
C, FORTRAN, Java, etc.) that assumes HDF4 will be supported by NCSA for
many years to come. To say that EOS is switching to HDF5 is overstating
the case. Some background may be in order.
In the early 1990s the EOS project chose to develop a wrapper API for
HDF4 that was named HDF-EOS. The intent was to provide an API targeted
towards earth science datasets that came in three flavors: Swaths (e.g.,
satellite images), Grids (e.g., numerical weather prediction
input/output), and Points (e.g., profiles of temperature through the
atmosphere). However it is important to realize that a HDF-EOS file is
just a specialized instance of a HDF4 file. Any HDF-EOS file can be read
using HDF4, because HDF-EOS files are built by combining various HDF4
objects (SDS and Vdata arrays) in a special way.
Because of this fact, many EOS developers chose to develop code that
*reads* HDF-EOS product files (from MODIS for example) using the HDF4
API, because the HDF4 API allows more flexibility in how EOS product
files are interpreted. For example, the MODIS Level-1B HDF-EOS product
reader I developed (available at
ftp://origin.ssec.wisc.edu/pub/MODIS/IDL/) reads the input file using
HDF4 calls *only*. I know many other developers who have used this
approach successfully in IDL, Matlab, C, FORTRAN, Java, and other
languages.
The EOS project is now *suggesting* that developers may wish to use a
new version of HDF-EOS that uses HDF5 as the underlying format (the API
remains unchanged). I believe it will be up to the individual EOS
instrument teams to decide whether they wish to use the new HDF-EOS
version. I expect the MODIS team will continue to use the HDF4 version
of HDF-EOS for the lifetime of the Terra and Aqua spacecraft.
The impact on IDL is that I expect RSI to continue to support HDF4
indefinitely. I also expect IDL to support a new HDF5 API in the future,
in addition to, and *not* instead of, the current HDF4 API. With regard
to HDF-EOS support in IDL, I think it's possible to envision a smart API
that checks the format of the input file to see if it's written in HDF4
or HDF5 flavor, and then handling it appropriately.
Cheers,
Liam.
Practical IDL Programming
http://www.gumley.com/
|
|
|
|
| Re: will RSI support HDF5? [message #27459 is a reply to message #26527] |
Tue, 23 October 2001 04:35 |
hcp
Messages: 41
Registered: August 1995
|
Member |
|
|
|> H C Pumphrey wrote:
|> > Let's just spell that out again in words of one syllable:
|> > HDF5 is NOT the next version of HDF4.[1]
The footnote never made it into the post -- it was just going to
point out that I know that version has two syllables. Getting back
to the important stuff,
In article <0fbB7.9542$R9.2917197@typhoon.we.rr.com>,
Phillip David <phillip_david@xontech.com> writes:
|> Currently, one of the most promising for our work is VisAD. This is
|> a client/server tool written in Java that currently supports HDF5,
|> and would work well to assist us with visualizing our data from a
|> central data facility without needing to transfer all of the data
|> across our internet lines. Instead, we'd just transfer over the
|> actual images. And Java's been optimized to do that very
|> efficiently.
Thanks for passing that on, Philip, I had not heard of VisAD until now.
Readers might like to note that (a) VisAD is LGPL, so it is (very) free
and (b) the VisAD homepage is at http://www.ssec.wisc.edu/~billh/visad.html
It looks a bit as if one would have to learn java and/or Python to use it.
This is not necessarily a bad thing.
All the best
Hugh
--
============================================================ ==============
Hugh C. Pumphrey | Telephone 0131-650-6026
Department of Meteorology | FAX 0131-650-5780
The University of Edinburgh | Replace 0131 with +44-131 if outside U.K.
EDINBURGH EH9 3JZ, Scotland | Email hcp@met.ed.ac.uk
OBDisclaimer: The views expressed herein are mine, not those of UofE.
============================================================ ==============
|
|
|
|
| Re: will RSI support HDF5? [message #27464 is a reply to message #26527] |
Tue, 23 October 2001 03:02 |
Phillip David
Messages: 36
Registered: April 1999
|
Member |
|
|
H C Pumphrey wrote:
> Let's just spell that out again in words of one syllable:
> HDF5 is NOT the next version of HDF4.[1] Apart from the facts that
> they are both file formats for storing large wads of scientific data,
> and
> that they both come from NCSA, they are totally and utterly different
> things.
>
> Struan Gray spake thusly:
>> My impression is that RSI is more interested in servicing its few
>> large customers than using people like me to kick-start interest in
>> IDL among new users. So be it.
>
> Now, NASA's EOS program is (a) a Large IDL customer, and (b) is
> actively switching from HDF4 to HDF5. Do I have to go on?
RSI has given us the impression that XonTech IS a large customer (we
have something like 40 licenses to IDL; most are for Windows, but a
few are for Macs. We're porting things over to Linux for some of our
codes, and really need HDF5 support NOW. We've been asking for the
last ... let's see ... about 2 years to get this. So apparently,
being a big customer isn't enough. Telling them that it's important
didn't do it either. We keep hearing "In version 6, we'll have HDF5
support", but then we keep getting more version 5 releases instead.
Now, with the dropping of plans for supporting Mac OSX and Alpha, and
the future of other Unix platforms possibly hanging in the balance,
we're going to have to look seriously at other options available to
us.
Currently, one of the most promising for our work is VisAD. This is
a client/server tool written in Java that currently supports HDF5,
and would work well to assist us with visualizing our data from a
central data facility without needing to transfer all of the data
across our internet lines. Instead, we'd just transfer over the
actual images. And Java's been optimized to do that very
efficiently.
It sounds like we're not the only ones looking into alternatives
to IDL, and I'm betting that RSI's recent announcements are only
going to speed the demise of sales they're experiencing.
I hope I'm wrong.
Phillip
|
|
|
|
| Re: will RSI support HDF5? [message #27465 is a reply to message #27376] |
Tue, 23 October 2001 02:34 |
hcp
Messages: 41
Registered: August 1995
|
Member |
|
|
Some while ago, "Richard W. Cooke" wrote:
|> > RSI will indeed support HDF 5, but it will not be until the release
|> > after the one just now coming out. Part of the reason for this is that
|> > there are still many customers using various version 4 libraries. I
|> > would expect something in the summer of 2002.
Rather sloppily, I didn't pick up the importance of this remark until...
In article <3BD48D50.21C4D907@xontech.com>, Phillip David wrote:
|> Apparently, RSI hasn't heard that HDF5 and HDF4 are DISTINCTLY DIFFERENT
|> programs.
Let's just spell that out again in words of one syllable:
HDF5 is NOT the next version of HDF4.[1] Apart from the facts that they are
both file formats for storing large wads of scientific data, and
that they both come from NCSA, they are totally and utterly different things.
|> Anyone using HDF4 will have to rewrite their programs to use
|> HDF5. But we can't do that until RSI SUPPORTS HDF5. So we can't even
|> begin to tackle that issue. [snip]
|> as long as RSI
|> doesn't provide HDF5 support, users will continue to use HDF4. The only
|> way for RSI to get customers to stop using HDF4 libraries is to wait for
|> people to get so fed up with the lack of HDF5 support that they leave
|> IDL entirely in favor of some other product with HDF5 support.
And he is 100% correct. Not a lot I can add to that, except to note that
over in the thread "Message From RSI VP of Engineering", Struan Gray
spake thusly:
> My impression is that RSI is more interested in servicing its few
> large customers than using people like me to kick-start interest in
> IDL among new users. So be it.
Now, NASA's EOS program is (a) a Large IDL customer, and (b) is actively
switching from HDF4 to HDF5. Do I have to go on?
Hugh
P.S. Huge extra OBDisclaimer. I certainly do not speak for any part
of NASA in any capacity whatsoever and any views in this article are
entirely my own personal opinions. This article may also have been
manufactured in a facility that processes nuts.
--
============================================================ ==============
Hugh C. Pumphrey | Telephone 0131-650-6026
Department of Meteorology | FAX 0131-650-5780
The University of Edinburgh | Replace 0131 with +44-131 if outside U.K.
EDINBURGH EH9 3JZ, Scotland | Email hcp@met.ed.ac.uk
OBDisclaimer: The views expressed herein are mine, not those of UofE.
============================================================ ==============
|
|
|
|
| Re: will RSI support HDF5? [message #27516 is a reply to message #27452] |
Wed, 24 October 2001 07:53 |
Richard Cooke
Messages: 2
Registered: October 2001
|
Junior Member |
|
|
Liam is absolutely correct. There will be continued support for the HDF4
API even after HDF5 support is available. The original response was to a
specific question about when HDF5 support would be available, nothing more.
"Liam E. Gumley" <Liam.Gumley@ssec.wisc.edu> wrote in message
I also expect IDL to support a new HDF5 API in the future,
> in addition to, and *not* instead of, the current HDF4 API. With regard
> to HDF-EOS support in IDL, I think it's possible to envision a smart API
> that checks the format of the input file to see if it's written in HDF4
> or HDF5 flavor, and then handling it appropriately.
>
> Cheers,
> Liam.
> Practical IDL Programming
> http://www.gumley.com/
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
