| scientific data format [message #13985] |
Thu, 14 January 1999 00:00  |
Vincent Fournier-Sicr
Messages: 7
Registered: June 1998
|
Junior Member |
|
|
I am still undecided whether I should use NetCDF, CDF or
HDF format to store data. Could somebody tell me what the
pros/cons are for each one of these formats? Is there a
document available somewhere that compares those formats?
By advance thanks,
Vincent
___________________________________________________
Vincent Fournier-Sicre vfs@acri.fr
http://www.acri.fr/
ACRI S.A.
260 Route du Pin Montard, BP 234
06904 Sophia Antipolis - FRANCE
TEL (+33) (0) 4 92 96 75 14
FAX (+33) (0) 4 93 95 80 98
___________________________________________________
|
|
|
|
| Re: scientific data format [message #14063 is a reply to message #13985] |
Fri, 15 January 1999 00:00  |
David McClain
Messages: 17
Registered: January 1999
|
Junior Member |
|
|
My own experience with NetCDF (quite extensive for the past 4 years) is
mixed:
1. You should definitely get the source code and fix it up. There are
numerous bugs in the code. It was written in a fairly crude manner in
pre-ANSII C.
2. It is a 32-bit implementation that places a limit of around 2 GBytes to
the largest file.
3. It is being supplanted by HDF as the premier file format. However, if
NetCDF appears complicated, then HDF is even more so.
I have developed a universal interface to scientific data repositories
called SciDS. NetCDF is only one of its available formats. We also use it to
attach to live hardware. The advantage is that the application code needs
only to understand SciDS (very simple, even compared with NetCDF), and the
data source can be attached at runtime to whatever SciDS data source you
have, be it NetCDF, Excel CSV, live hardware, or whatever. The application
can't tell the difference.
I plan to incorporate HDF into the SciDS system also, as much to control the
complexity of the interface, as for any other reason.
CDF Appears to be an ancient NASA standard that was supplanted by NetCDF.
Hope this helps...
- DM
David McClain
Sr. Scientist
Raytheon Missile Systems Co.
Tucson, AZ
http://www.azstarnet.com/~dmcclain/homepage.htm
-----Original Message-----
From: Vincent Fournier-Sicre <vfs@acri.fr>
Newsgroups: comp.lang.idl-pvwave
Date: Thursday, January 14, 1999 05:16
Subject: scientific data format
> I am still undecided whether I should use NetCDF, CDF or
> HDF format to store data. Could somebody tell me what the
> pros/cons are for each one of these formats? Is there a
> document available somewhere that compares those formats?
> By advance thanks,
> Vincent
>
> ___________________________________________________
> Vincent Fournier-Sicre vfs@acri.fr
> http://www.acri.fr/
> ACRI S.A.
> 260 Route du Pin Montard, BP 234
> 06904 Sophia Antipolis - FRANCE
> TEL (+33) (0) 4 92 96 75 14
> FAX (+33) (0) 4 93 95 80 98
> ___________________________________________________
|
|
|
|
| Re: scientific data format [message #14160 is a reply to message #13985] |
Tue, 26 January 1999 00:00 |
Liam Gumley
Messages: 473
Registered: November 1994
|
Senior Member |
|
|
raph@phy.ornl.gov wrote:
> Vincent Fournier-Sicre <vfs@acri.fr> wrote:
>> I am still undecided whether I should use NetCDF, CDF or
>> HDF format to store data. Could somebody tell me what the
>> pros/cons are for each one of these formats? Is there a
>> document available somewhere that compares those formats?
>> By advance thanks,
>> Vincent
>
> I asked the same question to the consultants at NERSC about 6 months
> ago. Their response was to go with HDF since it was a little better
> supported (both at NERSC and in general), a little faster evolving, and
> accessable by more tools (though there are a fair number of tools which
> can access netCDF and not HDF). There are in fact efforts under way to
> make HDF and netCDF interoperable, check out
> http://www.unidata.ucar.edu/packages/netcdf/faq.html#hdf
Faster evolution is not necessarily a good thing. For example, the
latest version of HDF (HDF5) is to my knowledge completely incompatible
with HDF4, and NCSA has no plans to include a backwards compatible API
(to read HDF4 and HDF3) in HDF5.
As an example, HDF files written with HDF3.3r1 (1993 vintage) that
contain SDS objects cannot be read correctly by HDF4.1r2 (the current
release). Thus I have HDF files that were created in 1993 which cannot
be read with the current version of HDF.
In contrast netCDF V3 is completely compatible with netCDF V2
(see http://www.unidata.ucar.edu/packages/netcdf/release-notes-3. 3.html)
The data model offered by netCDF is also much more straightforward than
the data model offered by HDF4 (I don't know about HDF5). This is a big
plus in my opinion, since it means the API is much simpler. Yet the
netCDF data model is flexible enough to allow users to be very creative
in how they structure datasets.
Anyway, I've used both netCDF and HDF since 1991, and my personal
preference is for netCDF.
Cheers,
Liam.
---
Liam E. Gumley
Space Science and Engineering Center, UW-Madison
1225 W. Dayton St., Madison WI 53706, USA
Phone (608) 265-5358, Fax (608) 262-5974
http://cimss.ssec.wisc.edu/~gumley
|
|
|
|
| Re: scientific data format [message #14164 is a reply to message #13985] |
Tue, 26 January 1999 00:00 |
raph
Messages: 8
Registered: October 1997
|
Junior Member |
|
|
Vincent Fournier-Sicre <vfs@acri.fr> wrote:
> I am still undecided whether I should use NetCDF, CDF or
> HDF format to store data. Could somebody tell me what the
> pros/cons are for each one of these formats? Is there a
> document available somewhere that compares those formats?
> By advance thanks,
> Vincent
I asked the same question to the consultants at NERSC about 6 months
ago. Their response was to go with HDF since it was a little better
supported (both at NERSC and in general), a little faster evolving, and
accessable by more tools (though there are a fair number of tools which
can access netCDF and not HDF). There are in fact efforts under way to
make HDF and netCDF interoperable, check out
http://www.unidata.ucar.edu/packages/netcdf/faq.html#hdf
Raph
------------------------------------------------------------ -----------
William Raphael Hix Email: raph@phy.ornl.gov
Joint Institute for Heavy Ion Research Voice: (423) 576-8763
Oak Ridge National Laboratory Fax: (423) 576-5780
P.O. Box 2008
Oak Ridge, TN 37831-6374
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
