| Re: CDF vs HDF [message #38863 is a reply to message #38862] |
Tue, 30 March 2004 13:25   |
mmiller3
Messages: 81
Registered: January 2002
|
Member |
|
|
>>>> > "Michael" == Michael Wallace <mwallace.removethis@swri.edu.invalid> writes:
> This isn't an IDL question exactly, but I figured that
> there are several here who have to deal with these file
> types. Is there any real difference between CDF and HDF?
> I have looked at some web pages that describe the
> differences, but it doesn't mean much to me since I've
> never delved into either one. The only reason I'm even
> looking at these file types is that these seem to be the
> types preferred by the data center we will be working with
> later on for long-term archival of our data. If it means
> anything, my core data sets are nothing more than really
> big arrays. Later on, other data like PNG images may be
> included.
> Unless there's some big difference between the two, I'm
> going to go with the one that's easier to work with in IDL.
> So, that brings me to the question, is one easier to work
> with than the other? Or do I flip a coin to determine
> which one to use?
CDF is pretty much superseded by netCDF, as far as I know, so you
might consider netCDF instead. In that case, the biggest
difference that I know of is that HDF 5 is capable of handling
arbitrary sized data sets. NetCDF cannot handle files larger
than 2 Gbytes due to an internal 32 bit integer. This has caused
me all sorts of headaches. I've had to write various wrappers
that split data sets into multiple netCDF files that are smaller
than 2 Gbytes each. HDF 5 handles that sort of thing
transparently. Not only can it handle arbitrarily large data
sets, but it will split large data sets across multiple files so
that OS limitations on individual file size are not surpassed.
That said, we've got such a large base of netCDF-based codes,
that we have not yet bitten the bullet and switched to HDF. If
we were to start a new, independent project, we'd certainly
choose HDF 5 instead.
For a summary and comparison to other formats, see
http://hdf.ncsa.uiuc.edu/HDF5/RD100-2002/All_About_HDF5.pdf .
That paper compares HDF 5 to the HDF 4, netCDF, PDB, TIFF, FITS
and OpenDX formats.
Mike
--
Michael A. Miller mmiller3@iupui.edu
Imaging Sciences, Department of Radiology, IU School of Medicine
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
