| Re: CDF vs. netCDF [message #8459] |
Sun, 16 March 1997 00:00 |
Mark Hadfield
Messages: 783
Registered: May 1995
|
Senior Member |
|
|
I have used both CDF and (more recently) netCDF for most of my data
storage/transfer needs for some time now. They're pretty similar really,
but in choosing between them you might like to consider the following
random comments:
General:
* IDL implementations of the scientific data formats are typically
one or two versions behind the latest release, so if there's a
specific feature you want, check if it's available in the IDL
implementation.
* Recent devlopments in the scientific data formats relate mainly to
sparse data & compression.
* The scientific data formats are much more pleasant to use than
unformatted binary because they're self-cataloguing.
* Compared with unformatted binary, file sizes are similar (with just
a little overhead for catalogue information) but I/O is definitely
slower.
CDF vs netCDF:
* CDF has a more flexible data model (through the use of Z variables).
With netCDF (and with CDF Z variables) all dimensions but one must
have their sizes fixed before data can be written.
* If interchange of data with other groups is an issue, check out which
package is most often used in your area. In my area
(meteorology/oceanography) netCDF is the norm.
* If access by non-IDL code is an issue, then I would recommend netCDF
over CDF because it has a better interface. (The CDF interface uses
variable-length argument lists and is a little TOO clever IMHO.)
* netCDF is slower, in my experience, particularly in retrieving large
numbers of records over a network link.
Finally:
* If you're using IDL for Windows, then netCDF is the clear winner,
because
the IDL CDF routines can't access data in chunks of more than 64KB.
(This
is a thoroughly unnecessary restriction that wasn't present in the
earliest
versions of CDF for IDL and has been the subject of several emails from
me to RSI. Nevertheless it's still there in the IDL 5.0 pre-release. I
still get angry just thinking about it .......!!!!!!!!!!!!!!)
Hope this helps.
--
============================================================ ==
Mark Hadfield NIWA (Taihoro Nukurangi)
PO Box 14-901
m.hadfield@niwa.cri.nz Wellington, New Zealand
Michael Ochs <m_ochs@fccc.edu> wrote in article <3329913C.685F@fccc.edu>...
> We are converting a moderately large set of programs to
> IDL for distribution. I am reviewing options for the
> data format to use and downloaded the CDF and netCDF
> user's guides. What is not clear from what I have seen
> are the advantages and disadvantages to the two formats.
>
> Any comments on this would be useful. Our software handles
> large MRI images and spectra (thus IDL) and is going to
> be designed to run on many platforms (thus IDL again). One
> thing I would like to know is whether netCDF adds significantly
> to file size and how both compare to using IDL's unformatted
> routines like ASSOC.
>
> -- Mike
>
>
|
|
|
|
| Re: CDF vs. netCDF [message #8466 is a reply to message #8459] |
Fri, 14 March 1997 00:00  |
David Foster
Messages: 341
Registered: January 1996
|
Senior Member |
|
|
Michael Ochs wrote:
>
> We are converting a moderately large set of programs to
> IDL for distribution. I am reviewing options for the
> data format to use and downloaded the CDF and netCDF
> user's guides. What is not clear from what I have seen
> are the advantages and disadvantages to the two formats.
>
> Any comments on this would be useful. Our software handles
> large MRI images and spectra (thus IDL) and is going to
> be designed to run on many platforms (thus IDL again). One
> thing I would like to know is whether netCDF adds significantly
> to file size and how both compare to using IDL's unformatted
> routines like ASSOC.
>
I have been writing software for analysis of MRI images at UCSD for
quite some time, and have never encountered the need for CDF or
netCDF. Of course, we are using UNIX systems exclusively, but I
would still recommend keeping your MR images in their native
format. There are other tools (and some IDL "shareware") that
will expect your data to be in native form (GE Cigna?).
For I/O, we're just using plain vanilla unformatted binary
READU/WRITEU. When we need access to the entire series,
we read the images into a 3D volume. You could concatenate the
images into a single volume file of some sort, but then you
lose the header information that comes with the images (and
we haven't really found a good reason to use ASSOC; the speed
improvement isn't that much compared to binary reads of individual
images).
Good luck!
Dave
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~
David S. Foster Univ. of California, San Diego
Programmer/Analyst Brain Image Analysis Laboratory
foster@bial1.ucsd.edu Department of Psychiatry
(619) 622-5892 8950 Via La Jolla Drive, Suite 2200
La Jolla, CA 92037
[ UCSD Mail Code 0949 ]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
