| Re: scientific data compression [message #38933] |
Sat, 03 April 2004 18:53 |
Mark Hadfield
Messages: 783
Registered: May 1995
|
Senior Member |
|
|
> Benjamin Hornberger wrote:
>
>
>> Hi all,
>>
>> I am trying to find out if there is a scientific data format which
>> suppports data compression and can be handled by IDL. We use netCDF so
>> far, but that doesn't support compression yet. HDF seems to support
>> compression as of version 4.2, but IDL supports only 4.1, is that
>> correct? Any alternatives?
It would be surprsing if there were any major file-format
incompatibility between HDF versions 4.1 & 4.2, though it's conceivable
that there are minor incompatibilities (for example, new compression
methods that were added to 4.2 and are not supported in 4.1).
It should take only a minute or two of rummaging around the HDF site to
check this out for yourself
--
Mark Hadfield "Ka puwaha te tai nei, Hoea tatou"
m.hadfield@niwa.co.nz
National Institute for Water and Atmospheric Research (NIWA)
|
|
|
|
| Re: scientific data compression [message #38943 is a reply to message #38933] |
Fri, 02 April 2004 16:11  |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Benjamin Hornberger wrote:
> Hi all,
>
> I am trying to find out if there is a scientific data format which
> suppports data compression and can be handled by IDL. We use netCDF so
> far, but that doesn't support compression yet. HDF seems to support
> compression as of version 4.2, but IDL supports only 4.1, is that
> correct? Any alternatives?
>
> Thanks,
> Benjamin
Some data could be quite good packed (not compressed) by using the
scale_factor and add_offset attribute. This data could then be saved as
integer or byte. This will save a lot of space.
We are using netCDF too. If we don't have much _FillValues or missing_values
then the filesize would not get much smaller if we try this with bzip2 or
gzip for our files. Did you have tried such a compression method before ?
HDF SD and netCDF will be in Version 4 of netCDF the same.
Reimar
--
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
http://www.fz-juelich.de/icg/icg-i/
============================================================ ======
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg-i/idl_icglib/idl_lib_intro. html
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
