| Writing large [x,x,x,x] files [message #24082] |
Fri, 09 March 2001 07:33  |
Sean Heukels
Messages: 25
Registered: November 1999
|
Junior Member |
|
|
What write procedure can I use when I want to write a
matrix to file that is 128x128, multislice and with multi timepoint
Say data(128,128,4,50).
I tried PNG, but it says it will only take (1-4, n, m)
Does anyone know of a better file format ???
And furthermore, would that format be able to allow a user to write a little
bit of extra info, say a description to a header/footer in that same file ??
Its just that I've written a nice program, but I want to export the data to
some kind of format. So I can make it portanle and re-open the again.
The raw format that I imported takes a long while to be decoded, so I
wouldn't mind having some other more common format
Greets Sean
|
|
|
|
| Re: Writing large [x,x,x,x] files [message #24171 is a reply to message #24082] |
Fri, 09 March 2001 15:06  |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
"Sean Heukels" <sean77=cuthere=@dds.nl> writes:
> What write procedure can I use when I want to write a
> matrix to file that is 128x128, multislice and with multi timepoint
>
> Say data(128,128,4,50).
> I tried PNG, but it says it will only take (1-4, n, m)
> Does anyone know of a better file format ???
>
> And furthermore, would that format be able to allow a user to write a little
> bit of extra info, say a description to a header/footer in that same file ??
>
> Its just that I've written a nice program, but I want to export the data to
> some kind of format. So I can make it portanle and re-open the again.
> The raw format that I imported takes a long while to be decoded, so I
> wouldn't mind having some other more common format
I'll chime in.
Astronomy folks use FITS for data exchange, but other people can too.
I know that a lot of people complain that FITS is archaic but it's
pretty effective, especially with the excellent packages in the IDL
Astronomy Library. For example:
To save the data
x = findgen(3,4,5)
fxhmake, header, x
fxaddpar, header, 'COMMENT', 'This data contains bogus values on June 3'
fxwrite, 'myfile.fits', header, x
To read the data later
fxread, 'myfile.fits', y
The cool thing about FITS files is that the header is written in ASCII
text so any editor can look at it.
Another option is to use IDL SAVE files, which are native to IDL. IDL
has enough to save and later read the data. With my additional SAVE
library you can selectively restore data, or just find out what's
inside the file by making a listing.
To save the data:
comment = 'This data contains bogus values on June 3'
save, x, comment, file='myfile.sav'
To read the data:
restore, 'myfile.sav'
To selectively restore X only (using my SAVE library):
cmrestore, x, 'myfile.sav'
Good luck,
Craig
P.S. My CMSVLIB library is here:
http://cow.physics.wisc.edu/~craigm/idl/idl.html (under Save Files)
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
| Re: Writing large [x,x,x,x] files [message #24174 is a reply to message #24082] |
Fri, 09 March 2001 09:58 |
Liam E. Gumley
Messages: 378
Registered: January 2000
|
Senior Member |
|
|
Sean Heukels wrote:
> What write procedure can I use when I want to write a
> matrix to file that is 128x128, multislice and with multi timepoint
>
> Say data(128,128,4,50).
> I tried PNG, but it says it will only take (1-4, n, m)
> Does anyone know of a better file format ???
>
> And furthermore, would that format be able to allow a user to write a little
> bit of extra info, say a description to a header/footer in that same file ??
>
> Its just that I've written a nice program, but I want to export the data to
> some kind of format. So I can make it portanle and re-open the again.
> The raw format that I imported takes a long while to be decoded, so I
> wouldn't mind having some other more common format
For a simple format that can be used in IDL and other applications such
as C, FORTRAN, Matlab:
http://cimss.ssec.wisc.edu/~gumley/binarytools.html
For a more general and flexible solution, use netCDF as Paul
recommended.
Cheers,
Liam.
|
|
|
|
| Re: Writing large [x,x,x,x] files [message #24260 is a reply to message #24082] |
Mon, 19 March 2001 08:49 |
Paul van Delst
Messages: 364
Registered: March 1997
|
Senior Member |
|
|
Martin Schultz wrote:
>
> Craig Markwardt wrote:
>>
>> [...]
>> I'll chime in.
>>
>> Astronomy folks use FITS for data exchange, but other people can too.
>> I know that a lot of people complain that FITS is archaic but it's
>> pretty effective, [...]
>
> Oh let me advocate GRIB, a format commonly used by meteorological
> agencies (and established as standard by the WMO). It's archaic, it
> has a complicated user interface, but it is effective in packing data.
> And, most wonderful of all: if you would go for it, we would finally
> get a nice interface to read and write GRIB files from IDL ;-)
> [without call_external, i.e.]
>
> Cheers,
>
> Don't-take-me-serious-Martin
>
> PS: The fact that there are no GRIB utilities available in any library
> that I know of shows that not too many met people are using IDL. Or is
> everyone happy with the wgrib solution??
I convert GRIB->netCDF and read the latter in IDL (and other) code. :o)
(http://www.unidata.ucar.edu/packages/decoders)
Disk space nowadays is cheap enough and readily available - packing data to save space is
usually more trouble than it's worth (unless you're dealing with GB->TB/day from, say,
satellite instruments). If your internet connection is too slow and your data absolutely,
positively has to be there overnight - Fedex a DAT tape. :o)
paulv
--
Paul van Delst A little learning is a dangerous thing;
CIMSS @ NOAA/NCEP Drink deep, or taste not the Pierian spring;
Ph: (301)763-8000 x7274 There shallow draughts intoxicate the brain,
Fax:(301)763-8545 And drinking largely sobers us again.
paul.vandelst@noaa.gov Alexander Pope.
|
|
|
|
| Re: Writing large [x,x,x,x] files [message #24262 is a reply to message #24171] |
Mon, 19 March 2001 07:31 |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
Craig Markwardt wrote:
>
>
> I'll chime in.
>
> Astronomy folks use FITS for data exchange, but other people can too.
> I know that a lot of people complain that FITS is archaic but it's
> pretty effective,
Oh let me advocate GRIB, a format commonly used by meteorological
agencies (and established as standard by the WMO). It's archaic, it
has a complicated user interface, but it is effective in packing data.
And, most wonderful of all: if you would go for it, we would finally
get a nice interface to read and write GRIB files from IDL ;-)
[without call_external, i.e.]
Cheers,
Don't-take-me-serious-Martin
PS: The fact that there are no GRIB utilities available in any library
that I know of shows that not too many met people are using IDL. Or is
everyone happy with the wgrib solution??
--
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
[[ Dr. Martin Schultz Max-Planck-Institut fuer Meteorologie [[
[[ Bundesstr. 55, 20146 Hamburg [[
[[ phone: +49 40 41173-308 [[
[[ fax: +49 40 41173-298 [[
[[ martin.schultz@dkrz.de [[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
