| netCDF tools? [message #19588] |
Mon, 03 April 2000 00:00  |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
Hi,
before I sit down and start writing my own stuff, I'd like to ask you
whether anyone
has created some suite of tools for netCDF files which allow for batch
processing
and possibly some interactive use. What I am especially interested in is
something
to extract a suite of variables from a series of netCDF files and
combine them into a new file (or append to an existing file).
Background: I need to create time series from model output which is
stored in huge one-file-contains-it-all files each covering only a
limited amount of time.
Any help appreciated,
Martin
PS: needn't necessarily be IDL, could also be some Unix tool, a C or
FORTRAN program, or a perl script.
--
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
[[ Dr. Martin Schultz Max-Planck-Institut fuer Meteorologie [[
[[ Bundesstr. 55, 20146 Hamburg [[
[[ phone: +49 40 41173-308 [[
[[ fax: +49 40 41173-298 [[
[[ martin.schultz@dkrz.de [[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
|
|
|
|
| Re: NetCDF tools [message #75560 is a reply to message #19588] |
Mon, 04 April 2011 06:43  |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Am 31.03.2011 20:34, schrieb Ed Hyer:
> Hi guys,
>
> I'm migrating some ugly FORTRAN binary blobs used by some applications
> into shiny new NetCDF files, and I realize I've never worked with
> NetCDF in IDL before.
> My first stop, as usual, was at Coyote, and it looks like the Coyote
> NetCDF object can be used to build whatever edifice I need.
>
> But before I get to building, I wanted to inquire of the assembled: is
> there a NetCDF equivalent to Liam Gumley's fantastic SDS_READ program
> for HDF files?
> If the answer is no, I may just be tempted to port that program to
> NetCDF, because I have become so used to it.
>
> What would be absolutely fantastic is a file that vacuums up all the
> attributes and data in a NetCDF file and returns a structure with
> everything, but I don't think that could be done without lots of
> EXECUTE(), which makes the baby Cthulhu cry.
How much data is it? I guess my read_ncdf does this.
Can you provide an example file for downloading. I like to try it out.
cheers
Reimar
>
> --Edward H.
|
|
|
|
| Re: NetCDF tools [message #75573 is a reply to message #19588] |
Thu, 31 March 2011 14:25 |
penteado
Messages: 866
Registered: February 2018
|
Senior Member
Administrator |
|
|
On Mar 31, 3:34 pm, Ed Hyer <ejh...@gmail.com> wrote:
> What would be absolutely fantastic is a file that vacuums up all the
> attributes and data in a NetCDF file and returns a structure with
> everything, but I don't think that could be done without lots of
> EXECUTE(), which makes the baby Cthulhu cry.
I have some old code that did it, with no execute, which I started to
migrate to a class, so that besides reading the whole file, variables
could be retrieved with the brackets.
If there is interest, I could dig those out and make them presentable.
|
|
|
|
| Re: NetCDF tools [message #75574 is a reply to message #19588] |
Thu, 31 March 2011 14:17 |
Michael Galloy
Messages: 1114
Registered: April 2006
|
Senior Member |
|
|
On 3/31/11 12:48 PM, Kenneth P. Bowman wrote:
> I have never had much use for a completely general-purpose
> routine that reads an entire file, because, among other things,
> the files are often larger than system memory. Adding the
> capability to read just parts of a file will make the program
> much more complex. At that point I find it easier to just
> write the code to get what I want.
I have a program that you can read just parts of a netCDF file, since
that is usually what I have to do. This works like:
IDL> im = tx_nc_getdata(sample_filename, '/image')
or even,
IDL> plot, tx_nc_getdata(sample_filename, '/image[*, 256]')
You can read attributes too:
title = tx_nc_getdata(sample_filename, '/image.TITLE')
I have a TX_NC_DUMP routine that prints a listing of the contents of a
file and a TX_NC_BROWSER that is like H5_BROWSER.
Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation
|
|
|
|
| Re: NetCDF tools [message #75576 is a reply to message #19588] |
Thu, 31 March 2011 11:48 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article
<ff693c57-2325-432f-bb05-705bd303932a@q12g2000prb.googlegroups.com>,
Ed Hyer <ejhyer@gmail.com> wrote:
> Hi guys,
>
> I'm migrating some ugly FORTRAN binary blobs used by some applications
> into shiny new NetCDF files, and I realize I've never worked with
> NetCDF in IDL before.
> My first stop, as usual, was at Coyote, and it looks like the Coyote
> NetCDF object can be used to build whatever edifice I need.
>
> But before I get to building, I wanted to inquire of the assembled: is
> there a NetCDF equivalent to Liam Gumley's fantastic SDS_READ program
> for HDF files?
> If the answer is no, I may just be tempted to port that program to
> NetCDF, because I have become so used to it.
>
> What would be absolutely fantastic is a file that vacuums up all the
> attributes and data in a NetCDF file and returns a structure with
> everything, but I don't think that could be done without lots of
> EXECUTE(), which makes the baby Cthulhu cry.
>
> --Edward H.
This function will read a variable and its attributes and return
a data structure.
It is slightly specialized for my use. I like to include the
number of elements in the 'n' field, but that can potentially
conflict with an existing attribute name.
To get all the variables in a file you would need to make a
wrapper function that will get the list of variables and
loop over them.
I have never had much use for a completely general-purpose
routine that reads an entire file, because, among other things,
the files are often larger than system memory. Adding the
capability to read just parts of a file will make the program
much more complex. At that point I find it easier to just
write the code to get what I want.
Ken Bowman
FUNCTION NCDF_READ_VAR, id, name, _EXTRA = extra
;+
; Name:
; NCDF_READ_VAR
; Purpose:
; This function reads a variable and all of its attributes from an open
; netCDF file and returns the variable values and attributes in a structure.
; Category:
; NCDF utility.
; Calling sequence:
; data = NCDF_READ_VAR(id, name)
; Input:
; id : netCDF file id of the input file. The file must already be open.
; name : variable id or name of the variable to be read. The variable must
; exist in the file.
; Output:
; Structure containing requested variable values and all attributes.
; Keywords:
; If present, the keywords COUNT, OFFSET, and STRIDE are passed to NCDF_VARGET.
; Author:
; K. Bowman. 2004-05-22.
;-
COMPILE_OPT IDL2 ;Set compile options
IF (N_PARAMS() NE 2) THEN $ ;Check parameters
MESSAGE, 'You must specify a file id and variable id or name.'
var_info = NCDF_VARINQ(id, name) ;Get variable info
NCDF_VARGET, id, name, values, _EXTRA = extra ;Read data
IF (var_info.datatype EQ 'CHAR') THEN values = STRING(values) ;If CHAR, convert to STRING
IF (var_info.natts GT 0) THEN BEGIN ;Read attributes
att_name = NCDF_ATTNAME(id, var_info.name, 0) ;Get first attribute name
att_info = NCDF_ATTINQ(id, var_info.name, att_name) ;Get first attribute info
NCDF_ATTGET, id, name, att_name, att ;Read first attribute value
IF (att_info.datatype EQ 'CHAR') THEN att = STRING(att) ;Convert CHAR to STRING
atts = CREATE_STRUCT(att_name, att) ;Create attribute structure
FOR i = 1, var_info.natts - 1 DO BEGIN ;For each attribute
att_name = NCDF_ATTNAME(id, var_info.name, i) ;Get attribute name
att_info = NCDF_ATTINQ(id, var_info.name, att_name) ;Get attribute info
NCDF_ATTGET, id, name, att_name, att ;Read attribute value
IF (att_info.datatype EQ 'CHAR') THEN att = STRING(att) ;Convert CHAR to STRING
IF (att_name NE 'n') THEN atts = CREATE_STRUCT(atts, att_name, att) ;Add attribute to structure, skip attributes named 'n'
ENDFOR
data = CREATE_STRUCT('name', var_info.name, $ ;Create structure
'values', values, $
'n', N_ELEMENTS(values), $
atts)
ENDIF ELSE BEGIN
data = CREATE_STRUCT('name', var_info.name, $ ;Create structure
'values', values, $
'n', N_ELEMENTS(values))
ENDELSE
RETURN, data ;Return data structure
END
|
|
|
|
| Re: NetCDF tools [message #75577 is a reply to message #19588] |
Thu, 31 March 2011 11:59 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Ed Hyer writes:
> I'm migrating some ugly FORTRAN binary blobs used by some applications
> into shiny new NetCDF files, and I realize I've never worked with
> NetCDF in IDL before.
> My first stop, as usual, was at Coyote, and it looks like the Coyote
> NetCDF object can be used to build whatever edifice I need.
>
> But before I get to building, I wanted to inquire of the assembled: is
> there a NetCDF equivalent to Liam Gumley's fantastic SDS_READ program
> for HDF files?
> If the answer is no, I may just be tempted to port that program to
> NetCDF, because I have become so used to it.
>
> What would be absolutely fantastic is a file that vacuums up all the
> attributes and data in a NetCDF file and returns a structure with
> everything, but I don't think that could be done without lots of
> EXECUTE(), which makes the baby Cthulhu cry.
Well, there are no EXECUTEs, but that's more or less what
the ncdf_data object does. It is usually accessed via the
NCDF_BROWSER. The "Read Entire File" button is the one you
want. It just calls the Read_File method and returns a
structure will all the file information in it. :-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
