Fanning Software Consulting

Browsing and Reading netCDF Files in IDL

QUESTION: Is there an easy way in IDL to find out what is inside a netCDF file, and to read the data and metadata (attributes) from the file? The scientific data formats seem complicated.

ANSWER: It is true that the scientific data formats (netCDF, CDF, and HDF) seem complicated, but they are less so after you spend a couple of days working with them. But even then, there are surprises.

I have been working with netCDF files recently and, like you, have found it difficult to know what is in the file. Most of the available IDL software I found on the Internet to read netCDF files required that I know the name of the variables in the files, and I don't always. So I wrote my own netCDF Browser to allow me to easily see what is in a netCDF file and to quickly read the data and metadata into IDL variables. You will find the netCDF Browser among the routines in the Coyote Library.

The Unidata folks, who maintain the netCDF library and netCDF standards, provide example netCDF data files for downloading. I have downloaded one such example file to use in this article.

I have put the ability to read and browse netCDF data files into an object of class NCDF_DATA. This gives me great flexibility in how I work with netCDF files, but at the cost of greater complexity. It is possible to use the object to work with the netCDF files through what I call a Browser Window or graphical user interface, but it is also possible to work with netCDF files in a programmable and non-interactive way, using the object's methods to read the data and metadata.

Most users, however, will probably want to use the IDL procedure front-end to the object, which avoids the complexities of object manipulation in IDL, and provides a straightforward and intuitive interface to the netCDF file. You can see an example of the Browser Window interface in the figure below, with the example netCDF file from above loaded into the object. The code to start this interface looks like this.

   IDL> file = 'WMI_Lear.nc'
   IDL> nCDF_Browser, file
The netCDF Browser Window interface.
The netCDF Browser Window interface.
 

The user clicks on the representation of the data and metadata in the window on the left, and information about the data or metadata appears in the window on the right. Buttons are available to read single variables, or the entire data file at once. When variables are read, they are read into IDL variables at the main IDL level. As shown in the figure below, the user has the opportunity to name the variables as he or she wishes. New data files can be loaded into the browser at any time by choosing the Open New File button. The user is permitted to have multiple Browser Windows open at once.

The netCDF Browser Window interface.
Individual variables or the entire data file can be read into IDL variables at the main IDL level. New data files can be opened from within the interface.
 

The netCDF file browser programs require other programs from the Coyote Library, so you must download the entire library. Additional documentation is provided in the code itself.

Additionally, Ben Tupper has written a 15 line program that saves three lines of typing, so that you can use the ncdf_data object in an IDL function. He kindly named the program FSC_READ_NCDF.

Updates

Note: The nCDF_Browser has been updated to also read HDF 4 files that contain scientific datasets in them. Other HDF file types (vdata, etc.) are not currently supported.

Further Updates

I had a general need to be able to create netCDF files in an extremely flexible way. I wanted to be able to copy global attributes, dimensions, and variables from one file to another, while still adding information to the new file. To this end, I borrowed heavily (well, I stole most of the good ideas, really) from Mark Hadfield's Motley Library and its wonderful netCDF routines. I wanted to do this primarily because I thought Mark's routines needed better documenation and error handling. In the end, I came to the conclusion that quite a lot of the work I went to in creating these new routines could have been saved by spending more time with Mark's routines. But, of course, hindsight is 20-20.

The point is, I created a set of objects for working with netCDF files, that makes reading, creating, and modifying netCDF files extremely easy for me. If you are unfamiliar with objects, you may be intimidated to try these. You shouldn't be. They are extremely easy to use. To help you along, I have provided an example program that shows you how to create a netCDF file, how to copy the information in this new netCDF file to another file, and finally, how to read the information in this netCDF file to do something useful in IDL. The example file is named NCDF_File_Examples. I think you will find these new files well documented and easy to use. All of the files are available in the NCDF_Tools zip file, which is a subset of the programs in the Coyote Library.

Version of IDL used to prepare this article: IDL 7.0.1.

Google
 
Web Coyote's Guide to IDL Programming