| plotting missing values [message #86781] |
Tue, 03 December 2013 03:09  |
Krishnapriya M
Messages: 3
Registered: December 2013
|
Junior Member |
|
|
|
I just tried to plot a number of ncdf data sets using idl. Now I want to find where the missing values and also want to plot the values and missing values in the given time. Can u help me to solve this problem.
|
|
|
|
|
|
|
|
|
|
|
|
| Re: plotting missing values [message #86793 is a reply to message #86792] |
Wed, 04 December 2013 05:22  |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
Hello,
On 12/04/13 03:57, Fabien wrote:
> On 04.12.2013 02:00, David Fanning wrote:
>> Well, I've made changes over time that people who "say" they using
>> netCDF4 files have been happy with. I'm not currently working with
>> netCDF4 files, so I don't know for sure whether more is needed.
>
> NCDF_FILE is compatible with netCDF4 features (read, create, compression
> options). I just tried NCDF_BROWSER and it seems to work on my NetCDF4
> files too.
My netcdf4 files are replete with groups. Each group is self contained
and written by a separate object method - either as a subgroup (or
subsubgroup), or as a single file in its own right (all the processing
and writing code is Fortran2003, not IDL).
This flexibility is needed since all the subgroups are optional (e.g.
microwave sensors won't have non-LTE data, and infrared sensors won't
have antenna correction data).
All the datafile viewing code is IDL - but my simple netCDF reader
doesn't grok netcdf4.
Here is a (heavily truncated) ncdump of one of my files:
-----%<-----
netcdf amsua_metop-a.SpcCoeff {
// global attributes:
:write_module_history = "$Id: ..." ;
:creation_date_and_time = "2013/10/16, ..." ;
:Release = 9 ;
:Version = 1 ;
:Title = "Spectral coefficients for MetOp-A AMSU-A" ;
:History = "$Id: ..." ;
:Comment = "Converted from ..." ;
group: spccoeff {
group: sensorinfo {
} // group sensorinfo
group: accoeff {
group: sensorinfo {
} // group sensorinfo
} // group accoeff
} // group spccoeff
}
-----%<-----
Only the global attributes are read in with ncdf_data. When I click the
"Read Entire File" button I get:
IDL> ncdf_browser, 'amsua_metop-a.SpcCoeff.nc'
IDL>
A variable named "data" has been created at the main IDL level.
IDL> help, data
** Structure <1d6dc28>, 2 tags, length=120, data length=120, refs=1:
_FILENAME STRING 'amsua_metop-a.SpcCoeff.nc'
_GLOBAL_ATTR STRUCT -> <Anonymous> Array[1]
IDL> help, data._global_attr
** Structure <2013138>, 8 tags, length=104, data length=104, refs=2:
NCDF_FILENAME STRING 'amsua_metop-a.SpcCoeff.nc'
WRITE_MODULE_HISTORY
STRING '$Id: SpcCoeff_netCDF_IO.f90 '...
CREATION_DATE_AND_TIME
STRING '2013/10/16, 15:05:07 -0400UTC'
RELEASE LONG 9
VERSION LONG 1
TITLE STRING 'Spectral coefficients for '...
HISTORY STRING '$Id: SpcCoeff_Convert.f90 32636'...
COMMENT STRING 'Converted from Release-8 format'...
cheers,
paulv
|
|
|
|
|
|
|
|
| Re: plotting missing values [message #86798 is a reply to message #86797] |
Wed, 04 December 2013 07:45 |
Fabzi
Messages: 305
Registered: July 2010
|
Senior Member |
|
|
Hi David,
On 04.12.2013 16:41, David Fanning wrote:
> Does anyone know if there is a simple test I can perform in IDL
> to tell if the file is a netCDF3 as opposed to a netCDF4 file?
I had a support request on this a couple of months ago. It will be
included in IDL 8.3, I've been told. In the mean time they gave me
following code, which works:
;+
; :Description:
; Checks the format of a NCDF file without opening it
; (CLASSIC, 64BIT or NETCDF4)
;
; :Returns:
; A string::
; - 'FORMAT_CLASSIC'
; - 'FORMAT_64BIT'
; - 'FORMAT_NETCDF4'
; - 'Unknown'
;
; :Params:
; filename: in, string
; the path to the file to check
;
; :Author: Anonymous Developer from Exelis
;-
function w_ncdf_format, filename
ON_ERROR, 2
openr, unit, filename, /get_lun
data = bytarr(4)
readu, unit, data
free_lun, unit
case string(data) of
string([67b, 68b, 70b, 1b]): return, 'FORMAT_CLASSIC'
string([67b, 68b, 70b, 2b]): return, 'FORMAT_64BIT'
string([137b, 72b, 68b, 70b]): return, 'FORMAT_NETCDF4'
else: return, 'UNKNOWN'
endcase
end
|
|
|
|
|
|
|
|
| Re: plotting missing values [message #86801 is a reply to message #86799] |
Wed, 04 December 2013 08:27 |
chris_torrence@NOSPAM
Messages: 528
Registered: March 2007
|
Senior Member |
|
|
On Wednesday, December 4, 2013 9:22:28 AM UTC-7, David Fanning wrote:
> Paul van Delst writes:
>
>
>
>>> I'd have to have a look. Can you send me a file of the type you think
>
>>> doesn't work well with the software? I'll have a look and see how much
>
>>> time I think it will take to make the software compatible with the file.
>
>>
>
>> Done.
>
>
>
> The netCDF4 file you sent me can be browsed with the H5_Browser supplied
>
> with IDL. If the good folks at ExelisVis would give me the code for that
>
> browser, I'd fix it so that it worked more intuitively. ;-)
>
>
>
> So, I guess the question is, do we need a better HDF5/netCDF4 browser or
>
> do we just need a better way to get at the data in these files? I
>
> haven't yet looked at Mike Galloy's routines, but these are generally
>
> extremely reliable for these kinds of things.
>
>
>
> Cheers,
>
>
>
> David
>
> --
>
> David Fanning, Ph.D.
>
> Fanning Software Consulting, Inc.
>
> Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
>
> Sepore ma de ni thue. ("Perhaps thou speakest truth.")
Hi David,
The h5_browser.pro should be in your lib directory. Feel free to hack away!
-Chris
p.s. the hdf_browser is shipped as a save file. Not sure why...
|
|
|
|
| Re: plotting missing values [message #86802 is a reply to message #86799] |
Wed, 04 December 2013 08:45 |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
Hello,
On 12/04/13 11:22, David Fanning wrote:
> Paul van Delst writes:
>
>>> I'd have to have a look. Can you send me a file of the type you think
>>> doesn't work well with the software? I'll have a look and see how much
>>> time I think it will take to make the software compatible with the file.
>>
>> Done.
>
> The netCDF4 file you sent me can be browsed with the H5_Browser supplied
> with IDL. If the good folks at ExelisVis would give me the code for that
> browser, I'd fix it so that it worked more intuitively. ;-)
>
> So, I guess the question is, do we need a better HDF5/netCDF4 browser or
> do we just need a better way to get at the data in these files? I
The latter. Browsing is nice, but being able to read the data simply,
over and over again, in a production-like setting is the goal.
Typically, I know what the variables names are. Once the data has been
read into a structure/object, getting the particular variable is trivial
(e.g. via a tag name match, or get_property method).
> haven't yet looked at Mike Galloy's routines, but these are generally
> extremely reliable for these kinds of things.
?
Mike has netCDF4 readers?
Crikey, I can't keep up!
cheers,
paulv
|
|
|
|
| Re: plotting missing values [message #86803 is a reply to message #86799] |
Wed, 04 December 2013 08:58 |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
Hello,
On 12/04/13 11:22, David Fanning wrote:
> Paul van Delst writes:
>
>>> I'd have to have a look. Can you send me a file of the type you think
>>> doesn't work well with the software? I'll have a look and see how much
>>> time I think it will take to make the software compatible with the file.
>>
>> Done.
>
> The netCDF4 file you sent me can be browsed with the H5_Browser supplied
> with IDL. If the good folks at ExelisVis would give me the code for that
> browser, I'd fix it so that it worked more intuitively. ;-)
>
> So, I guess the question is, do we need a better HDF5/netCDF4 browser or
> do we just need a better way to get at the data in these files? I
> haven't yet looked at Mike Galloy's routines, but these are generally
> extremely reliable for these kinds of things.
Based on the to-and-fro about HDF5, I tried the following:
IDL> result=h5_parse('amsua_metop-a.SpcCoeff.nc',/read_data)
and bugger me if it didn't work, making all the groups accessible:
IDL> help, result
** Structure <1e95a58>, 14 tags, length=33952, data length=33548, refs=1:
_NAME STRING 'amsua_metop-a.SpcCoeff.nc'
_ICONTYPE STRING 'hdf'
_TYPE STRING 'GROUP'
_FILE STRING 'amsua_metop-a.SpcCoeff.nc'
_PATH STRING '/'
_COMMENT STRING ''
SPCCOEFF STRUCT -> <Anonymous> Array[1]
WRITE_MODULE_HISTORY
STRUCT -> <Anonymous> Array[1]
CREATION_DATE_AND_TIME
STRUCT -> <Anonymous> Array[1]
RELEASE STRUCT -> <Anonymous> Array[1]
VERSION STRUCT -> <Anonymous> Array[1]
TITLE STRUCT -> <Anonymous> Array[1]
HISTORY STRUCT -> <Anonymous> Array[1]
COMMENT STRUCT -> <Anonymous> Array[1]
IDL> help, result.spccoeff
** Structure <208a1e8>, 19 tags, length=33032, data length=32628, refs=2:
_NAME STRING 'spccoeff'
_ICONTYPE STRING ''
_TYPE STRING 'GROUP'
_FILE STRING 'amsua_metop-a.SpcCoeff.nc'
_PATH STRING '/'
_COMMENT STRING ''
BAND_C1 STRUCT -> <Anonymous> Array[1]
BAND_C2 STRUCT -> <Anonymous> Array[1]
CHANNEL_FLAG STRUCT -> <Anonymous> Array[1]
COSMIC_BACKGROUND_RADIANCE
STRUCT -> <Anonymous> Array[1]
FREQUENCY STRUCT -> <Anonymous> Array[1]
PLANCK_C1 STRUCT -> <Anonymous> Array[1]
PLANCK_C2 STRUCT -> <Anonymous> Array[1]
SENSOR_CHANNEL STRUCT -> <Anonymous> Array[1]
SOLAR_IRRADIANCE
STRUCT -> <Anonymous> Array[1]
WAVENUMBER STRUCT -> <Anonymous> Array[1]
ACCOEFF STRUCT -> <Anonymous> Array[1]
N_CHANNELS STRUCT -> <Anonymous> Array[1]
SENSORINFO STRUCT -> <Anonymous> Array[1]
IDL> help, result.spccoeff.accoeff
** Structure <1d538e8>, 13 tags, length=14608, data length=14424, refs=2:
_NAME STRING 'accoeff'
_ICONTYPE STRING ''
_TYPE STRING 'GROUP'
_FILE STRING 'amsua_metop-a.SpcCoeff.nc'
_PATH STRING '/spccoeff'
_COMMENT STRING ''
A_EARTH STRUCT -> <Anonymous> Array[1]
A_PLATFORM STRUCT -> <Anonymous> Array[1]
A_SPACE STRUCT -> <Anonymous> Array[1]
SENSOR_CHANNEL STRUCT -> <Anonymous> Array[1]
N_CHANNELS STRUCT -> <Anonymous> Array[1]
N_FOVS STRUCT -> <Anonymous> Array[1]
SENSORINFO STRUCT -> <Anonymous> Array[1]
IDL> help, result.spccoeff.accoeff.sensorinfo
** Structure <1e82768>, 20 tags, length=9976, data length=9876, refs=2:
_NAME STRING 'sensorinfo'
_ICONTYPE STRING ''
_TYPE STRING 'GROUP'
_FILE STRING 'amsua_metop-a.SpcCoeff.nc'
_PATH STRING '/spccoeff/accoeff'
_COMMENT STRING ''
NOISE STRUCT -> <Anonymous> Array[1]
POLARIZATION_TYPE
STRUCT -> <Anonymous> Array[1]
SATELLITE_NAME STRUCT -> <Anonymous> Array[1]
SENSOR_CHANNEL STRUCT -> <Anonymous> Array[1]
SENSOR_ID STRUCT -> <Anonymous> Array[1]
SENSOR_NAME STRUCT -> <Anonymous> Array[1]
SENSOR_TYPE STRUCT -> <Anonymous> Array[1]
USE_FLAG STRUCT -> <Anonymous> Array[1]
WMO_SATELLITE_ID
STRUCT -> <Anonymous> Array[1]
WMO_SENSOR_ID STRUCT -> <Anonymous> Array[1]
DL_STRLEN STRUCT -> <Anonymous> Array[1]
N_CHANNELS STRUCT -> <Anonymous> Array[1]
N_FOVS STRUCT -> <Anonymous> Array[1]
SL_STRLEN STRUCT -> <Anonymous> Array[1]
And the data is there:
IDL> help, result.spccoeff.frequency
** Structure <1ea1388>, 18 tags, length=856, data length=848, refs=2:
_NAME STRING 'Frequency'
_ICONTYPE STRING 'binary'
_TYPE STRING 'DATASET'
_FILE STRING 'amsua_metop-a.SpcCoeff.nc'
_PATH STRING '/spccoeff'
_DATA DOUBLE Array[15]
_NDIMENSIONS LONG 1
_DIMENSIONS ULONG64 Array[1]
_NELEMENTS ULONG64 15
_DATATYPE STRING 'H5T_FLOAT'
_STORAGESIZE ULONG 8
_PRECISION LONG 64
_SIGN STRING ''
LONG_NAME STRUCT -> <Anonymous> Array[1]
DESCRIPTION STRUCT -> <Anonymous> Array[1]
UNITS STRUCT -> <Anonymous> Array[1]
_FILLVALUE STRUCT -> <Anonymous> Array[1]
DIMENSION_LIST STRUCT -> <Anonymous> Array[1]
IDL> print, result.spccoeff.frequency._data
23.800904 31.400728 50.300069 52.799890
53.596155 54.400633 54.940002 55.499802
57.290327 57.290327 57.290327 57.290327
57.290327 57.290327 88.997000
Yay.
So I guess netcdf4 files are just hdf5 files with a fancier name?
cheers,
paulv
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Re: plotting missing values [message #86815 is a reply to message #86814] |
Wed, 04 December 2013 17:16 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Michael Galloy writes:
> Yes, my understanding is netCDF 4 files are just HDF 5 files with some
> naming conventions. netCDF 4 is not a file type; it is an API.
What does "naming conventions" mean? Is this like conventions to make
the file CF-compliant?
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
