| Re: NCDF_ATTPUT _FillValue problem for string arrays? [message #67134] |
Tue, 07 July 2009 00:17  |
Foldy Lajos
Messages: 268
Registered: October 2001
|
Senior Member |
|
|
On Mon, 6 Jul 2009, Paul van Delst wrote:
> AHA!!!
>
> When I change your above test code to use the conventional fill value
> attribute name as specified in the netCDF Interface Guide, "_FillValue", I
> get the following:
>
> ncid=ncdf_create('test.nc', /clobber)
> dim32=ncdf_dimdef(ncid, 'dim1', 32)
> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
> varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
> ncdf_attput, ncid, varid, '_FillValue', ' ' ; <---**** Note the attribute name
> ncdf_control, ncid, /endef
> ncdf_close, ncid
>
> IDL> .run blah
> % Compiled module: $MAIN$.
> % NCDF_CONTROL: Attempt to take the file out of define mode (ENDEF) failed.
> % (NC_ERROR=-45)
> % Execution halted at: $MAIN$ 6 scratch/blah.pro
>
>
> So:
> - if I use "_fillvalue" for the attribute name, the code works fine.
> - if I use "_FillValue" for the attribute name, the code crashes.
>
I have found the solution: _FillValue needs to be the same type as the
variable. ' ' is an IDL string, which is converted to something, probably
BYTE. So let's add an explicite /CHAR to ncdf_attput:
ncid=ncdf_create('test.nc', /clobber)
dim32=ncdf_dimdef(ncid, 'dim1', 32)
n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
ncdf_attput, ncid, varid, '_FillValue', ' ', /CHAR
ncdf_control, ncid, /endef
ncdf_close, ncid
It works now! :-)
regards,
lajos
|
|
|
|
| Re: NCDF_ATTPUT _FillValue problem for string arrays? [message #67140 is a reply to message #67134] |
Mon, 06 July 2009 14:41  |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
F�LDY Lajos wrote:
>
>
>
> NCDF_VARDEF needs dimension ID, not dimension size. The following code
> works for me:
>
> ncid=ncdf_create('test.nc', /clobber)
> dim32=ncdf_dimdef(ncid, 'dim1', 32)
> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
> varid = ncdf_vardef( ncid, 'absorber_units_name',
> [dim32,n_absorbers_dimid], /char)
> ncdf_attput, ncid, varid, '_fillvalue', ' '
> ncdf_control, ncid, /endef
> ncdf_close, ncid
Just a final followup - I changed the definition of the offending attribute to
"_fillvalue" in my production code and now the ncdf_attput works as expected.
thanks again. I have little hair left to tear out when these annoyances arise so you've
saved me a tuft or two! :o)
cheers,
paulv
p.s. I should note that the futzing up of the _fillvalue attribute *only* occurred for
character variable output. Any other datatype (byte, long, float, double etc) handled the
camel case attribute name fine. Weird.
|
|
|
|
| Re: NCDF_ATTPUT _FillValue problem for string arrays? [message #67144 is a reply to message #67140] |
Mon, 06 July 2009 13:41 |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
F�LDY Lajos wrote:
>
>
> On Mon, 6 Jul 2009, Paul van Delst wrote:
>
>> Hello,
>>
>> I've encountered a strange problem with some netCDF output using IDL.
>>
>> I define a character string array variable in my output netCDF file
>> like so:
>>
>> VarId = NCDF_VARDEF( fid, 'Absorber_Units_Name',
>> [32,n_Absorbers_DimId], /CHAR)
>>
>> where the "32" is the maximum string length.
>>
>> Now if I try to set a fill value attribute for that variable, like so
>>
>> NCDF_ATTPUT, fid, VarId, '_FillValue', ' '
>> or
>> NCDF_ATTPUT, fid, VarId, '_FillValue', 0B
>>
>> when I run the code I get the following error when I take the file out
>> of define mode:
>>
>> % NCDF_CONTROL: Attempt to take the file out of define mode (ENDEF)
>> failed. (NC_ERROR=-45)
>>
>> If I simply comment out the call to NCDF_ATTPUT for these character
>> variables, there's no problem.
>>
>> I do this sort of thing (i.e. _FillValue of " " for character string
>> variables) all the time using the Fortran90 API to netCDF so I assume
>> an empty space is a valid fill value.
>>
>> Has anyone else encountered this behaviour in IDL? I.e. is this a
>> known bug in the IDL netCDF interface, or do I need to do something
>> special with character fill values?
>>
>> Thanks for any info.
>>
>> cheers,
>>
>> paulv
>>
>>
>
> NCDF_VARDEF needs dimension ID, not dimension size. The following code
> works for me:
Yeah, I fat-fingered my code when I typed the message. Cutting-and-pasting directly from
my script I have:
; ...Define dimensions
n_Levels_DimId = NCDF_DIMDEF(fid, LEVEL_DIMNAME , self.n_Levels)
n_Layers_DimId = NCDF_DIMDEF(fid, LAYER_DIMNAME , self.n_Layers)
n_Absorbers_DimId = NCDF_DIMDEF(fid, ABSORBER_DIMNAME , self.n_Absorbers)
pdsl_DimId = NCDF_DIMDEF(fid, DESCRIPTION_DIMNAME , PDSL)
aunsl_DimId = NCDF_DIMDEF(fid, ABSORBER_UNITS_DIMNAME, AUNSL)
n_Profiles_DimId = NCDF_DIMDEF(fid, PROFILE_DIMNAME , /UNLIMITED)
; Define variables
VarId = NCDF_VARDEF( fid, DESCRIPTION_VARNAME, [pdsl_DimId,n_Profiles_DimId], /CHAR)
NCDF_ATTPUT, fid, VarId, LONGNAME_ATTNAME , DESCRIPTION_LONGNAME
NCDF_ATTPUT, fid, VarId, DESCRIPTION_ATTNAME, DESCRIPTION_DESCRIPTION
NCDF_ATTPUT, fid, VarId, UNITS_ATTNAME , DESCRIPTION_UNITS
; NCDF_ATTPUT, fid, VarId, FILLVALUE_ATTNAME , DESCRIPTION_FILLVALUE
where I have a parameters include file containing:
LONGNAME_ATTNAME = 'long_name'
DESCRIPTION_ATTNAME = 'description'
UNITS_ATTNAME = 'units'
FILLVALUE_ATTNAME = '_FillValue'
DESCRIPTION_LONGNAME = 'Profile Description'
DESCRIPTION_DESCRIPTION = 'Description of atmospheric profile and modification'
DESCRIPTION_UNITS = 'N/A'
DESCRIPTION_FILLVALUE = ' '
....etc for other variables
Your example code works for me too. Hmmm....
>
> ncid=ncdf_create('test.nc', /clobber)
> dim32=ncdf_dimdef(ncid, 'dim1', 32)
> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
> varid = ncdf_vardef( ncid, 'absorber_units_name',
> [dim32,n_absorbers_dimid], /char)
> ncdf_attput, ncid, varid, '_fillvalue', ' '
> ncdf_control, ncid, /endef
> ncdf_close, ncid
AHA!!!
When I change your above test code to use the conventional fill value attribute name as
specified in the netCDF Interface Guide, "_FillValue", I get the following:
ncid=ncdf_create('test.nc', /clobber)
dim32=ncdf_dimdef(ncid, 'dim1', 32)
n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
ncdf_attput, ncid, varid, '_FillValue', ' ' ; <---**** Note the attribute name
ncdf_control, ncid, /endef
ncdf_close, ncid
IDL> .run blah
% Compiled module: $MAIN$.
% NCDF_CONTROL: Attempt to take the file out of define mode (ENDEF) failed. (NC_ERROR=-45)
% Execution halted at: $MAIN$ 6 scratch/blah.pro
So:
- if I use "_fillvalue" for the attribute name, the code works fine.
- if I use "_FillValue" for the attribute name, the code crashes.
Given that netCDF attribute, dimension, and variable names are case-sensitive, this would
appear to be a bug....somehow.
Thanks for writing the little test case. I never would have figured it out otherwise.
cheers,
paulv
>
> and 'ncdump test.nc' produces:
>
> netcdf test {
> dimensions:
> dim1 = 32 ;
> dim2 = 9 ;
> variables:
> char absorber_units_name(dim2, dim1) ;
> absorber_units_name:_fillvalue = " " ;
> data:
>
> absorber_units_name =
> "",
> "",
> "",
> "",
> "",
> "",
> "",
> "",
> "" ;
> }
>
> NCDF_VARDEF does not print an error message, it returns -1 on failure :-(
>
> regards,
> lajos
|
|
|
|
| Re: NCDF_ATTPUT _FillValue problem for string arrays? [message #67145 is a reply to message #67144] |
Mon, 06 July 2009 12:38 |
Foldy Lajos
Messages: 268
Registered: October 2001
|
Senior Member |
|
|
On Mon, 6 Jul 2009, Paul van Delst wrote:
> Hello,
>
> I've encountered a strange problem with some netCDF output using IDL.
>
> I define a character string array variable in my output netCDF file like so:
>
> VarId = NCDF_VARDEF( fid, 'Absorber_Units_Name', [32,n_Absorbers_DimId], /CHAR)
>
> where the "32" is the maximum string length.
>
> Now if I try to set a fill value attribute for that variable, like so
>
> NCDF_ATTPUT, fid, VarId, '_FillValue', ' '
> or
> NCDF_ATTPUT, fid, VarId, '_FillValue', 0B
>
> when I run the code I get the following error when I take the file out of
> define mode:
>
> % NCDF_CONTROL: Attempt to take the file out of define mode (ENDEF) failed.
> (NC_ERROR=-45)
>
> If I simply comment out the call to NCDF_ATTPUT for these character
> variables, there's no problem.
>
> I do this sort of thing (i.e. _FillValue of " " for character string
> variables) all the time using the Fortran90 API to netCDF so I assume an
> empty space is a valid fill value.
>
> Has anyone else encountered this behaviour in IDL? I.e. is this a known bug
> in the IDL netCDF interface, or do I need to do something special with
> character fill values?
>
> Thanks for any info.
>
> cheers,
>
> paulv
>
>
NCDF_VARDEF needs dimension ID, not dimension size. The following code
works for me:
ncid=ncdf_create('test.nc', /clobber)
dim32=ncdf_dimdef(ncid, 'dim1', 32)
n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
ncdf_attput, ncid, varid, '_fillvalue', ' '
ncdf_control, ncid, /endef
ncdf_close, ncid
and 'ncdump test.nc' produces:
netcdf test {
dimensions:
dim1 = 32 ;
dim2 = 9 ;
variables:
char absorber_units_name(dim2, dim1) ;
absorber_units_name:_fillvalue = " " ;
data:
absorber_units_name =
"",
"",
"",
"",
"",
"",
"",
"",
"" ;
}
NCDF_VARDEF does not print an error message, it returns -1 on failure :-(
regards,
lajos
|
|
|
|
| Re: NCDF_ATTPUT _FillValue problem for string arrays? [message #67200 is a reply to message #67134] |
Wed, 08 July 2009 12:12 |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
F�LDY Lajos wrote:
>
> On Mon, 6 Jul 2009, Paul van Delst wrote:
>
>> AHA!!!
>>
>> When I change your above test code to use the conventional fill value
>> attribute name as specified in the netCDF Interface Guide,
>> "_FillValue", I get the following:
>>
>> ncid=ncdf_create('test.nc', /clobber)
>> dim32=ncdf_dimdef(ncid, 'dim1', 32)
>> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
>> varid = ncdf_vardef( ncid, 'absorber_units_name',
>> [dim32,n_absorbers_dimid], /char)
>> ncdf_attput, ncid, varid, '_FillValue', ' ' ; <---**** Note the
>> attribute name
>> ncdf_control, ncid, /endef
>> ncdf_close, ncid
>>
>> IDL> .run blah
>> % Compiled module: $MAIN$.
>> % NCDF_CONTROL: Attempt to take the file out of define mode (ENDEF)
>> failed. % (NC_ERROR=-45)
>> % Execution halted at: $MAIN$ 6 scratch/blah.pro
>>
>>
>> So:
>> - if I use "_fillvalue" for the attribute name, the code works fine.
>> - if I use "_FillValue" for the attribute name, the code crashes.
>>
>
> I have found the solution: _FillValue needs to be the same type as the
> variable. ' ' is an IDL string, which is converted to something,
> probably BYTE. So let's add an explicite /CHAR to ncdf_attput:
>
> ncid=ncdf_create('test.nc', /clobber)
> dim32=ncdf_dimdef(ncid, 'dim1', 32)
> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
> varid = ncdf_vardef( ncid, 'absorber_units_name',
> [dim32,n_absorbers_dimid], /char)
> ncdf_attput, ncid, varid, '_FillValue', ' ', /CHAR
> ncdf_control, ncid, /endef
> ncdf_close, ncid
>
> It works now! :-)
Yes, it does for me also.
You, sir, are a star!
Thanks,
paulv
|
|
|
|
| Re: NCDF_ATTPUT _FillValue problem for string arrays? [message #67212 is a reply to message #67140] |
Wed, 08 July 2009 06:55 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Paul van Delst schrieb:
> F�LDY Lajos wrote:
>>
>>
>>
>> NCDF_VARDEF needs dimension ID, not dimension size. The following code
>> works for me:
>>
>> ncid=ncdf_create('test.nc', /clobber)
>> dim32=ncdf_dimdef(ncid, 'dim1', 32)
>> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
>> varid = ncdf_vardef( ncid, 'absorber_units_name',
>> [dim32,n_absorbers_dimid], /char)
>> ncdf_attput, ncid, varid, '_fillvalue', ' '
>> ncdf_control, ncid, /endef
>> ncdf_close, ncid
>
> Just a final followup - I changed the definition of the offending
> attribute to "_fillvalue" in my production code and now the ncdf_attput
> works as expected.
>
Not really. Now you just use the internal defined _FillValue of the
netCDF format. You have now just added an additional attribute which is
not used as initial value for creating the netCDF variable.
That var is similiar to make_array(10,value=FillValue, /string)
Also it breaks the standard definition of attributes but this would make
only problems when you share your data.
And we have run in this problems too. using a double fill var and having
float arrays makes also some fun. q_icg_struct was changed in 2003 to
care on that.
(since idl6.0 netCDF3.5.0 fill values must checked to be same type as
param. missing value is checked too.)
cheers
Reimar
> thanks again. I have little hair left to tear out when these annoyances
> arise so you've saved me a tuft or two! :o)
>
> cheers,
>
> paulv
>
> p.s. I should note that the futzing up of the _fillvalue attribute
> *only* occurred for character variable output. Any other datatype (byte,
> long, float, double etc) handled the camel case attribute name fine. Weird.
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
