| Can't exit DEFINE mode in netCDF File [message #68119] |
Tue, 06 October 2009 13:42  |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Folks,
I am writing my first netCDF file in IDL and I am practically
copying the code out of Ken Bowman's book. I open a file:
fID = NCDF_CREATE(filename, /CLOBBER)
Then, I define several global attributes, two dimensions,
and two variables. I also define a couple of attributes
of the variables. All in that order.
Then, I attempt to get out of DEFINE mode and into DATA
mode, so I can write the actual variables.
NCDF_CONTROL, fID, /ENDEF
But at this point I get the following error message:
% NCDF_CONTROL: Attempt to take the file out of
define mode (ENDEF) failed. (NC_ERROR=-45)
% Execution halted at: TIMESERIES::CREATE_CLIMATOLOGY 2589
Does anyone have ANY idea what this might be about?
Thanks,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68220 is a reply to message #68119] |
Wed, 07 October 2009 07:30  |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
David Fanning wrote:
> Paul van Delst writes:
>
>> F�LDY Lajos came up with a solution,
>>
>>
>>> I have found the solution: _FillValue needs to be the same type as the
>>> variable. ' ' is an IDL string, which is converted to something, probably
>>> BYTE. So let's add an explicite /CHAR to ncdf_attput:
>>>
>>> ncid=ncdf_create('test.nc', /clobber)
>>> dim32=ncdf_dimdef(ncid, 'dim1', 32)
>>> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
>>> varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
>>> ncdf_attput, ncid, varid, '_FillValue', ' ', /CHAR
>>> ncdf_control, ncid, /endef
>>> ncdf_close, ncid
>>>
>>> It works now! :-)
>>>
>>> regards,
>>> lajos
>> Does that help?
>
> Ah! Yes, I remember reading that and thinking it was too
> esoteric for an article on my web page. I should have known
> better. I won't make that mistake again!
>
> Where is it written, though, that the _FillValue attribute has
> to be the same data type as the variable it is an attribute of?
> And why doesn't *that* cause an error, instead of this action at
> a distance thing?
All good questions. Although I find myself in the strange position of *providing*
David-Fanning-type wisdom (to said namesake no less!) rather than being on the receiving end:
It is like it is because that's the way it is.
:o)
> I'm not sure this was worth three hours of effort, not counting the
> time it will take to write the damn article. :-(
BTW, was it the fillvalue attribute in particular, or some other one? Your example was a
little bit different from mine in that my fillvalue was a character (where I believe yours
was a numeric quantity).
> Thanks for the help.
Ultimate thanks go to Mr. Lajos (or Mr. F�ldy? Don't know the correct order)
cheers,
paulv
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68254 is a reply to message #68119] |
Wed, 07 October 2009 11:22 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article
<1a1cedb2-981d-4b52-af3e-1af44caf7830@x5g2000prf.googlegroups.com>,
"dcleon@gmail.com" <dcleon@gmail.com> wrote:
>> Where is it written, though, that the _FillValue attribute has
>> to be the same data type as the variable it is an attribute of?
>> And why doesn't *that* cause an error, instead of this action at
>> a distance thing?
>
> I've run into this problem several times. I think with the ncdf_
> routines that if the type on any attribute or variable fails to match
> the declared type, that you're going to get an error when you close
> the file. Seems like the ncdf routines should give you the error when
> you define the attribute/variable not down the road when you go to
> close the file. Anyway, the bottom line is check types very carefully
> when writing NetCDF files.
>
> Cheers,
> dave
Well, you learn something new every day (if you're lucky). Or
perhaps unlearn something wrong that you thought you knew.
I always interpreted the NCDF_CONTROL man page to imply that
fill values could not be changed, when it fact it (correctly)
states that the *default* fill values cannot be changed.
Use of the _FillValue attribute is rather well hidden on pages
91 and 133 of the netCDF C-interface documentation
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-c.pd f
I thought this usage was restricted to the netCDF utilities (such as
ncgen) and not a property of the API.
Section 6.30 of the docs does say that errors can result when
attempting to convert a fill value from one type to another.
IDL seems to be reporting this in a particularly opaque
manner. (That's a polite euphemism for damned incomprehensible
IDL error message.)
Ken Bowman
|
|
|
|
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68261 is a reply to message #68119] |
Wed, 07 October 2009 08:52 |
Foldy Lajos
Messages: 268
Registered: October 2001
|
Senior Member |
|
|
On Wed, 7 Oct 2009, dcleon@gmail.com wrote:
>
>> Where is it written, though, that the _FillValue attribute has
>> to be the same data type as the variable it is an attribute of?
>> And why doesn't *that* cause an error, instead of this action at
>> a distance thing?
>
> I've run into this problem several times. I think with the ncdf_
> routines that if the type on any attribute or variable fails to match
> the declared type, that you're going to get an error when you close
> the file. Seems like the ncdf routines should give you the error when
> you define the attribute/variable not down the road when you go to
> close the file. Anyway, the bottom line is check types very carefully
> when writing NetCDF files.
>
> Cheers,
> dave
>
Some ncdf_ routines silently return -1 on error. Users have to check these
return values manually. But I am sure there is some logic behind it, it's
IDL ;-)
regards,
lajos
|
|
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68266 is a reply to message #68119] |
Tue, 06 October 2009 15:33 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Paul van Delst writes:
> FÖLDY Lajos came up with a solution,
>
>
>> I have found the solution: _FillValue needs to be the same type as the
>> variable. ' ' is an IDL string, which is converted to something, probably
>> BYTE. So let's add an explicite /CHAR to ncdf_attput:
>>
>> ncid=ncdf_create('test.nc', /clobber)
>> dim32=ncdf_dimdef(ncid, 'dim1', 32)
>> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
>> varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
>> ncdf_attput, ncid, varid, '_FillValue', ' ', /CHAR
>> ncdf_control, ncid, /endef
>> ncdf_close, ncid
>>
>> It works now! :-)
>>
>> regards,
>> lajos
>
> Does that help?
Ah! Yes, I remember reading that and thinking it was too
esoteric for an article on my web page. I should have known
better. I won't make that mistake again!
Where is it written, though, that the _FillValue attribute has
to be the same data type as the variable it is an attribute of?
And why doesn't *that* cause an error, instead of this action at
a distance thing?
I'm not sure this was worth three hours of effort, not counting the
time it will take to write the damn article. :-(
Thanks for the help.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68267 is a reply to message #68119] |
Tue, 06 October 2009 15:03 |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
David Fanning wrote:
> David Fanning writes:
>
>> Folks,
>>
>> I am writing my first netCDF file in IDL and I am practically
>> copying the code out of Ken Bowman's book. I open a file:
>>
>> fID = NCDF_CREATE(filename, /CLOBBER)
>>
>> Then, I define several global attributes, two dimensions,
>> and two variables. I also define a couple of attributes
>> of the variables. All in that order.
>>
>> Then, I attempt to get out of DEFINE mode and into DATA
>> mode, so I can write the actual variables.
>>
>> NCDF_CONTROL, fID, /ENDEF
>>
>> But at this point I get the following error message:
>>
>> % NCDF_CONTROL: Attempt to take the file out of
>> define mode (ENDEF) failed. (NC_ERROR=-45)
>> % Execution halted at: TIMESERIES::CREATE_CLIMATOLOGY 2589
>>
>> Does anyone have ANY idea what this might be about?
>
> The code is pretty simple:
>
> ; Create a netCDF file for storing the climatology.
> fID = NCDF_Create(filename, /CLOBBER)
>
> ; Define dimensions.
> xsizeID = NCDF_DIMDEF(fID, 'cols', xsize)
> ysizeID = NCDF_DIMDEF(fID, 'rows', ysize)
> timeID = NCDF_DIMDEF(fID, 'time', 2)
>
> ; Define variables.
> vid = NCDF_VARDEF(fID, 'time', [timeID], /LONG)
> vid = NCDF_VARDEF(fID, 'climatology', [xsizeID, ysizeID], /DOUBLE)
>
> ; Define variable attributes.
> NCDF_ATTPUT, fID, 'time', 'units', 'Days since 1601-01-01 00:00:00'
> NCDF_ATTPUT, fID, 'time', 'long_name', 'Start and end time. '
> NCDF_ATTPUT, fID, 'climatology', '_FillValue', fillValue
> NCDF_ATTPUT, fID, 'climatology', 'long_name', 'longish name'
>
> ; Exit DEFINE mode and into DATA mode.
> NCDF_Control, fID, /ENDEF
>
> ; Write the variables.
> NCDF_VarPut, fID, 'time', [time[0], time[timepts-1]]
> NCDF_VarPut, fID, 'climatology', climatology
>
> ; Close the file.
> NCDF_Close, fID
>
> At one time I had global attributes in there, but I took them out
> and the code worked! So I added them back one by one. Then, it
> didn't work again, so I took them all back out. Now it *still*
> doesn't work! Something really, really weird going on here, I think. :-(
I think it's related to the _FillValue global attribute. I can't remember the details, but
I posted here about exactly the same thing a few months ago. Searching the web....
Paul van Delst wrote:
> Hello,
>
> I've encountered a strange problem with some netCDF output using IDL.
>
> I define a character string array variable in my output netCDF file like so:
> VarId = NCDF_VARDEF( fid, 'Absorber_Units_Name', [32,n_Absorbers_DimId], /CHAR)
> where the "32" is the maximum string length.
>
> Now if I try to set a fill value attribute for that variable, like so
> NCDF_ATTPUT, fid, VarId, '_FillValue', ' '
> or
> NCDF_ATTPUT, fid, VarId, '_FillValue', 0B
>
> when I run the code I get the following error when I take the file out of define mode:
>
> % NCDF_CONTROL: Attempt to take the file out of define mode (ENDEF) failed. (NC_ERROR=-45)
>
> If I simply comment out the call to NCDF_ATTPUT for these character variables, there's no
> problem.
>
> I do this sort of thing (i.e. _FillValue of " " for character string variables) all the
> time using the Fortran90 API to netCDF so I assume an empty space is a valid fill value.
>
> Has anyone else encountered this behaviour in IDL? I.e. is this a known bug in the IDL
> netCDF interface, or do I need to do something special with character fill values?
>
> Thanks for any info.
>
> cheers,
>
> paulv
F�LDY Lajos came up with a solution,
> I have found the solution: _FillValue needs to be the same type as the
> variable. ' ' is an IDL string, which is converted to something, probably
> BYTE. So let's add an explicite /CHAR to ncdf_attput:
>
> ncid=ncdf_create('test.nc', /clobber)
> dim32=ncdf_dimdef(ncid, 'dim1', 32)
> n_absorbers_dimid=ncdf_dimdef(ncid, 'dim2', 9)
> varid = ncdf_vardef( ncid, 'absorber_units_name', [dim32,n_absorbers_dimid], /char)
> ncdf_attput, ncid, varid, '_FillValue', ' ', /CHAR
> ncdf_control, ncid, /endef
> ncdf_close, ncid
>
> It works now! :-)
>
> regards,
> lajos
Does that help?
cheers,
paulv
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68268 is a reply to message #68119] |
Tue, 06 October 2009 14:45 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
David Fanning writes:
>
> Folks,
>
> I am writing my first netCDF file in IDL and I am practically
> copying the code out of Ken Bowman's book. I open a file:
>
> fID = NCDF_CREATE(filename, /CLOBBER)
>
> Then, I define several global attributes, two dimensions,
> and two variables. I also define a couple of attributes
> of the variables. All in that order.
>
> Then, I attempt to get out of DEFINE mode and into DATA
> mode, so I can write the actual variables.
>
> NCDF_CONTROL, fID, /ENDEF
>
> But at this point I get the following error message:
>
> % NCDF_CONTROL: Attempt to take the file out of
> define mode (ENDEF) failed. (NC_ERROR=-45)
> % Execution halted at: TIMESERIES::CREATE_CLIMATOLOGY 2589
>
> Does anyone have ANY idea what this might be about?
The code is pretty simple:
; Create a netCDF file for storing the climatology.
fID = NCDF_Create(filename, /CLOBBER)
; Define dimensions.
xsizeID = NCDF_DIMDEF(fID, 'cols', xsize)
ysizeID = NCDF_DIMDEF(fID, 'rows', ysize)
timeID = NCDF_DIMDEF(fID, 'time', 2)
; Define variables.
vid = NCDF_VARDEF(fID, 'time', [timeID], /LONG)
vid = NCDF_VARDEF(fID, 'climatology', [xsizeID, ysizeID], /DOUBLE)
; Define variable attributes.
NCDF_ATTPUT, fID, 'time', 'units', 'Days since 1601-01-01 00:00:00'
NCDF_ATTPUT, fID, 'time', 'long_name', 'Start and end time. '
NCDF_ATTPUT, fID, 'climatology', '_FillValue', fillValue
NCDF_ATTPUT, fID, 'climatology', 'long_name', 'longish name'
; Exit DEFINE mode and into DATA mode.
NCDF_Control, fID, /ENDEF
; Write the variables.
NCDF_VarPut, fID, 'time', [time[0], time[timepts-1]]
NCDF_VarPut, fID, 'climatology', climatology
; Close the file.
NCDF_Close, fID
At one time I had global attributes in there, but I took them out
and the code worked! So I added them back one by one. Then, it
didn't work again, so I took them all back out. Now it *still*
doesn't work! Something really, really weird going on here, I think. :-(
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68315 is a reply to message #68220] |
Fri, 09 October 2009 12:27 |
Foldy Lajos
Messages: 268
Registered: October 2001
|
Senior Member |
|
|
On Wed, 7 Oct 2009, Paul van Delst wrote:
>
> Ultimate thanks go to Mr. Lajos (or Mr. FÖldy? Don't know the correct order)
>
Hungarian names are in reverse order. Wait, no! English names are in
reverse order :-)
So, FÖLDY is the family name, Lajos is the given name. (I write the family
name in uppercase in emails, but this time it did not help :-) The o with
umlaut is pronounced like the German ö in the word 'schön' (a little bit
shorter).
regards,
Lajos
ps: the English middle name is the last in Hungarian, which is logical:
the second given name comes after the first.
|
|
|
|
| Re: Can't exit DEFINE mode in netCDF File [message #68316 is a reply to message #68220] |
Fri, 09 October 2009 12:00 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Paul van Delst writes:
> All good questions. Although I find myself in the strange position of *providing*
> David-Fanning-type wisdom (to said namesake no less!) rather than being on the receiving end:
>
> It is like it is because that's the way it is.
No wonder my hate mail runs about 10:1 verses the complements.(1) :-(
> BTW, was it the fillvalue attribute in particular, or some other one?
Your example was a
> little bit different from mine in that my fillvalue was a character (where I believe yours
> was a numeric quantity).
My variable and fill value where numerical values.
What happened was that I was reading a variable and its
fill value out of a file, computing some averages (climatologies),
then writing the climatologies back into another netCDF file.
I just tried to assign the climatologies the same fill value
I had read out of the file. But, of course, the original
data were long integers, and the climatologies were floats.
I never looked at (or even knew) the fill value.
> Ultimate thanks go to Mr. Lajos (or Mr. FÖldy?
> Don't know the correct order)
Yes, of course. I may have to investigate that FL language. ;-)
Cheers,
David
(1) Mostly from the Chinese though, thankfully. They are still
pissed about the Dahli Lama reference on my book page.
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
