| Can't get out of define mode in ncdf, not specific to variable. [message #82076] |
Thu, 15 November 2012 14:57  |
neleh.mac
Messages: 3
Registered: November 2012
|
Junior Member |
|
|
So I am using IDL to read in ncdf data, and write out (to a new file) the processed data. So I go to define mode, set up the variable, add some attributes, switch to data mode, stick in the data, and then go back to define for the next one.
e.g.
ncid = ncdf_create(file_name, /CLOBBER)
londim = ncdf_dimdef(ncid, 'longitude_dim', 144)
latdim = ncdf_dimdef(ncid, 'latitude_dim', 91)
vid = ncdf_vardef(ncid, 'emission', [londim,latdim], /float)
ncdf_attput, ncid, vid, 'long_name', "Some emissions"
ncdf_control, ncid, /ENDEF
ncdf_varput, ncid, vid, emiss_dat
ncdf_control, ncid, /REDEF
vid = ncdf_vardef(ncid, 'emission2', [londim,latdim], /float)
ncdf_attput, ncid, vid, 'long_name', "Some emissions 2"
ncdf_control, ncid, /ENDEF
ncdf_varput, ncid, vid, emiss_dat2
ncdf_control, ncid, /REDEF
... etc.
This works fine for several variables, over and over, until a certain point in the code and then at it will not get out of define mode. I commented out that particular part of the code where it was falling over, and got it to work fine (but my output file now has one less variable).
Now the weird part.
If I take the chunk of code for the variable that it falls at, and copy it back in further up the code, it works through it fine. It only fails when it gets to that line again, this time on a variable it wrote fine first time round.
It's like it's failing after going into define mode a certain number of times, it's not specific to any variable I'm trying to write, it just always fails on the nth one, regardless of what it is. I've swopped round the order it writes them to the file, and they all get written, just not if they happen to be at a certain point in the order. I have tried removing attributes from the variables but this doesn't seem to have any effect. I am very confused by this, please can you help?
|
|
|
|
| Re: Can't get out of define mode in ncdf, not specific to variable. [message #82129 is a reply to message #82076] |
Mon, 19 November 2012 12:25  |
neleh.mac
Messages: 3
Registered: November 2012
|
Junior Member |
|
|
So someone here has just pointed out the solution and it is frustratingly simple. I just needed to add the /NETCDF4_FORMAT keyword when creating the file. Pah.
I can't quite believe how in the exuberant amount of googling I've been doing that I didn't find this.
Thanks for the suggestions though :-)
Helen
On Monday, November 19, 2012 2:46:34 PM UTC-5, Helen Mac wrote:
> Thanks for your suggestions.
>
>
>
> Ok, I tried Chip's suggestion of syncing, but I still run into the problem.
>
>
>
> Paul, I am writing around 30 variables and have around 20 dimensions defined, so I don't think this is the problem.
>
>
>
> However, the file I am writing will be just larger than 2Gb, so Fab this looks like the most likely candidate for the problem. However, I have read that if I am using netcdf 3.6 or higher it should be ok. (I am using netcdf 3.6.3 and IDL 8). Is there something else I'm missing with the setup?
>
>
>
> FYI, I also tried writing and closing the file part way through so that it was opened up again and less than 2Gb was appended and written second time round, but I still encounter the error.
>
>
>
> Still confused, if anyone has any more suggestions I'd really appreciate it :-)
>
>
>
> On Friday, November 16, 2012 10:35:45 PM UTC-5, Fab wrote:
>
>> On 11/17/2012 12:25 AM, Paul van Delst wrote:
>
>>
>
>>> How many variables are you writing? What version of IDL? What version of
>
>>
>
>>> netCDF?
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> Also, before NCDF4 (7.1 patched and IDL8+), files could not be larger
>
>>
>
>> than 2Gb. If you attempt to define to many variables that will make the
>
>>
>
>> file larger than 2Gb (NCDF knows how big a file is going to be since you
>
>>
>
>> define the size of your variables first) this could also cause such an
>
>>
>
>> error...
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>
>
>>> If you are writing lots of variables, there used to be a limit to the
>
>>
>
>>> total number (in older netcdf's at least. I think it was 300).
>
>>
>
>>>
>
>>
>
>>> Complete shot in the dark.....
>
>>
>
>>>
>
>>
>
>>>
>
>>
>
>>> On 11/15/12 17:57, neleh.mac wrote:
>
>>
>
>>>> So I am using IDL to read in ncdf data, and write out (to a new
>
>>
>
>>>> file)
>
>>
>
>>>> the processed data. So I go to define mode, set up the variable, add
>
>>
>
>>>> some attributes, switch to data mode, stick in the data, and then go
>
>>
>
>>>> back to define for the next one.
>
>>
>
>>>>
>
>>
>
>>>> e.g.
>
>>
>
>>>> ncid = ncdf_create(file_name, /CLOBBER)
>
>>
>
>>>> londim = ncdf_dimdef(ncid, 'longitude_dim', 144)
>
>>
>
>>>> latdim = ncdf_dimdef(ncid, 'latitude_dim', 91)
>
>>
>
>>>>
>
>>
>
>>>> vid = ncdf_vardef(ncid, 'emission', [londim,latdim], /float)
>
>>
>
>>>> ncdf_attput, ncid, vid, 'long_name', "Some emissions"
>
>>
>
>>>> ncdf_control, ncid, /ENDEF
>
>>
>
>>>> ncdf_varput, ncid, vid, emiss_dat
>
>>
>
>>>> ncdf_control, ncid, /REDEF
>
>>
>
>>>>
>
>>
>
>>>> vid = ncdf_vardef(ncid, 'emission2', [londim,latdim], /float)
>
>>
>
>>>> ncdf_attput, ncid, vid, 'long_name', "Some emissions 2"
>
>>
>
>>>> ncdf_control, ncid, /ENDEF
>
>>
>
>>>> ncdf_varput, ncid, vid, emiss_dat2
>
>>
>
>>>> ncdf_control, ncid, /REDEF
>
>>
>
>>>>
>
>>
>
>>>> ... etc.
>
>>
>
>>>>
>
>>
>
>>>> This works fine for several variables, over and over, until a certain
>
>>
>
>>>> point in the code and then at it will not get out of define mode. I
>
>>
>
>>>> commented out that particular part of the code where it was falling
>
>>
>
>>>> over, and got it to work fine (but my output file now has one less
>
>>
>
>>>> variable).
>
>>
>
>>>>
>
>>
>
>>>> Now the weird part.
>
>>
>
>>>>
>
>>
>
>>>> If I take the chunk of code for the variable that it falls at, and
>
>>
>
>>>> copy it back in further up the code, it works through it fine. It
>
>>
>
>>>> only fails when it gets to that line again, this time on a variable
>
>>
>
>>>> it wrote fine first time round.
>
>>
>
>>>>
>
>>
>
>>>> It's like it's failing after going into define mode a certain number
>
>>
>
>>>> of times, it's not specific to any variable I'm trying to write, it
>
>>
>
>>>> just always fails on the nth one, regardless of what it is. I've
>
>>
>
>>>> swopped round the order it writes them to the file, and they all get
>
>>
>
>>>> written, just not if they happen to be at a certain point in the
>
>>
>
>>>> order. I have tried removing attributes from the variables but this
>
>>
>
>>>> doesn't seem to have any effect. I am very confused by this, please
>
>>
>
>>>> can you help?
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
