Thanks for your suggestions.
Ok, I tried Chip's suggestion of syncing, but I still run into the problem.
Paul, I am writing around 30 variables and have around 20 dimensions defined, so I don't think this is the problem.
However, the file I am writing will be just larger than 2Gb, so Fab this looks like the most likely candidate for the problem. However, I have read that if I am using netcdf 3.6 or higher it should be ok. (I am using netcdf 3.6.3 and IDL 8). Is there something else I'm missing with the setup?
FYI, I also tried writing and closing the file part way through so that it was opened up again and less than 2Gb was appended and written second time round, but I still encounter the error.
Still confused, if anyone has any more suggestions I'd really appreciate it :-)
On Friday, November 16, 2012 10:35:45 PM UTC-5, Fab wrote:
> On 11/17/2012 12:25 AM, Paul van Delst wrote:
>
>> How many variables are you writing? What version of IDL? What version of
>
>> netCDF?
>
>
>
>
>
> Also, before NCDF4 (7.1 patched and IDL8+), files could not be larger
>
> than 2Gb. If you attempt to define to many variables that will make the
>
> file larger than 2Gb (NCDF knows how big a file is going to be since you
>
> define the size of your variables first) this could also cause such an
>
> error...
>
>
>
>
>
>
>
>> If you are writing lots of variables, there used to be a limit to the
>
>> total number (in older netcdf's at least. I think it was 300).
>
>>
>
>> Complete shot in the dark.....
>
>>
>
>>
>
>> On 11/15/12 17:57, neleh.mac wrote:
>
>>> So I am using IDL to read in ncdf data, and write out (to a new
>
>>> file)
>
>>> the processed data. So I go to define mode, set up the variable, add
>
>>> some attributes, switch to data mode, stick in the data, and then go
>
>>> back to define for the next one.
>
>>>
>
>>> e.g.
>
>>> ncid = ncdf_create(file_name, /CLOBBER)
>
>>> londim = ncdf_dimdef(ncid, 'longitude_dim', 144)
>
>>> latdim = ncdf_dimdef(ncid, 'latitude_dim', 91)
>
>>>
>
>>> vid = ncdf_vardef(ncid, 'emission', [londim,latdim], /float)
>
>>> ncdf_attput, ncid, vid, 'long_name', "Some emissions"
>
>>> ncdf_control, ncid, /ENDEF
>
>>> ncdf_varput, ncid, vid, emiss_dat
>
>>> ncdf_control, ncid, /REDEF
>
>>>
>
>>> vid = ncdf_vardef(ncid, 'emission2', [londim,latdim], /float)
>
>>> ncdf_attput, ncid, vid, 'long_name', "Some emissions 2"
>
>>> ncdf_control, ncid, /ENDEF
>
>>> ncdf_varput, ncid, vid, emiss_dat2
>
>>> ncdf_control, ncid, /REDEF
>
>>>
>
>>> ... etc.
>
>>>
>
>>> This works fine for several variables, over and over, until a certain
>
>>> point in the code and then at it will not get out of define mode. I
>
>>> commented out that particular part of the code where it was falling
>
>>> over, and got it to work fine (but my output file now has one less
>
>>> variable).
>
>>>
>
>>> Now the weird part.
>
>>>
>
>>> If I take the chunk of code for the variable that it falls at, and
>
>>> copy it back in further up the code, it works through it fine. It
>
>>> only fails when it gets to that line again, this time on a variable
>
>>> it wrote fine first time round.
>
>>>
>
>>> It's like it's failing after going into define mode a certain number
>
>>> of times, it's not specific to any variable I'm trying to write, it
>
>>> just always fails on the nth one, regardless of what it is. I've
>
>>> swopped round the order it writes them to the file, and they all get
>
>>> written, just not if they happen to be at a certain point in the
>
>>> order. I have tried removing attributes from the variables but this
>
>>> doesn't seem to have any effect. I am very confused by this, please
>
>>> can you help?
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
