| Re: netcdf + idl: posixio.c:265: px_pgin: Assertion ... [message #75715] |
Thu, 14 April 2011 01:33 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Am 13.04.2011 15:24, schrieb Fabzou:
> Hi,
>
> Thanks for the information. Could it be related to my problem?
> https://groups.google.com/group/comp.lang.idl-pvwave/browse_ thread/thread/85b56dbd3e16d687/213ae8de36749fd9?hl=ky&ln k=gst&q=fabzou#213ae8de36749fd9
>
I guess it is.
>
> Anyway, I did not understand what you mean with:
>
>> The workaround is to to introduce a caching routine for copying the file
The workaround is a function with locking capability which copies a file
from NFS to local storage before it becomes readed.
https://idlsource.fz-juelich.de/hg/icg_lib-11.0/file/tip/idl _work/rb_lib/caching.pro
>> to a local storage with IO Block: 4096 size.
if you do a
> stat filename
From the man page
NAME
stat - display file or file system status
you get the IO Block information of the filesystem for your filename.
In our case we have on the fileserver the IO Block: 4096 size but not on
the NFS filesystem.
Reimar
>
> Is there a code template for this ?
>
> Thanks in advance,
>
> Fab
>
> On 04/13/2011 02:37 PM, Reimar Bauer wrote:
>> Hi
>>
>> we found a terrible netCDF problem which makes it hard to use a given
>> network storage for reading many many files.
>>
>> It is the same problem for reading as discussed in that thread
>> https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgr oup
>> /2011/msg00108.html
>>
>> The workaround is to to introduce a caching routine for copying the file
>> to a local storage with IO Block: 4096 size.
>>
>> We have some users which got a bit crazy because of random
>>
>> idl: posixio.c:265: px_pgin: Assertion `*posp == ((off_t)(-1)) || *posp
>> == lseek(nciop->fd, 0, 1)' failed.
>> Aborted
>>
>> while accessing a few hundred files.
>>
>> cheers
>> Reimar
>
|
|
|
|
| Re: netcdf + idl: posixio.c:265: px_pgin: Assertion ... [message #75716 is a reply to message #75715] |
Thu, 14 April 2011 01:26  |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Am 13.04.2011 15:34, schrieb Kenneth P. Bowman:
> In article <io451n$ujh$1@zam602.zam.kfa-juelich.de>,
> Reimar Bauer <R.Bauer@fz-juelich.de> wrote:
>
>> Hi
>>
>> we found a terrible netCDF problem which makes it hard to use a given
>> network storage for reading many many files.
>>
>> It is the same problem for reading as discussed in that thread
>> https://www.unidata.ucar.edu/mailing_lists/archives/netcdfgr oup
>> /2011/msg00108.html
>>
>> The workaround is to to introduce a caching routine for copying the file
>> to a local storage with IO Block: 4096 size.
>>
>> We have some users which got a bit crazy because of random
>>
>> idl: posixio.c:265: px_pgin: Assertion `*posp == ((off_t)(-1)) || *posp
>> == lseek(nciop->fd, 0, 1)' failed.
>> Aborted
>>
>> while accessing a few hundred files.
>>
>> cheers
>> Reimar
>
> Is this a netCDF library problem or an IDL problem? Have you
> reported it to UNIDATA and/or ITTVIS?
I have reported it to unidata by a mail to the mailinglist, see the
thread I posted. I guess it is a unidata netCDF problem and after it is
fixed we need a newer version in IDL.
Reimar
>
> Ken
|
|
|
|
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
