| Re: ncdf + x-connection + many files = crazy [message #77553 is a reply to message #77475] |
Thu, 08 September 2011 00:45  |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Am 08.09.2011 09:12, schrieb Reimar Bauer:
> Am 07.09.2011 19:03, schrieb Fabzou:
>> Hi,
>>
>> I will also subscribe, if it can help...
>>
>> http://www.rhinocerus.net/forum/lang-idl-pvwave/657163-rando m-netcdf-i-o-error.html
>>
>>
>>
>
> Hi there,
>
> I experienced yesterday with NFS and found probably a solution for me
>
> Somee time ago I already figured that I can read many times without a
> problem if the Block size is 4 K which usually is given on local storage.
>
> The problematic nfs mounts tells:
>
> stat -f .
> File: "."
> ID: 0 Namelen: 255 Type: nfs
> Block size: 1048576 Fundamental block size: 1048576
> Blocks: Total: 153525 Free: 24690 Available: 24690
>
>
>
> Currently I experienced changing rsize, wsize of Block size to a lower
> value on client side.
>
> I tried several values and found another fishy number.
>
> 4096 * 255 works! But the given old number is a multiplier by 256!
>
> I am not able by 255 * 4096 to crash IDL by reading ncdf files from NFS
> storage.
>
> That are the test params for my NFS test mount
>
> rw,rsize=1044480,wsize=1044480,nodev,nosuid,exec,auto,nouser ,async
>
> If I enter the defaults value of 1048576 IDL crashes reading many files.
> (Or one file in a loop of 10K iterations)
>
>
> For further understanding of that also a better NFS expert than me is
> wanted. I am not sure if it makes really sense to have such big values
> as defaults. Also I don't understand why that makes a problem for IDL
> because I have not got any issue requests for other languages (python,
> fortran, c, c++) reading netCDF files.
>
>
> Can you please verify if that workaround helps.
>
> cheers
> Reimar
Hmm, that did not solve the original testcase that stops at 65K
so this is a slightly different problem and a different fix.
Reimar
>
>
>
>>
>> On 09/07/2011 11:27 AM, Reimar Bauer wrote:
>>> Am 06.09.2011 16:54, schrieb savoie@nsidc.org:
>>>>
>>>> Yun<bernat.puigdomenech@gmail.com> writes:
>>>>
>>>> > https://groups.google.com/group/comp.lang.idl-pvwave/browse_ thread/thread/98cfca51043318d8/d175f2e21203a5bc?hl=en&ln k=gst&q=ncdf+connection#d175f2e21203a5bc
>>>> >
>>>> >
>>>> > I am reading 100 files from network volumes, and I get the error
>>>> > (NC_ERROR=-31) randomly. I am using the following version of NCDF /DLM
>>>> > on a Linux Machine.
>>>> >
>>>> > Any help will be appreciated,
>>>>
>>>>
>>>> Yun,
>>>>
>>>>
>>>> Hi there, that was me who posted that problem. It would be excellent if
>>>> you could also file a bug report with ITTVIS. I continue to be told it
>>>> will be fixed in the next version, but they don't have a work-around in
>>>> the mean time. I've begun the work-around of not using IDL for my
>>>> NETCDF work, which is becoming more and more of the work I must do. It
>>>> would be helpful for this problem to be solved and if I'm not the only
>>>> person having the issue it might help actually get it solved.
>>>>
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>
>>> Can you share the url of the tracked issue?
>>>
>>> May be subscribing or telling who all is affected brings it on top.
>>>
>>>
>>> Reimar
>>
>
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
