I made some tests with data reads at different separations to see how
opening/closing affects the caching. The effect's not large, but
appears to favour files held open.
Still, if we agree the cost of opening a file is very low, I don't see
why the number open needs to be rationed so strictly.
pro filetest
f1="D:\mydocs\data\hrsc\lev4\2112\h2112_0000.nd4.51" ; ~1 GB each
f2="D:\mydocs\data\hrsc\lev4\2123\h2123_0000.nd4.50" ;
a=bytarr(2000,2000)
for j=0,5 do begin
distance=(long([0,1e3,5e5,1e6,1e7,1e8]))[j]
s1=systime(/sec)
for i=0,9 do begin
openr,3,f1
point_lun,3,i*distance
readu,3,a
close,3
openr,3,f2
point_lun,3,i*distance
readu,3,a
close,3
endfor
s2=systime(/sec)
print, "repeat open:",s2-s1
s1=systime(/sec)
openr,1,f1
openr,2,f2
for i=0,9 do begin
point_lun,1,i*distance
readu,1,a
point_lun,2,i*distance
readu,2,a
endfor
close,1
close,2
s2=systime(/sec)
print, "single open:",s2-s1
print
endfor
end
IDL> filetest
repeat open: 0.12500000
single open: 0.094000101
repeat open: 0.10899997
single open: 0.078000069
repeat open: 0.14100003
single open: 0.12500000
repeat open: 0.14100003
single open: 0.13999987
repeat open: 0.14100003
single open: 0.12500000
repeat open: 0.15599990
single open: 0.12500000
IDL> filetest
repeat open: 0.13999987
single open: 0.094000101
repeat open: 0.093999863
single open: 0.078000069
repeat open: 0.14100003
single open: 0.13999987
repeat open: 0.14100003
single open: 0.14000010
repeat open: 0.15700006
single open: 0.13999987
repeat open: 0.14100003
single open: 0.14100003
and with the blocks reversed:
IDL> filetest
single open: 0.10899997
repeat open: 0.11000013
single open: 0.078000069
repeat open: 0.092999935
single open: 0.12500000
repeat open: 0.15700006
single open: 0.12500000
repeat open: 0.15599990
single open: 0.12500000
repeat open: 0.15599990
single open: 0.12500000
repeat open: 0.14100003
|