| Writing a very large file [message #57420] |
Fri, 07 December 2007 12:17 |
wlandsman
Messages: 743
Registered: June 2000
|
Senior Member |
|
|
I am writing a sequence of images to a single very large file on my
Linux system. I find that the processing dramatically slows down
after the first few images. The simplified code looks like the
following:
pro test
; Display the time required to write a series of image to a single
large file
im = intarr(4096,4096)
t = systime(1)
close,1 & openw,1,'test.dat'
for i=0,20 do begin
writeu,1,im
print,i,systime(1)-t & t = systime(1)
endfor
close,1
return
IDL> test
0 0.22054195
1 0.26708603
2 0.35127902
3 0.37285185
4 3.3877730
5 6.1666460
6 6.1697872
7 6.2481630
So the first four images take ~0.3s each to write, while subsequent
images require more than 6 seconds each. I suspect that the slowing
down is due to IDL (or the OS) needing to extend the file size. (I
checked that it is not a memory usage problem.) So I think
things would speed up if I could specify the final file size at the
beginning -- perhaps there is a way to do this in Unix? I have
experimented with the BUFSIZE and RAWIO keywords to OPENW but so far
without any improvement.
Thanks for any suggestions, --Wayne
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
