comp.lang.idl-pvwave archive: archive » Re: Writing a very large file

Home » Public Forums » archive » Re: Writing a very large file

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Writing a very large file [message #57413 is a reply to message #57411]

Fri, 07 December 2007 13:47

Jim Pendleton, ITT Vi is currently offline

Jim Pendleton, ITT Vi
Messages: 13
Registered: August 2006

Junior Member

"wlandsman" <wlandsman@gmail.com> wrote in message
news:f3c3ac4e-a5c2-4d57-a1f2-d0356d7a0d00@i12g2000prf.google groups.com...
> I am writing a sequence of images to a single very large file on my
> Linux system. I find that the processing dramatically slows down
> after the first few images. The simplified code looks like the
> following:
>
> pro test
> ; Display the time required to write a series of image to a single
> large file
> im = intarr(4096,4096)
> t = systime(1)
>
> close,1 & openw,1,'test.dat'
> for i=0,20 do begin
> writeu,1,im
> print,i,systime(1)-t & t = systime(1)
> endfor
>
> close,1
> return
>
>
> IDL> test
> 0 0.22054195
> 1 0.26708603
> 2 0.35127902
> 3 0.37285185
> 4 3.3877730
> 5 6.1666460
> 6 6.1697872
> 7 6.2481630
>
>
> So the first four images take ~0.3s each to write, while subsequent
> images require more than 6 seconds each. I suspect that the slowing
> down is due to IDL (or the OS) needing to extend the file size. (I
> checked that it is not a memory usage problem.) So I think
> things would speed up if I could specify the final file size at the
> beginning -- perhaps there is a way to do this in Unix? I have
> experimented with the BUFSIZE and RAWIO keywords to OPENW but so far
> without any improvement.
>
> Thanks for any suggestions, --Wayne

Wayne,
The POINT_LUN procedure can be used to define a file's
size at the outset. That is, you can OPENW a file then POINT_LUN
to define the maximum file size without having to write any data to
the file first. POINT_LUN to an offset of 0 to begin writing.
See also TRUNCATE_LUN, and the "Reading and Writing Very
Large Files" section of the on-line help.

Jim P.

Jim P.

Report message to a moderator

[Message index]

		Re: Writing a very large file By: Bringfried Stecklum on Sat, 08 December 2007 08:19
		Re: Writing a very large file By: Kenneth Bowman on Fri, 07 December 2007 20:50
		Re: Writing a very large file By: Jim Pendleton, ITT Vi on Fri, 07 December 2007 13:47
		Re: Writing a very large file By: Jean H. on Fri, 07 December 2007 13:23
		Re: Writing a very large file By: pgrigis on Fri, 07 December 2007 12:39
		Re: Writing a very large file By: David Fanning on Fri, 07 December 2007 12:23
		Re: Writing a very large file By: wlandsman on Mon, 10 December 2007 07:24
		Re: Writing a very large file By: Kenneth Bowman on Mon, 10 December 2007 07:43

Previous Topic:	Re: Version Control Conundrum
Next Topic:	SVN Question

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 04:09:27 PDT 2025

Total time taken to generate the page: 1.51823 seconds