comp.lang.idl-pvwave archive: archive » Trouble writing very large files

Home » Public Forums » archive » Trouble writing very large files

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Trouble writing very large files [message #64793 is a reply to message #62676]

Fri, 16 January 2009 13:21

wlandsman
Messages: 743
Registered: June 2000

Senior Member

On Jan 16, 9:38 am, Maarten <maarten.sn...@knmi.nl> wrote:

> I think the right answer was given by Wayne is his original post. What
> I don't get is why you'd want a 4G unformatted data file in the first
> place, I'd prefer to dump the data to an HDF-5 file.

Well, I encountered the problem when trying to write data in the FITS
data format used in astronomy, and I isolated the problem to the
writeu call used in the IDL FITS writer. You are right that the
problem probably would not occur when writing to HDF-5, since the IDL
HDF writer uses an external library, and not writeu.

As discussed in the earlier thread on this topic, the solution is to
write a big array in pieces, e.g.

for i=0,255 do writeu,1,im[*,*,i]

For the FITS writer, my kluge will be to write a wrapper procedure
writebigu.pro to replace writeu, which will write an array in pieces
if the number of elements exceeds 2^31. --Wayne

Report message to a moderator

[Message index]

		Trouble writing very large files By: Vince Hradil on Fri, 26 September 2008 13:12
		Re: Trouble writing very large files By: wlandsman on Fri, 16 January 2009 13:21
		Re: Trouble writing very large files By: Maarten[1] on Fri, 16 January 2009 06:38
		Re: Trouble writing very large files By: Juggernaut on Fri, 16 January 2009 04:53

Previous Topic:	Re: Question relating to IDL wavelet methods
Next Topic:	MPfit question

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:59:23 PDT 2025

Total time taken to generate the page: 0.00217 seconds