comp.lang.idl-pvwave archive: archive » Reading large files with restore

Home » Public Forums » archive » Reading large files with restore

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Reading large files with restore [message #85944 is a reply to message #85943]

Fri, 20 September 2013 08:30

Sir Loin Steak
Messages: 42
Registered: January 2012

Member

On Friday, 20 September 2013 16:25:36 UTC+1, David Fanning wrote:
> ljs15@fsmail.net writes:
>
>
>
>> I have a script (which I didn't write) which loops through different months and reads in atmospheric variables from .sav files, with a different file for each month. These files contain lots of data, and hence are around 200-500 MB in size.
>
>>
>
>> When I run the script it takes a long time to restore the data on each loop (obviously!), but it makes my entire computer go slow, and on occasion it completely freezes and I have no choice but to actually turn the computer off and on again.
>
>>
>
>> I am assuming this is a memory issue, and was wondering if anyone knew a way around these problems (re-writing the script and save files isn't really an option I want to consider). Is there perhaps some way of 'de-allocating' the memory after the completion of each loop?
>
>
>
> Wait, you have a program that does something crazy, but you don't want
>
> to change it? Do I have that right? I would say off-hand that this
>
> hampers our ability to provide a solution. (Even prayer is unlikely to
>
> work in this situation.)
>
>
>
> Is this "script" a main-level program by any chance?
>
>
>
> Cheers,
>
>
>
> David
>
>
>
>
>
>
>
> --
>
> David Fanning, Ph.D.
>
> Fanning Software Consulting, Inc.
>
> Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
>
> Sepore ma de ni thue. ("Perhaps thou speakest truth.")

Ha! Well, what I mean is that the .sav files have already been created, and I don't have access to any of the scripts or the original data to re-produce them. So whatever I do, I'm still going to have to read the .sav files in.

The script is a main-level program which restores the save files, gets the relevant data it wants from the complete data set (e.g. temperature and cloud opacities) and then writes this subset of data to a file.

As I say, I can't (easily or quickly) do anything about the save files, so just thought there may be some memory thing which can help me.

Cheers,

Liam

Report message to a moderator

[Message index]

		Reading large files with restore By: Sir Loin Steak on Fri, 20 September 2013 08:18
		Re: Reading large files with restore By: David Fanning on Fri, 20 September 2013 08:25
		Re: Reading large files with restore By: Sir Loin Steak on Fri, 20 September 2013 08:30
		Re: Reading large files with restore By: David Fanning on Fri, 20 September 2013 08:33
		Re: Reading large files with restore By: Sir Loin Steak on Fri, 20 September 2013 08:50
		Re: Reading large files with restore By: David Fanning on Fri, 20 September 2013 09:00
		Re: Reading large files with restore By: Phillip Bitzer on Fri, 20 September 2013 09:46
		Re: Reading large files with restore By: Sir Loin Steak on Mon, 23 September 2013 05:48
		Re: Reading large files with restore By: Paul Van Delst[1] on Tue, 24 September 2013 10:56

Previous Topic:	logical_true vs nothing in an if statment
Next Topic:	Reading part of a netcdf file?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 07:44:28 PDT 2025

Total time taken to generate the page: 0.31793 seconds