comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Read Large files

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Read Large files [message #81229]

Tue, 28 August 2012 11:23

wlandsman
Messages: 743
Registered: June 2000

Senior Member

It should be faster to read as much as your memory will allow in each READ statement. You say that you read "each data record". If each data record is a small fraction of your file size then it will take a long time to read the file. Preferably you can read your entire file in a single READU call.

--Wayne

On Tuesday, August 28, 2012 1:19:03 PM UTC-4, Gompie wrote:
> Hi,
>
> I want to read in 58 gb of data that is spread over several files ( having records)of size ranging from 7.6gb to 1gb.
>
> What is the fastest way to read these data files.
>
> Right now I read in each data record and it takes 6 hrs to do the reading.
>
> Thanks in advance
>
> Gompie

Report message to a moderator

[Message index]

		Re: Read Large files By: wlandsman on Tue, 28 August 2012 11:23
		Re: Read Large files By: David Fanning on Tue, 28 August 2012 10:28
		Re: Read Large files By: Gompie on Fri, 31 August 2012 18:29

Previous Topic:	Geodetic functions library GeoDLL 32 Bit and 64 Bit
Next Topic:	very simple but weird problem with CONTOUR

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:13:05 PDT 2025

Total time taken to generate the page: 0.00473 seconds