comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Large data sets

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Large data sets [message #24931 is a reply to message #24929]

Fri, 04 May 2001 06:47

Liam E. Gumley
Messages: 378
Registered: January 2000

Senior Member

Neil Talsania wrote:
> I am new to IDL, and everything that I have done so far works great on
> small data sets. But, I dont know how to use it for a large (>2 gig)
> dataset, as I obviously just cant load that into an array. The data is on
> disk and can either be tiled so that it is stored in 1024x1024 segments, or
> it is stripped which is one complete row at a time. Even a complete row is
> large (>80000 samples).
>
> Any thoughts as to how I would be able to use this data?
>
> Another question: is there thread support in IDL? I have a machine with 18
> processors, so it sure would be nice to process this large dataset in 18
> parallel threads. Any suggestions?

The following may be of interest:

http://sag-www.ssl.berkeley.edu/~korpela/mmap/

Cheers,
Liam.
http://cimss.ssec.wisc.edu/~gumley/

Report message to a moderator

[Message index]

		Re: Large data sets By: Alex Schuster on Fri, 04 May 2001 07:12
		Re: Large data sets By: Paul van Delst on Fri, 04 May 2001 07:00
		Re: Large data sets By: Liam E. Gumley on Fri, 04 May 2001 06:47

Previous Topic:	Re: write_ppm (newbie)
Next Topic:	Reading .xls files

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 04:13:23 PDT 2025

Total time taken to generate the page: 0.95773 seconds