| Re: Input Files - Size Limit? [message #8978 is a reply to message #8977] |
Sun, 18 May 1997 00:00   |
davidf
Messages: 2866
Registered: September 1996
|
Senior Member |
|
|
Mark Harvey <mark@vortex.shm.monash.edu.au> writes:
> Hi. I'm pretty much an IDL rookie but have been handed quite
> a big project - developing an IDL-based analysis and
> visualisation package for meteorological data.
>
> My question is this: can IDL handle extremely large files (in
> various data formats) without crashing out? I'm thinking of
> sizes in the region of 100mb+.
Sure, put it into the variable "a". :-)
Seriously, Mark, there is no special limitations on the size
of data inside of IDL, but there are some practical limitations.
For example, most system administrators limit the amount
of process memory any one process can have. You have to
pretty much hand over your first born child to get them to
give you more.
Large files are usually not a problem with IDL, as long
as you don't read all the data into memory at once. Use
the associated variable method to read just that portion
of the data you need at any one time, and practice good
memory management techniques while you are working
with data. In practice, this means deleting or undefining
variables when you are done with them, etc. (See my
web page for ideas about this.) In IDL 5.0 you will
want to take advantage of pointers for sure. Just be
sure you delete the pointer data when you are done
with it.
If you run into trouble, there are lots of good IDL
programmers nearby. Justin Baker and Robert Dahni
at the Bureau of Meteorology there in Melbourne are
two names that come immediately to mind.
Oh, and ask for a machine that has *LOTS* of RAM!
Best of luck!
David
----------------------------------------------------------
David Fanning, Ph.D.
Fanning Software Consulting
Customizable IDL Programming Courses
Phone: 970-221-0438 E-Mail: davidf@dfanning.com
Coyote's Guide to IDL Programming: http://www.dfanning.com
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
