comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: large array

large array
type: item
description: I need to read a file that is ca. 7 GB large (the file size is defineda s 2048 x 2048 x 900 are the dimensions, x 2 (integers) + 512 bytes header). The format is known (.ISQ) and I have a routine that deals with the header information efficiently; I have some similarly formatted files of the same scan process around 800-900 MB that I can read without problems. Generally, I had no problems reading files up to 1.4 GB in size directly into one variable under IDL. The data is read into an intarr (2048,2048,900) like this: imagearray = assoc (.., intarr(...), headersize) imagearray = temporary (imagearray [0]) Now, the reading takes a little while but it seems to work alright. The machine - most of the time - does not crash (12 GB RAM, 64-bit AIX version of IDL 6.1, IBM-Workstation). However, the visualisation of slice subscripts to this array later does not display any interesting information; instead, the images of these large data files look different each time and they do not reflect the content of the data. Question: What do I need to know in order to set up an array for very large data? Is there a basic difference between arrays < 2 GB and arrays > GB of size? Are the subscript variables all multiplied before the array is really looked at, so do all of the subscript variables need to be Long64? Such as intarr (long64(x),long64(y),long64(z))? Thanks, Wolf.
creator: Wolf Schweitzer
date: 2004-10-12T14:52:58-00:00