Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
I need to read a file that is ca. 7 GB large (the file size is defineda s 2048 x 2048 x 900 are the dimensions, x 2 (integers) + 512 bytes header). The format is known (.ISQ) and I have a routine that deals with the header information efficiently; I have some similarly formatted files of the same scan process around 800-900 MB that I can read without problems. Generally, I had no problems reading files up to 1.4 GB in size directly into one variable under IDL. The data is read into an intarr (2048,2048,900) like this: imagearray = assoc (.., intarr(...), headersize) imagearray = temporary (imagearray [0]) Now, the reading takes a little while but it seems to work alright. The machine - most of the time - does not crash (12 GB RAM, 64-bit AIX version of IDL 6.1, IBM-Workstation). However, the visualisation of slice subscripts to this array later does not display any interesting information; instead, the images of these large data files look different each time and they do not reflect the content of the data. Question: What do I need to know in order to set up an array for very large data? Is there a basic difference between arrays < 2 GB and arrays > GB of size? Are the subscript variables all multiplied before the array is really looked at, so do all of the subscript variables need to be Long64? Such as intarr (long64(x),long64(y),long64(z))? Thanks, Wolf.