| VARRAY, memory & extracting subarrays [message #25609] |
Sun, 01 July 2001 23:21 |
Kristine Hensel
Messages: 26
Registered: June 1999
|
Junior Member |
|
|
I'm processing large (105 MB) arrays of images, and I've been running
into memory problems. (Not surprisingly, right?) I've started using
Eric Korpela's VARRAY routine, which has helped, but I still can't
manage to extract a subarray without using all of the available memory.
Theoretically I have 1 GB of memory, and we've tried maximizing every
system variable that we can, but I'm still crashing ("Unable to allocate
memory to create array") when I try to run my image processing program.
I can declare the main array and subarray using VARRAY, and I can read
images into the main array. However, once I try to put anything from
main array in the subarray, e.g. subarray=main_array[i:j,k:l,*], or even
subarray=main_array, I start chewing up memory.
Two not-very-good workarounds are:
- using subarray=temporary(main_array), but then I lose main_array. I
suppose I could read the main array into main_array and then into some
temporary variable, so that I could do subarray=temporary(tmp_array),
but that involves more I/O
- writing the subsection of main_array to a file, then reading it
directly into subarray - this is slow
I've attached sample code below, along with the results of
memory(/current) along the way.
Any assistance with VARRAY would be greatly appreciated.
Thanks,
Kristine
--
Kristine Hensel
Environmental Systems & Services Phone: +61-3-9864-5300
405 Toorongo Rd FAX: +61-3-9822-8028
Hawthorn East, VIC 3123 Australia e-mail: kristine@esands.com
>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>
pro sector_noaa
filename = '/home/meteor/mapped_data/200105171839N14.map'
openr, lun, filename, /get_lun, error=err
nlon=0L
nlat=0L
nchannel=0L
readu, lun, nlon, nlat, nchannel
lons=fltarr(nlon)
lats=fltarr(nlat)
readu, lun, lons, lats
; declare main images array:
file_delete, '/home/meteor/mapped_data/images.dat', /quiet
*** Memory in use: 71 kB
images = varray("/home/meteor/mapped_data/images.dat",byte(0), $
nlon,nlat,nchannel,/writable)
*** Memory in use: 71 kB
; read in images array:
readu, lun, images
*** Memory in use: 71 kB
free_lun, lun
; declare subarray:
file_delete, '/home/meteor/mapped_data/sector_images.dat', /quiet
*** Memory in use: 70 kB
sector_images = varray("/home/meteor/mapped_data/sector_images.dat", $
byte(0),nlon,nlat,nchannel,/writable)
*** Memory in use: 70 kB
; write out subarray:
openw, lun, filename+'_sector', /get_lun, error=err
writeu, lun, images[0:nlon-1,0:nlat-1,*]
free_lun, lun
; read in sector images:
openr, lun, filename+'_sector', /get_lun, error=err
readu, lun, sector_images
*** Memory in use: 70 kB
free_lun, lun
; try to copy directly:
sector_images = images
*** Memory in use: 10531 kB
end
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
