| Re: How to store compressed data ? [message #68808] |
Tue, 01 December 2009 13:03 |
rogass
Messages: 200
Registered: April 2008
|
Senior Member |
|
|
On 1 Dez., 16:00, nata <bernat.puigdomen...@gmail.com> wrote:
> Using HDF_PACKDATA I don't see the difference:
>
> aa=fltarr(240,360,24,47,2)
> help, /mem
> heap memory used: 780326120, max: 1559999875, gets: 1126,
> frees: 317
>
> rr=hdf_packdata(aa)
> aa=0l
> help, /mem
> heap memory used: 780326085, max: 1559999860, gets: 1143,
> frees: 334
Strange... Maybe you will find some appraoches here:
http://www.astro.washington.edu/docs/idl/htmlhelp/index.html or maybe
one of the IDL-gurus has an idea :) If not, then you may use some ENVI
routines which can handle handle compressed data in memory (slows
drastically down). Some of lossless compressing techniques such as RLE
or LZW maybe also interesting...
CR
|
|
|
|
| Re: How to store compressed data ? [message #68815 is a reply to message #68808] |
Tue, 01 December 2009 07:00  |
natha
Messages: 482
Registered: October 2007
|
Senior Member |
|
|
Using HDF_PACKDATA I don't see the difference:
aa=fltarr(240,360,24,47,2)
help, /mem
heap memory used: 780326120, max: 1559999875, gets: 1126,
frees: 317
rr=hdf_packdata(aa)
aa=0l
help, /mem
heap memory used: 780326085, max: 1559999860, gets: 1143,
frees: 334
|
|
|
|
| Re: How to store compressed data ? [message #68816 is a reply to message #68815] |
Tue, 01 December 2009 06:56 |
natha
Messages: 482
Registered: October 2007
|
Senior Member |
|
|
MmmmMMm..... I don't understand the ASSOC function. Here is what I do:
aa=FLTARR(240,360,24,97,2)
OPENW, lun, 'temporary_file.dat', /GET_LUN
ll=ASSOC(lun,aa,/PACKED)
HELP, ll
LL FLOAT = File<temporary_file.dat> Array[240, 360,
24, 97, 2]
Ok. And now:
aa=0l
aa=ll[*,*,*,0,0]
% End of file encountered. Unit: 100, File: temporary_file.dat
% Execution halted at: $MAIN$
OUPS ! What's happening here ?
|
|
|
|
| Re: How to store compressed data ? [message #68871 is a reply to message #68816] |
Tue, 01 December 2009 00:02 |
rogass
Messages: 200
Registered: April 2008
|
Senior Member |
|
|
On 30 Nov., 22:27, nata <bernat.puigdomen...@gmail.com> wrote:
> Hi guys,
>
> See the following example:
>
> help, /mem
> aa=fltarr(240,360,24,47,2)
> help, /mem
>
> IDL returns:
> heap memory used: 708636, max: 780382426, gets: 13719,
> frees: 12874
> heap memory used: 780382333, max: 780382428, gets: 13731,
> frees: 12885
>
> That's 780 Mb for one big array. My question is if there is a way to
> store this array (in memory) compressed. I'm developing an application
> and I don't know if that's possible in order to use less RAM of the
> computer.
> Thanks,
>
> nata
IF you can implement ASSOC, THEN you would be able to use the /PACKED
switch ELSE you can just use HDF_PACKDATA :)
Maybe it helps
CR
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
