comp.lang.idl-pvwave archive: archive » Re: Dealing with Large data arrays, reducing memory and ASSOC

Home » Public Forums » archive » Re: Dealing with Large data arrays, reducing memory and ASSOC

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Dealing with Large data arrays, reducing memory and ASSOC [message #54467 is a reply to message #54466]

Thu, 14 June 2007 06:08

Kenneth Bowman
Messages: 86
Registered: November 2006

Member

In article <1181824433.145388.26020@d30g2000prg.googlegroups.com>,
Ambrosia_Everlovely <ambrosia_everlovely@hotmail.com> wrote:

> Hi,
> I have a fairly large datacube, DC(x,y,t)=DC(512,512,2048) and I want
> to perform an FFT in the t direction. Now I can do,
> FFTDC=fft(DC,-1,dim=3) which takes an excessive amount of memory (19 G
> + 50 G virtual) and slows the whole system down.
> Since this must be a fairly common practice amongst astronomers, can
> anyone provide - or link to - a small IDL algorithm which will allow
> me to use ASSOC or reduce the memory in some way? I have also tried
> TEMPORARY, but this doesn't seem to help at all.
>
> Thankyou!!!!

I would just do it in slices

dct = COMPLEXARR(512,512,2048)
FOR j = 0, 511 do dct[*,j,*] = FFT(REFORM(dc[*,j,*]), -1, DIM = 2)

This does access memory in nearly the worst possible way. If you are going
to be doing this a lot, you might want to consider rearranging the
data so that t is the first dimension

dct = COMPLEXARR(2048,512,512)
FOR k = 0, 255 D0 xt[0,0,k] = FFT(REFORM(x[*,*,k]), -1, DIM = 1)

Ken Bowman

Report message to a moderator

[Message index]

		Re: Dealing with Large data arrays, reducing memory and ASSOC By: Kenneth Bowman on Fri, 15 June 2007 12:54
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: bill.dman on Fri, 15 June 2007 11:40
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: Ambrosia_Everlovely on Fri, 15 June 2007 00:26
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: Kenneth Bowman on Thu, 14 June 2007 11:52
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: Kenneth Bowman on Thu, 14 June 2007 11:50
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: JD Smith on Thu, 14 June 2007 09:32
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: Haje Korth on Thu, 14 June 2007 09:18
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: bill.dman on Thu, 14 June 2007 06:41
		Re: Dealing with Large data arrays, reducing memory and ASSOC By: Kenneth Bowman on Thu, 14 June 2007 06:08

Previous Topic:	Read particular band from air temperature image among MOD07 air products
Next Topic:	Yikes! Time for a blow-out party!

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 17:19:28 PDT 2025

Total time taken to generate the page: 0.00352 seconds