comp.lang.idl-pvwave archive: archive » Big arrays, reducing data

Home » Public Forums » archive » Big arrays, reducing data

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Big arrays, reducing data [message #53095]

Wed, 21 March 2007 10:15

Eric Hudson
Messages: 19
Registered: June 2006

Junior Member

Hi,

I have what I hope is an easy question and then probably a hard one.

1) I need to make some big arrays (ideally 16000^2 elements or more)
but find I often get "unable to allocate memory" errors. Is there
some way of determining (at run time) the largest array that I can
make? In C, for example, I'd try to allocate the memory and check for
whether it was allocated, then cut the array size if it wasn't. Is
there an equivalent technique in IDL?

2) The reason I want to make these big arrays is that I have sets of
on the order of 20,000 data curves (50-200 pts each). I'd like to
reduce these to a set of "common curves" -- around 100 averaged/
extracted/smoothed curves which are representative of the larger set.
The curves are complex -- I don't have anything to fit to them -- and
they are noisy. But I get the feeling that if I handed someone a
stack of 20,000 of them and said "sort these into groups which are
similar" that they'd be able to do it. The question is, is there a
good way to do this programmatically?

My original thought was to calculate an rms difference between each
pair of curves and then reduce by partnering up the curves which are
good matches to each other. Repeat. To do this easily seems to
require big arrays and a pretty large amount of computation time. I'd
appreciate any thoughts.

Thanks,
Eric

Report message to a moderator

[Message index]

		Big arrays, reducing data By: Eric Hudson on Wed, 21 March 2007 10:15
		Re: Big array By: xqinshan on Tue, 05 February 2013 04:35
		Re: Big array By: d.poreh on Tue, 05 February 2013 02:15
		Re: Big array By: Yngvar Larsen on Tue, 05 February 2013 00:45
		Re: Big array By: xqinshan on Mon, 04 February 2013 15:57
		Re: Big array By: lecacheux.alain on Mon, 04 February 2013 06:34
		Re: Big array By: Matthew on Mon, 04 February 2013 06:31
		Re: Big array By: David Fanning on Mon, 04 February 2013 06:11

Previous Topic:	Re: Request for guidance/advice related to time series analysis and forecasting using MODIS imagery
Next Topic:	Re: Constructing an exact Euclidean distance map

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 15:53:57 PDT 2025

Total time taken to generate the page: 0.00476 seconds