comp.lang.idl-pvwave archive: archive » Memory management when concatenating arrays

Home » Public Forums » archive » Memory management when concatenating arrays

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Memory management when concatenating arrays [message #92185]

Wed, 28 October 2015 06:49

rjp23
Messages: 97
Registered: June 2010

Member

I have a large multi-dimensional array that is split across several files by time.

i.e. file1 contains the first 1000 timesteps, [360,180,1000], file2 contains the next 1000 timesteps [360,180,1000], etc.

What I want to end up with is one big array that's read say all 10 files in and is (360, 180, 10000).

What I'm doing is this in a loop:

all_data=[[[all_data]], [[data]]]

But I quickly run out of memory trying to concatenate in this way.

I tried using temporary

all_data=[[[temporary(all_data)]], [[data]]]

but this doesn't help.

Is there an efficient way of doing this?

Cheers

Report message to a moderator

[Message index]

		Memory management when concatenating arrays By: rjp23 on Wed, 28 October 2015 06:49
		Re: Memory management when concatenating arrays By: Alain Kattnig on Wed, 28 October 2015 06:58
		Re: Memory management when concatenating arrays By: greg.addr on Wed, 28 October 2015 07:16
		Re: Memory management when concatenating arrays By: Phillip Bitzer on Wed, 28 October 2015 07:41
		Re: Memory management when concatenating arrays By: Yngvar Larsen on Wed, 28 October 2015 07:44
		Re: Memory management when concatenating arrays By: Yngvar Larsen on Wed, 28 October 2015 07:54

Previous Topic:	Display of hyperspectral image in the window of a specified size
Next Topic:	interpolation

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 16:03:03 PDT 2025

Total time taken to generate the page: 0.00402 seconds