comp.lang.idl-pvwave archive: archive » Memory management when concatenating arrays

Home » Public Forums » archive » Memory management when concatenating arrays

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Memory management when concatenating arrays [message #92186 is a reply to message #92185]

Wed, 28 October 2015 06:58

Alain Kattnig
Messages: 9
Registered: November 2009

Junior Member

Le mercredi 28 octobre 2015 14:49:38 UTC+1, rj...@le.ac.uk a écrit :
> I have a large multi-dimensional array that is split across several files by time.
>
> i.e. file1 contains the first 1000 timesteps, [360,180,1000], file2 contains the next 1000 timesteps [360,180,1000], etc.
>
> What I want to end up with is one big array that's read say all 10 files in and is (360, 180, 10000).
>
> What I'm doing is this in a loop:
>
> all_data=[[[all_data]], [[data]]]
>
> But I quickly run out of memory trying to concatenate in this way.
>
> I tried using temporary
>
> all_data=[[[temporary(all_data)]], [[data]]]
>
> but this doesn't help.
>
> Is there an efficient way of doing this?
>
> Cheers

Use ASSOC, it associates a variable to a file, thus allowing you to address unlimited siezd variables

Report message to a moderator

[Message index]

		Memory management when concatenating arrays By: rjp23 on Wed, 28 October 2015 06:49
		Re: Memory management when concatenating arrays By: Alain Kattnig on Wed, 28 October 2015 06:58
		Re: Memory management when concatenating arrays By: greg.addr on Wed, 28 October 2015 07:16
		Re: Memory management when concatenating arrays By: Phillip Bitzer on Wed, 28 October 2015 07:41
		Re: Memory management when concatenating arrays By: Yngvar Larsen on Wed, 28 October 2015 07:44
		Re: Memory management when concatenating arrays By: Yngvar Larsen on Wed, 28 October 2015 07:54

Previous Topic:	Display of hyperspectral image in the window of a specified size
Next Topic:	interpolation

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Nov 28 08:35:20 PST 2025

Total time taken to generate the page: 0.13042 seconds