comp.lang.idl-pvwave archive: archive » Re: array concatenation and optimization

Home » Public Forums » archive » Re: array concatenation and optimization

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: array concatenation and optimization [message #26779 is a reply to message #26777]

Wed, 26 September 2001 22:05

alt
Messages: 28
Registered: August 2001

Junior Member

Our coryphaei are seem to keep silence so I will try to help you
...:-)

> Is there a reason this is so slow?
Because in each concatenation step IDL initializes new variable with
memory freeing, allocation, coping, etc. And it becomes slower and
slower with increasing of array size.
> Is there a better way to do this?
Using temporary() will not help as it seems to.

Very obvious solution is to allocate array of maximum possible size
for data and clip it to filled size at the end.

arr = bytarr(10000,/nozero)
i = 0L
while ... do begin
arr[i] = item
i = i + 1
endwhile
arr = temporary(arr[0:i-1])

This way is not always good because one have to think of maximum
number of data and allocate huge superfluous memory. The next idea is
to lengthen array by the lump only on demand. I am using for this
purpose my simple "AS IS" procedure.

pro AddItem, arr, Q, item, step, clear = clear
; arr - array of data
; Q - number of valid elements in array
; item - item to be added
; step - size of superfluous elements of array
; /clear - initialize arr and Q (if they exist before)
if keyword_set(clear) then begin
if n_elements(arr) NE 0 then tmp = temporary(arr)
Q = 0L
return
endif
Qitem = n_elements(item)
Qarr = n_elements(arr)
if Qarr EQ 0 then begin
arr = [item,replicate(item[0],step)] ; first
add to arr
Q = Qitem
endif else begin
if Qarr GE (Q+Qitem) then $
arr[Q:Q+Qitem-1] = item $ ; item fit
in arr
else $
arr = [arr[0:Q-1],item,replicate(item[0],step)] ; item do not
fit in arr, extending
Q = Q + Qitem
endelse
end

AddItem usage:

AddItem, arr, Qarr, /clear

while ... do begin
AddItem, arr, Qarr, item, 10000L
endwhile

arr = temporary(arr[0:Qarr-1])

You can rewrite AddItem as object if you will. I guess it is done
already by someone.

What is interesting for me is file version of this procedure.
Sometimes I need file that keeps different data arrays. I need to add,
insert, and delete items from arrays in this file. And it should be
relatively fast. I understand that this task is solved by database
management systems but often using DBMS seems very excessive. And it
would be nice to have save/restore style of parameters setting. Does
anyone have written something like that?

Regards,
Altyntsev Dmitriy

Report message to a moderator

[Message index]

		Re: array concatenation and optimization By: R.Bauer on Thu, 27 September 2001 00:25
		Re: array concatenation and optimization By: alt on Wed, 26 September 2001 22:05
		Re: array concatenation and optimization By: Andrew Cool on Wed, 26 September 2001 16:49
		Re: array concatenation and optimization By: Pavel A. Romashkin on Wed, 26 September 2001 14:51
		Re: array concatenation and optimization By: Brian Jackel on Wed, 26 September 2001 14:29
		Re: array concatenation and optimization By: Paul van Delst on Wed, 26 September 2001 12:59
		Re: array concatenation and optimization By: R.G.S. on Wed, 26 September 2001 12:22
		Re: array concatenation and optimization By: Craig Markwardt on Thu, 27 September 2001 09:34

Previous Topic:	Re: IDL 5.4 and win2000 problems
Next Topic:	IDL for Windows and MAC...

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 06:26:36 PDT 2025

Total time taken to generate the page: 0.23854 seconds