comp.lang.idl-pvwave archive: archive » Re: array concatenation and optimization

Home » Public Forums » archive » Re: array concatenation and optimization

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: array concatenation and optimization [message #26792 is a reply to message #26791]

Wed, 26 September 2001 12:22

R.G.S.
Messages: 46
Registered: September 2000

Member

Sean Raffuse <sean@me.wustl.edu> wrote in message
news:9ot613$3ra$1@newsreader.wustl.edu...
> Hello.
>
> I am trying to read a bunch of data from a file to a structure array. I'm
> not sure many data entries the file will have until I have read it and so
I
> am increasing the size of the structure array after reading each line. I
do
> this by concatenating.
>
> adp_struct_single is the structure as a "scalar"
> adp_struct is the array
>
> I concatenate like so:
> adp_struct =[adp_struct, adp_struct_single]
>
> This is working but it has increased the processing time of my loop by an
> order of magnitude. Is there a better way to do this? Is there a reason
> this is so slow?
>
> Thanks in advance.
>
> -Sean Raffuse

Some ideas:
1) estimate array size from file size (get that from fstat), and
create an appropriately size array.

len = fstat(lun).size/nbytesperelement
adp_struct =replicate(adp_struct, len)

2) "buffer the concatenate operation
len = 500 ; blocks of 500
adp_struct_block =replicate(adp_struct, len)
while not ( !eof) do begin
for i = 0,len-1 do begin
if stillreading then adp_struct_block(i) = adp_struct_single
endfor
adp_struct =[adp_struct, adp_struct_block]
endwhile

This is just fakecode, merely illustrating the ideas.

Cheers,
bob

Report message to a moderator

[Message index]

		Re: array concatenation and optimization By: R.Bauer on Thu, 27 September 2001 00:25
		Re: array concatenation and optimization By: alt on Wed, 26 September 2001 22:05
		Re: array concatenation and optimization By: Andrew Cool on Wed, 26 September 2001 16:49
		Re: array concatenation and optimization By: Pavel A. Romashkin on Wed, 26 September 2001 14:51
		Re: array concatenation and optimization By: Brian Jackel on Wed, 26 September 2001 14:29
		Re: array concatenation and optimization By: Paul van Delst on Wed, 26 September 2001 12:59
		Re: array concatenation and optimization By: R.G.S. on Wed, 26 September 2001 12:22
		Re: array concatenation and optimization By: Craig Markwardt on Thu, 27 September 2001 09:34

Previous Topic:	Re: IDL 5.4 and win2000 problems
Next Topic:	IDL for Windows and MAC...

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 19:19:51 PDT 2025

Total time taken to generate the page: 1.41705 seconds