comp.lang.idl-pvwave archive: archive » Re: comparing and concatenating arrays...please help!!

Home » Public Forums » archive » Re: comparing and concatenating arrays...please help!!

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: comparing and concatenating arrays...please help!! [message #37590 is a reply to message #37587]

Thu, 08 January 2004 13:22

Michael Wallace
Messages: 409
Registered: December 2003

Senior Member

> Hi Pepijn,
>
> Many thanks for your reply.
>
> My files are _huge_. File 1 is about 250000 lines and file2 about half
> that. which is why my code was taking about 4 days!
>
> I'll give the sorting a try, but if anyone else has any suggestions,
> they'll be gratefully received. I'm not a newbie, but am daunted by
> some of the operations you suggested below... I certainly take
> inspiration from all you guys though!

I'd try the sorting attack myself -- sort first and then merge the
sorted data. I don't know what kind of memory and processing
constraints you're under, so this might not be applicable if you have
sufficient memory. If you run into a memory problem trying to sort this
much data, you can divide the sort into chunks. Read and sort the first
X many lines, then read and sort the next X many, and so on. Then take
all the chunks and merge them. Finally, merge of the sorted file1 and
sorted file2. While there will be more operations than the first
approach, the running time will still be on the order of n*log(n) which
is still better than your original running time of n^2, and you will
chew up a lot less memory. Again, this is just a suggestion if you run
into memory problems trying to do such a large sort.

Mike W

Report message to a moderator

[Message index]

		Re: comparing and concatenating arrays...please help!! By: R.Bauer on Fri, 09 January 2004 09:55
		Re: comparing and concatenating arrays...please help!! By: btt on Fri, 09 January 2004 07:30
		Re: comparing and concatenating arrays...please help!! By: wmconnolley on Fri, 09 January 2004 01:50
		Re: comparing and concatenating arrays...please help!! By: Michael Wallace on Thu, 08 January 2004 15:36
		Re: comparing and concatenating arrays...please help!! By: Pepijn Kenter on Thu, 08 January 2004 14:57
		Re: comparing and concatenating arrays...please help!! By: Michael Wallace on Thu, 08 January 2004 13:22
		Re: comparing and concatenating arrays...please help!! By: m.doyle on Thu, 08 January 2004 12:26
		Re: comparing and concatenating arrays...please help!! By: Pepijn Kenter on Thu, 08 January 2004 05:25

Previous Topic:	Passing from from IDL to external lib (DLM)
Next Topic:	Re: Problem masking multiple regions.

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 06:01:49 PDT 2025

Total time taken to generate the page: 1.44044 seconds