comp.lang.idl-pvwave archive: archive » Re: comparing and concatenating arrays...please help!!

Home » Public Forums » archive » Re: comparing and concatenating arrays...please help!!

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: comparing and concatenating arrays...please help!! [message #37587 is a reply to message #37584]

Thu, 08 January 2004 14:57

Pepijn Kenter
Messages: 31
Registered: April 2002

Member

>
> My files are _huge_. File 1 is about 250000 lines and file2 about half
> that. which is why my code was taking about 4 days!

You also might want to consider using another programming language. IDL
is designed for doing (scientific) calculations, not for a task like
this. Loop constructs are particularly slow in IDL.

I can imagine that if you're in a hurry, you start programming in
whatever language you're most familiar with (I'm also lazy in that
respect ;-). But if you've got the time, a project like this is a good
opportunity to learn a new language. IMHO, the only reason for using IDL
for a task like this would be when the input is generated by another IDL
program (or the output is used by an IDL program). In that case the
programs could be integrated thightly if necessary.

Languages designed for processing text files are e.g. perl and awk. If
speed is really important you can use C for everything; but I guess it
would take a lot of effort to beat a specialized language, so better try
these first.

And like I said before, the sort program is very fast in sorting text
files. I've just generated a 480000 line test file. It took 11 seconds
to sort on an atlon 1200.

>
> I'll give the sorting a try, but if anyone else has any suggestions,
> they'll be gratefully received. I'm not a newbie, but am daunted by
> some of the operations you suggested below... I certainly take
> inspiration from all you guys though!
>

If you want to read some more about computational complexity, check out:

http://en2.wikipedia.org/wiki/Computational_complexity_theor y
http://users.forthnet.gr/ath/kimon/CC/CCC1b.htm

Pepijn.

Report message to a moderator

[Message index]

		Re: comparing and concatenating arrays...please help!! By: R.Bauer on Fri, 09 January 2004 09:55
		Re: comparing and concatenating arrays...please help!! By: btt on Fri, 09 January 2004 07:30
		Re: comparing and concatenating arrays...please help!! By: wmconnolley on Fri, 09 January 2004 01:50
		Re: comparing and concatenating arrays...please help!! By: Michael Wallace on Thu, 08 January 2004 15:36
		Re: comparing and concatenating arrays...please help!! By: Pepijn Kenter on Thu, 08 January 2004 14:57
		Re: comparing and concatenating arrays...please help!! By: Michael Wallace on Thu, 08 January 2004 13:22
		Re: comparing and concatenating arrays...please help!! By: m.doyle on Thu, 08 January 2004 12:26
		Re: comparing and concatenating arrays...please help!! By: Pepijn Kenter on Thu, 08 January 2004 05:25

Previous Topic:	Passing from from IDL to external lib (DLM)
Next Topic:	Re: Problem masking multiple regions.

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 19:21:05 PDT 2025

Total time taken to generate the page: 1.20270 seconds