comp.lang.idl-pvwave archive: archive » Re: Faster way to search/split a string?

Home » Public Forums » archive » Re: Faster way to search/split a string?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Faster way to search/split a string? [message #78478 is a reply to message #78473]

Wed, 23 November 2011 06:40

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

Rob writes:

> I was hoping that someone might have a cleverer way of approaching
> this problem.
>
> The following command is the bootleneck in my code:
>
> row_of_data=strsplit(all_rows[(where(stregex(all_rows, id, /boolean)
> eq 1))[0]],' ', /extract)
>
> I have a large text file with lots of columns of data (which I don't
> know exactly what the columns are until I've read them in). There are
> then say 10000 rows of this data.
>
> This is all read into one large string array (all_rows) which contains
> each row as a single very long string.
> The first 20 characters of the row contain a unique id which I need to
> search the rows for and then extract the entire matching row. This row
> then needs to be split up into it's columns (space delimited).
>
> Hopefully that all makes sense.
>
> The problem is having to do this 10000 times, (once for for each id)
> is very slow and the time to do all of the other stuff done in the
> code, reading, writing, some maths, etc is being dominated by this one
> line.
>
> Any thoughts or suggestions?

I guess I would try just reading the IDs. Then sort the IDs,
and use Value_Locate to find all the unique IDs at once. Then
you could subset your string array and do the string extraction.
Hard to tell if this would be faster.

> p.s. This needs to be GDL compatible as well which I think most
> solutions would be anyway.

This is like telling most of us "be sure it works in the
Martian atmosphere." How the hell would I know!? :-)

Cheers,

David

--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

[Message index]

		Re: Faster way to search/split a string? By: rjp23 on Thu, 24 November 2011 06:41
		Re: Faster way to search/split a string? By: rjp23 on Thu, 24 November 2011 06:16
		Re: Faster way to search/split a string? By: wlandsman on Thu, 24 November 2011 01:15
		Re: Faster way to search/split a string? By: David Fanning on Wed, 23 November 2011 10:21
		Re: Faster way to search/split a string? By: Vincent Sarago on Wed, 23 November 2011 10:17
		Re: Faster way to search/split a string? By: Jeremy Bailin on Wed, 23 November 2011 08:02
		Re: Faster way to search/split a string? By: rjp23 on Wed, 23 November 2011 07:37
		Re: Faster way to search/split a string? By: David Fanning on Wed, 23 November 2011 06:40

Previous Topic:	Faster way to search/split a string?
Next Topic:	Contour dimension problem

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Thu Dec 04 02:05:13 PST 2025

Total time taken to generate the page: 0.01227 seconds