comp.lang.idl-pvwave archive: archive » Efficient pattern-matching in a large array

Home » Public Forums » archive » Efficient pattern-matching in a large array

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Efficient pattern-matching in a large array [message #77156 is a reply to message #77061]

Tue, 02 August 2011 13:04

Foldy Lajos
Messages: 268
Registered: October 2001

Senior Member

On Tue, 2 Aug 2011, Chris O'Dell wrote:

> I have a byte array of tens of millions values (read from a binary
> file). I want to find an 8-byte pattern where ever it occurs. Similar
> to where, but 8 values at a time instead of one. The problem is
> finding a way to do this that is as fast as possible. The pattern is:
> [234, 203, 138, 216, 21, 52, 117, 39].
>
> Right now, I have this in a big loop and it takes a while. We tried
> "match2" but it ran out of memory.
>
> Thanks,
> Chris
>
>

Hint: use ULONG64 (8 byte) comparison:

n=1000
barr=bindgen(n)
bpat=[42b, 43b, 44b, 45b, 46b, 47b, 48b, 49b]
lpat=fix(bpat, 0, type=15)

for j=0,7 do begin
larr=fix(barr, j, j eq 0 ? n/8 : n/8-1, type=15)
w=where(larr eq lpat, count)
if count gt 0 then print, 'found:', 8*w+j
endfor

regards,
Lajos

Report message to a moderator

[Message index]

		Efficient pattern-matching in a large array By: Chris O'Dell on Tue, 02 August 2011 11:05
		Re: Efficient pattern-matching in a large array By: Foldy Lajos on Tue, 02 August 2011 13:04
		Re: Efficient pattern-matching in a large array By: ez569x on Fri, 12 August 2011 05:18
		Re: Efficient pattern-matching in a large array By: Brian Daniel on Fri, 12 August 2011 07:38
		Re: Efficient pattern-matching in a large array By: guillermo.castilla.ca on Tue, 16 August 2011 09:41
		Re: Efficient pattern-matching in a large array By: guillermo.castilla.ca on Wed, 17 August 2011 06:41
		Re: Efficient pattern-matching in a large array By: JDS on Tue, 16 August 2011 16:01
		Re: Efficient pattern-matching in a large array By: Bob[4] on Wed, 17 August 2011 14:14
		Re: Efficient pattern-matching in a large array By: JDS on Thu, 18 August 2011 10:00

Previous Topic:	Re: /RELAXED keyword not allowed?
Next Topic:	Confession

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 13:44:51 PDT 2025

Total time taken to generate the page: 0.71449 seconds