comp.lang.idl-pvwave archive: archive » Locating sequence of bytes within binary file

Home » Public Forums » archive » Locating sequence of bytes within binary file

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Locating sequence of bytes within binary file [message #71444 is a reply to message #71334]

Fri, 18 June 2010 15:12

JDS
Messages: 94
Registered: March 2009

Member

On Jun 18, 12:47 pm, Axel Martínez <axe...@gmail.com> wrote:
>> As usual, we can (over-)use IDL's array strengths to brute-force this
>> using REBIN and dimensional TOTAL:
>
> The problem of this solution might be the size of the variable quill
> for a large "needle" (not my case).

Memory usage is actually fairly stable with needle size, and
comparable to the other solution. The larger the needle, the fewer
the rows. Put another way, "quill" is always of order the size of the
search array, and no larger.

> And also the speed, as you acknowledge...

This is where a large needle hurts you; the need to wrap the search
array around in n_needle steps.

>> That motivates another deeply unsatisfying, but resoundingly faster
>> (20-50x) option: simply replace all 0b's with 1b's in both input byte
>> array and search array, and just double-check for spurious matches as
>> you go:
>
> This is good, but it just uses double the amount of memory to preserve
> the original and the modified array. Works for me, but could be bad
> for others. And, honestly, it is really not so elegant.

Agreed. That said, if elegance is on your short list of programming
language features, I fear you shouldn't go anywhere near IDL. I think
the other message here is that finding where a given value occurs in
an array is, although seemingly just a short extrapolation away,
actually a completely different problem from finding where a sequence
of values appears. The latter has considerable room for optimization
over the former. For example, you might notice that STRPOS runs in a
time that is almost independent of needle length. Some algorithms run
*faster* the longer the needle: http://en.wikipedia.org/wiki/BoyerMoore_string_search_algori thm.
Brains always triumph over brute-force.

>> We might also lobby ITT to let STRPOS and its sort accept byte arrays
>> (since frankly there is very little difference between them
>> internally).
>
> This would be by far the best solution. I believe a VIP like you have
> more options to lobby ITT than I do :)

I don't know about that. Most new features start with a simple
request to ITT support; give it a try.

JD

Report message to a moderator

[Message index]

		Locating sequence of bytes within binary file By: medd on Tue, 15 June 2010 04:30
		Re: Locating sequence of bytes within binary file By: JDS on Thu, 17 June 2010 10:48
		Re: Locating sequence of bytes within binary file By: Axel Martínez on Fri, 18 June 2010 09:47
		Re: Locating sequence of bytes within binary file By: medd on Thu, 17 June 2010 01:03
		Re: Locating sequence of bytes within binary file By: penteado on Wed, 16 June 2010 10:46
		Re: Locating sequence of bytes within binary file By: David Fanning on Fri, 18 June 2010 15:21
		Re: Locating sequence of bytes within binary file By: kamal on Thu, 24 June 2010 01:42
		Re: Locating sequence of bytes within binary file By: JDS on Fri, 18 June 2010 15:12

Previous Topic:	pixel information using envi+idl
Next Topic:	WINDOW: Routine is not defined for current graphics device.

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Nov 28 21:37:00 PST 2025

Total time taken to generate the page: 0.16292 seconds