comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » histogram and UINT

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

histogram and UINT [message #70128]

Wed, 17 March 2010 10:33

Jeremy Bailin
Messages: 618
Registered: April 2008

Senior Member

I've discovered an interesting "feature" in histogram... I'm sure it's
the expected behaviour, but in this case it had a consequence that
bit me.

I have some unsigned integer input data that I want to histogram. I'm
looking at things right around the saturation level, so most of the
data is exactly 65535, but I also want a bin past that. The following
works:

h = histogram(data, min=65500ul, max=65535ul)

But the following doesn't:

h = histogram(data, min=65500ul, max=65536ul)
% HISTOGRAM: Illegal binsize or max/min.

What seems to be happening is that the min/max values are cast into
the same data type as data, in this case giving a nonsensical max of
0u. To do what I want, I need to cast data into a ULONG in this
histogram call... or use max=65535u and append an appropriate number
of 0s onto h.

-Jeremy.

Report message to a moderator

[Message index]

histogram and UINT

By: Jeremy Bailin on Wed, 17 March 2010 10:33

Previous Topic:	Predefined iTool Visualization in existing iTool
Next Topic:	Re: histogram and UINT

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 09:00:41 PDT 2025

Total time taken to generate the page: 1.44060 seconds