| Re: histogram and UINT [message #70127 is a reply to message #70126] |
Wed, 17 March 2010 11:36  |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article
<ab4b613a-31e0-48ab-9472-6c840fb0fafa@o30g2000yqb.googlegroups.com>,
Jeremy Bailin <astroconst@gmail.com> wrote:
> I've discovered an interesting "feature" in histogram... I'm sure it's
> the expected behaviour, but in this case it had a consequence that
> bit me.
>
> I have some unsigned integer input data that I want to histogram. I'm
> looking at things right around the saturation level, so most of the
> data is exactly 65535, but I also want a bin past that. The following
> works:
>
> h = histogram(data, min=65500ul, max=65535ul)
>
> But the following doesn't:
>
> h = histogram(data, min=65500ul, max=65536ul)
> % HISTOGRAM: Illegal binsize or max/min.
>
> What seems to be happening is that the min/max values are cast into
> the same data type as data, in this case giving a nonsensical max of
> 0u. To do what I want, I need to cast data into a ULONG in this
> histogram call... or use max=65535u and append an appropriate number
> of 0s onto h.
>
> -Jeremy.
It seems logical to me that HISTOGRAM would take type information
from the input variable, not the keywords. (The keywords are optional.)
And since integer arithmetic 'wraps around'
IDL> print, 65535US + 1US
0
the result is to be expected.
I did notice that with IDL 7.1.1 on Mac OS X, using the U suffix
creates a ULONG, not a UINT. This clearly contradicts the documentation.
IDL> help, 65U, 65US
<Expression> ULONG = 65
<Expression> UINT = 65
You have to use US to create a UINT.
Ken Bowman
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
