| Unsigned Integer Math Problem [message #70531] |
Wed, 21 April 2010 07:12  |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Folks,
I've run into a problem with my Histoplot code this morning.
It is extremely important to the Histogram command that the
data type of the BINSIZE argument be the same as the data
type of the data for which the histogram is being calculated.
I don't know why this is the case, but it is.
In any case, I'm extremely careful about this. But this
is giving me a problem when I try to make a histogram plot
of an image that is stored as unsigned integers (UINT).
Basically, to make my plot I take the output minimum
and maximum from the histogram command and subtract (or
add) a full binsize to those numbers to give the X axis
range of the plot.
My problem is this. The OMIN of the histogram is 0, the
binsize is 726.
IDL> help, omin, binsize
OMIN UINT = 0
BINSIZE UINT = 726
When I make the calculation for the minimum data range
of my axis, I do this:
IDL> min_xrange = omin - binsize
IDL> Help, min_xrange
MIN_XRANGE UINT = 64810
Now, this causes the minimum x range to be larger than the
maximum x range and results in complete chaos downstream.
Clearly, I don't want the minimum x range value to be less
than zero in this case, but I also don't want to force the
value to be zero if the minimum I want is somewhat higher
than this, say 1200. How do I test for this? Clearly, this
does not work:
min_xrange = (omin - binsize) > 0
Since this number 64810 *is* larger than zero, and WRONG!
I guess my real question is this: How do I do arithmetic
operations with unsigned integers in a way that preserves
the nature of unsigned integers?
Any ideas on this?
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Unsigned Integer Math Problem [message #70601 is a reply to message #70531] |
Thu, 22 April 2010 04:10  |
MC
Messages: 50
Registered: September 1996
|
Member |
|
|
When you subtract unsigned numbers you are on a sticky wicket. Before
_any_ subtraction you need to be sure of the range so it needs a test
like:
If (binsize gt omin) then xrange =0 else ...
In assembler you catch the incorrect negative result with the flag
(which is how you implement "gt" anyway in assembler)...
Cheers
On Apr 22, 2:12 am, David Fanning <n...@dfanning.com> wrote:
> Folks,
>
> I've run into a problem with my Histoplot code this morning.
>
> It is extremely important to the Histogram command that the
> data type of the BINSIZE argument be the same as the data
> type of the data for which the histogram is being calculated.
> I don't know why this is the case, but it is.
>
> In any case, I'm extremely careful about this. But this
> is giving me a problem when I try to make a histogram plot
> of an image that is stored as unsigned integers (UINT).
>
> Basically, to make my plot I take the output minimum
> and maximum from the histogram command and subtract (or
> add) a full binsize to those numbers to give the X axis
> range of the plot.
>
> My problem is this. The OMIN of the histogram is 0, the
> binsize is 726.
>
> IDL> help, omin, binsize
> OMIN UINT = 0
> BINSIZE UINT = 726
>
> When I make the calculation for the minimum data range
> of my axis, I do this:
>
> IDL> min_xrange = omin - binsize
> IDL> Help, min_xrange
> MIN_XRANGE UINT = 64810
>
> Now, this causes the minimum x range to be larger than the
> maximum x range and results in complete chaos downstream.
>
> Clearly, I don't want the minimum x range value to be less
> than zero in this case, but I also don't want to force the
> value to be zero if the minimum I want is somewhat higher
> than this, say 1200. How do I test for this? Clearly, this
> does not work:
>
> min_xrange = (omin - binsize) > 0
>
> Since this number 64810 *is* larger than zero, and WRONG!
>
> I guess my real question is this: How do I do arithmetic
> operations with unsigned integers in a way that preserves
> the nature of unsigned integers?
>
> Any ideas on this?
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
