| Re: Unsigned Integer Math Problem [message #70601 is a reply to message #70531] |
Thu, 22 April 2010 04:10  |
MC
Messages: 50
Registered: September 1996
|
Member |
|
|
When you subtract unsigned numbers you are on a sticky wicket. Before
_any_ subtraction you need to be sure of the range so it needs a test
like:
If (binsize gt omin) then xrange =0 else ...
In assembler you catch the incorrect negative result with the flag
(which is how you implement "gt" anyway in assembler)...
Cheers
On Apr 22, 2:12 am, David Fanning <n...@dfanning.com> wrote:
> Folks,
>
> I've run into a problem with my Histoplot code this morning.
>
> It is extremely important to the Histogram command that the
> data type of the BINSIZE argument be the same as the data
> type of the data for which the histogram is being calculated.
> I don't know why this is the case, but it is.
>
> In any case, I'm extremely careful about this. But this
> is giving me a problem when I try to make a histogram plot
> of an image that is stored as unsigned integers (UINT).
>
> Basically, to make my plot I take the output minimum
> and maximum from the histogram command and subtract (or
> add) a full binsize to those numbers to give the X axis
> range of the plot.
>
> My problem is this. The OMIN of the histogram is 0, the
> binsize is 726.
>
> IDL> help, omin, binsize
> OMIN UINT = 0
> BINSIZE UINT = 726
>
> When I make the calculation for the minimum data range
> of my axis, I do this:
>
> IDL> min_xrange = omin - binsize
> IDL> Help, min_xrange
> MIN_XRANGE UINT = 64810
>
> Now, this causes the minimum x range to be larger than the
> maximum x range and results in complete chaos downstream.
>
> Clearly, I don't want the minimum x range value to be less
> than zero in this case, but I also don't want to force the
> value to be zero if the minimum I want is somewhat higher
> than this, say 1200. How do I test for this? Clearly, this
> does not work:
>
> min_xrange = (omin - binsize) > 0
>
> Since this number 64810 *is* larger than zero, and WRONG!
>
> I guess my real question is this: How do I do arithmetic
> operations with unsigned integers in a way that preserves
> the nature of unsigned integers?
>
> Any ideas on this?
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
