| Absolute value of negative number is negative [message #92691] |
Mon, 08 February 2016 11:00  |
wlandsman
Messages: 743
Registered: June 2000
|
Senior Member |
|
|
I was surprised to find that taking the absolute value of a 16 bit integer vector was yielding negative numbers.
While this is non-intuitive, I think it is unavoidable. --Wayne
IDL> tab= -32768S ;IDL won't let us define -32768 as a short integer
tab= -32768
^
% Integer constant must be less than 32768.
IDL> tab = intarr(1) ;but we can assign a short integer to -32768
IDL> tab[0] = -32768
IDL> help,tab ;Still a short integer
TAB INT = Array[1]
IDL> print,tab[0] ;Still a value of -32768
-32768
IDL> print,abs(tab[0]) ;Absolute value is negative!
-32768
|
|
|
|
| Re: Absolute value of negative number is negative [message #92698 is a reply to message #92691] |
Tue, 09 February 2016 13:26  |
Dick Jackson
Messages: 347
Registered: August 1998
|
Senior Member |
|
|
On Monday, 8 February 2016 11:00:20 UTC-8, wlandsman wrote:
> I was surprised to find that taking the absolute value of a 16 bit integer vector was yielding negative numbers.
> While this is non-intuitive, I think it is unavoidable. --Wayne
>
>
> IDL> tab= -32768S ;IDL won't let us define -32768 as a short integer
>
> tab= -32768
> ^
> % Integer constant must be less than 32768.
>
> IDL> tab = intarr(1) ;but we can assign a short integer to -32768
> IDL> tab[0] = -32768
> IDL> help,tab ;Still a short integer
> TAB INT = Array[1]
> IDL> print,tab[0] ;Still a value of -32768
> -32768
> IDL> print,abs(tab[0]) ;Absolute value is negative!
> -32768
Good find, Wayne. That's the trouble when there are more available negative values than positive ones.
In case this is helpful, here's an easier way to get that constant:
IDL> tab = Fix(-32768)
IDL> help, tab, Abs(tab)
TAB INT = -32768
<Expression> INT = -32768
And, to show the problem appears on larger integer types as well:
IDL> tab = Fix(2L^15L)
IDL> help, tab, Abs(tab)
TAB INT = -32768
<Expression> INT = -32768
IDL> tab = 2L^31
IDL> help, tab, Abs(tab)
TAB LONG = -2147483648
<Expression> LONG = -2147483648
IDL> tab = 2LL^63
IDL> help, tab, Abs(tab)
TAB LONG64 = -9223372036854775808
<Expression> LONG64 = -9223372036854775808
Cheers,
-Dick
Dick Jackson Software Consulting Inc.
Victoria, BC, Canada --- http://www.d-jackson.com
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
