comp.lang.idl-pvwave archive: archive » Re: Could you explain why this happen?

Home » Public Forums » archive » Re: Could you explain why this happen?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Could you explain why this happen? [message #70707 is a reply to message #70706]

Wed, 28 April 2010 10:04

Michael Galloy
Messages: 1114
Registered: April 2006

Senior Member

On 4/28/10 10:46 AM, Bryan wrote:
> This may be a stupid question, but I really want to know why.
> Please, see below and explain. Thanks.
>
> IDL> print, 132*30
> 3960
> IDL> print, 132*30*10
> -25936
> IDL> print, 132*300
> -25936
> IDL> data1 = 132
> IDL> data2 = 300
> IDL> data3 = data1*data2
> IDL> print, data3
> -25936

The default integer in IDL is a 16-bit signed integer (i.e., INT), range
-32768 to 32767. Your calculation involved two of these 16-bit integers,
so IDL calculated a result which was a 16-bit integer, wrapping around
to negative values in the process:

IDL> help, 132
<Expression> INT = 132
IDL> help, 300
<Expression> INT = 300
IDL> help, 132 * 300
<Expression> INT = -25936

The way to fix this to get the "correct" result is to make one or both
of the operands in the operation be of type long, making the result of
type long (range -2^31 to 2^31 - 1):

IDL> help, 132L * 300L
<Expression> LONG = 39600

Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation

Report message to a moderator

[Message index]

		Re: Could you explain why this happen? By: Carsten Lechte on Thu, 29 April 2010 03:57
		Re: Could you explain why this happen? By: David Fanning on Wed, 28 April 2010 12:26
		Re: Could you explain why this happen? By: bryan.s.hong on Wed, 28 April 2010 10:54
		Re: Could you explain why this happen? By: David Fanning on Wed, 28 April 2010 10:06
		Re: Could you explain why this happen? By: Michael Galloy on Wed, 28 April 2010 10:04
		Re: Could you explain why this happen? By: jeanh on Wed, 28 April 2010 10:01

Previous Topic:	Re: How to read field attributes in HDF-EOS
Next Topic:	ms2gt MODIS reprojection toolkit

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 21:41:53 PDT 2025

Total time taken to generate the page: 0.47489 seconds