| IDL mathematics [message #2705] |
Wed, 31 August 1994 10:49  |
dean
Messages: 55
Registered: March 1993
|
Member |
|
|
How come the following expression gives the wrong answer.
J = 201+((1461*(1994+4799))/4)-(3*((1994+4899)/100)/4)-2465022
J = -2457713
It should be:
J = 201+((1461.*(1994+4799.))/4.)-(3.*((1994+4899.)/100.)/4.)-24 65022.
J = 16270.5
Kelly Dean
|
|
|
|
| Re: IDL mathematics [message #2886 is a reply to message #2705] |
Fri, 02 September 1994 04:32  |
pjclinch
Messages: 27
Registered: May 1993
|
Junior Member |
|
|
dean@phobos.cira.colostate.edu wrote:
: How come the following expression gives the wrong answer.
: J = 201+((1461*(1994+4799))/4)-(3*((1994+4899)/100)/4)-2465022
: J = -2457713
: It should be:
: J = 201+((1461.*(1994+4799.))/4.)-(3.*((1994+4899.)/100.)/4.)-24 65022.
: J = 16270.5
The system starts off assuming j will be an integer, because that's what all the
arguments are. After a while, it runs out of room in a 16 bit integer and
converts to a long, where it stays, having acquired some novel errors on the way.
To get what you *really* want, tell IDL/Wave you expect a floating point
calculation, which is achieved by making the arguments floating point to begin
with, so:
j=201.0+((1461.0* etc etc. will give you the right answer.
If you want to see exactly where the problem lies, follow through your original
calculation step by step and check the *type* of j after each step. This should
throw some light onto how and why your original went wrong.
Pete.
--
Peter Clinch University of Dundee
voice 44 382 60111 x 2604 Department of Medical Physics
fax 44 382 640177 Ninewells Hospital
email p.j.clinch@dundee.ac.uk Dundee, DD1 9SY, Scotland, UK
|
|
|
|
| Re: IDL mathematics [message #2898 is a reply to message #2705] |
Thu, 01 September 1994 05:13 |
sjt
Messages: 72
Registered: November 1993
|
Member |
|
|
dean@phobos.cira.colostate.edu wrote:
: How come the following expression gives the wrong answer.
: J = 201+((1461*(1994+4799))/4)-(3*((1994+4899)/100)/4)-2465022
: J = -2457713
: It should be:
: J = 201+((1461.*(1994+4799.))/4.)-(3.*((1994+4899.)/100.)/4.)-24 65022.
: J = 16270.5
: Kelly Dean
The problem is IDL's conversion rules. You have two problems here:
a) 1461*(1994+4799) is a combination of 3 ints and is thus evaluated as
an int whereas it needs to be a long or float to get the right answer
b) Division of integers is an integer devide e.g. 3/2 = 1 and 4/5 = 0.
I think the minimal modification to get the right answer is:
J = 201+((1461.*(1994+4799))/4)-(3*((1994+4899)/100.)/4)-2465022
^ ^
--
James Tappin, School of Physics & Space Research
University of Birmingham
sjt@xun8.sr.bham.ac.uk
"If all else fails--read the instructions!"
O__
-- \/`
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
