comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » IDL mathematics

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: IDL mathematics [message #2886 is a reply to message #2705]

Fri, 02 September 1994 04:32

pjclinch
Messages: 27
Registered: May 1993

Junior Member

dean@phobos.cira.colostate.edu wrote:

: How come the following expression gives the wrong answer.

: J = 201+((1461*(1994+4799))/4)-(3*((1994+4899)/100)/4)-2465022

: J = -2457713

: It should be:

: J = 201+((1461.*(1994+4799.))/4.)-(3.*((1994+4899.)/100.)/4.)-24 65022.

: J = 16270.5

The system starts off assuming j will be an integer, because that's what all the
arguments are. After a while, it runs out of room in a 16 bit integer and
converts to a long, where it stays, having acquired some novel errors on the way.

To get what you *really* want, tell IDL/Wave you expect a floating point
calculation, which is achieved by making the arguments floating point to begin
with, so:

j=201.0+((1461.0* etc etc. will give you the right answer.

If you want to see exactly where the problem lies, follow through your original
calculation step by step and check the *type* of j after each step. This should
throw some light onto how and why your original went wrong.

Pete.
--
Peter Clinch University of Dundee
voice 44 382 60111 x 2604 Department of Medical Physics
fax 44 382 640177 Ninewells Hospital
email p.j.clinch@dundee.ac.uk Dundee, DD1 9SY, Scotland, UK

Report message to a moderator

[Message index]

		IDL mathematics By: dean on Wed, 31 August 1994 10:49
		Re: IDL mathematics By: pjclinch on Fri, 02 September 1994 04:32
		Re: IDL mathematics By: sjt on Thu, 01 September 1994 05:13

Previous Topic:	WIDGETS - AGAIN
Next Topic:	Re: WIDGETS - AGAIN

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Thu Oct 09 21:49:45 PDT 2025

Total time taken to generate the page: 7.68094 seconds