comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Inaccuracies

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Inaccuracies [message #5306 is a reply to message #5303]

Tue, 14 November 1995 00:00

Andy Loughe
Messages: 174
Registered: November 1995

Senior Member

> L. Paul Mix wrote:
> I'm not sure what you want to do, but expecting perfect math with
> floats is not generally possible.

I accept the explanation given by Ken Bowman, but it is hard to
explain the values assigned to
(1) findgen(15)*.2 -1.4 versus
(2) dindgen(15)*(.2D)-(1.4D)
(3) and, taken separately, the results of using the total function
on (1) and (2) matched with the ability to perform "perfect math"
with only 13 values.

I am describing a small permutation of inaccuracies here, (1) looks ok,
but (2) does not. total( (1) ) and total( (2) ) are not accurate for
the reasons given by Ken.

BTW one IDL user indicated that there was *no* trouble with this math on
his VMS system! Now that is interesting.

----
Andrew F. Loughe (afl@cdc.noaa.gov)
University of Colorado, CIRES * Campus Box 449 * Boulder, CO 80309
phone: (303) 492-0707 fax: (303) 497-7013

Report message to a moderator

[Message index]

		Re: Inaccuracies By: C R Shaw on Thu, 16 November 1995 00:00
		Re: Inaccuracies By: Hermann Mannstein on Thu, 16 November 1995 00:00
		Re: Inaccuracies By: wclodius on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: Andy Loughe on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: thompson on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: hahn on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: lpmix on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: Andy Loughe on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: wclodius on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: thompson on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: Jackel on Tue, 14 November 1995 00:00

Previous Topic:	Re: How can I add parameter to curvefit?
Next Topic:	XLoadCT in a procedure ?????

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:21:44 PDT 2025

Total time taken to generate the page: 0.00435 seconds