comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Inaccuracies

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Inaccuracies [message #5311 is a reply to message #5309]

Tue, 14 November 1995 00:00

Jackel
Messages: 30
Registered: April 1993

Member

In article <30A7BC4D.7018@cdc.noaa.gov> Andy Loughe <afl@cdc.noaa.gov> writes:

> Ok, I am sure this has been discussed before, but let
> me start this thread again. I wish to create a 15-element
> vector which contains the numbers -1.4 to 1.4 by an increment
> of 0.2 I also wish the sum of these elements to be zero
> (No, this isn't the new math). Here is what I tried...

Well, this works:
a= (INDGEN(29) - 14) / 10.0d0

but that still doesn't address the fundamental problem. Start with

a= 7.0d0 * 0.2d0 which when PRINTed gives 1.4000000

then

b= a - 1.4000000d0 when PRINTed gives 2.2204460e-016

but

c= a
b= a - c gives 0.0

(note that c= 1.400000d0, then b=a-c gives a non-zero result). So, it
looks like the internal representation of 7.0d0*0.2d0 is not quite 1.4,
but for display purposes IDL does a bit of rounding (truncation?).

So, try

PRINT,a,FORMAT='(f27.25)'

and get

1.4000000000000001000000000

which has a 1d-16 difference. The rest of the puzzle is solved by PRINTing
1.4d0 (with the FORMATing as above):

1.3999999999999999000000000

Basically, it looks like a combination of the usual representation error,
combined with a short default format for output.

Brian Jackel
University of Western Ontario

Report message to a moderator

[Message index]

		Re: Inaccuracies By: C R Shaw on Thu, 16 November 1995 00:00
		Re: Inaccuracies By: Hermann Mannstein on Thu, 16 November 1995 00:00
		Re: Inaccuracies By: wclodius on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: Andy Loughe on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: thompson on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: hahn on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: lpmix on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: Andy Loughe on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: wclodius on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: thompson on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: Jackel on Tue, 14 November 1995 00:00

Previous Topic:	Re: How can I add parameter to curvefit?
Next Topic:	XLoadCT in a procedure ?????

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:35:58 PDT 2025

Total time taken to generate the page: 0.00454 seconds