comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: rounding errors

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: rounding errors [message #24919 is a reply to message #24917]

Fri, 27 April 2001 06:44

Craig Markwardt
Messages: 1869
Registered: November 1996

Senior Member

"Dominic R. Scales" <Dominic.Scales@aerosensing.de> writes:

> Alex Schuster wrote:
>> You just have to define the 2.56989 as a double: b = 2.56989d :)
>>
>> Remember that the floating point value 2.56989 is not the same as the
>> real number 2.56989000000000000..., but just some value around
>> 2.5698900. Converting it to a double introduces more digits to the
>> right, but they are pretty random.
>>
>> Alex
>> --
>> Alex Schuster Wonko@weird.cologne.de PGP Key available
>> alex@pet.mpin-koeln.mpg.de
>
> Danke Alex :)
>
> the prints were more of an example. In the 'real' world,
> the data is read in from a file with readu and is an array
> of float variables. I want to perform the following maths
> in double but with the cast to double I already introduce
> 'pretty random digits to the right', as you say.
> I'ld really like to avoid calling something like double(string(a))
> for some large array...

In principle you should not have to worry about those extra random
digits. Since your original data are floating point, then they
contain no more than 7 digits of precision. Any digits beyond the 7th
are simply *undefined*. If you want higher precision then you should
recreate your file using double precision arithmetic from the start.

Even if you use double precision then you will still have random
digits at the end, but just farther out:

IDL> print, 2.56989d, format='(D30.20)'
2.56989000000000000767

This is a consequence of how numbers are represented in the base-two
number system. Generally speaking the relative uncertainty will be
(MACHAR()).EPS for floating point in IDL.

Craig

--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------

Report message to a moderator

[Message index]

		Re: rounding errors By: thompson on Fri, 27 April 2001 11:34
		Re: rounding errors By: Liam E. Gumley on Fri, 27 April 2001 10:01
		Re: rounding errors By: Karl Schultz on Fri, 27 April 2001 09:07
		Re: rounding errors By: Paul van Delst on Fri, 27 April 2001 08:47
		Re: rounding errors By: Craig Markwardt on Fri, 27 April 2001 08:52
		Re: rounding errors By: John-David T. Smith on Fri, 27 April 2001 08:39
		Re: rounding errors By: thompson on Fri, 27 April 2001 08:23
		Re: rounding errors By: Dominic R. Scales on Fri, 27 April 2001 07:41
		Re: rounding errors By: Randall Skelton on Fri, 27 April 2001 07:51
		Re: rounding errors By: Liam E. Gumley on Fri, 27 April 2001 07:21
		Re: rounding errors By: Craig Markwardt on Fri, 27 April 2001 06:44
		Re: rounding errors By: Randall Skelton on Fri, 27 April 2001 06:17
		Re: rounding errors By: Dominic R. Scales on Fri, 27 April 2001 04:38
		Re: rounding errors By: Alex Schuster on Fri, 27 April 2001 03:52
		Re: rounding errors By: James Kuyper on Wed, 02 May 2001 08:06
		Re: rounding errors By: James Kuyper on Wed, 02 May 2001 08:07

Previous Topic:	Re: AAARGH-2 !!! CALL_EXTERNAL and IDL5.2 (the solution)
Next Topic:	TLB-Widget focus

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 11:49:31 PDT 2025

Total time taken to generate the page: 1.60109 seconds