comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: rounding errors

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: rounding errors [message #24922 is a reply to message #24921]

Fri, 27 April 2001 04:38

Dominic R. Scales
Messages: 12
Registered: April 2001

Junior Member

Alex Schuster wrote:
> You just have to define the 2.56989 as a double: b = 2.56989d :)
>
> Remember that the floating point value 2.56989 is not the same as the
> real number 2.56989000000000000..., but just some value around
> 2.5698900. Converting it to a double introduces more digits to the
> right, but they are pretty random.
>
> Alex
> --
> Alex Schuster Wonko@weird.cologne.de PGP Key available
> alex@pet.mpin-koeln.mpg.de

Danke Alex :)

the prints were more of an example. In the 'real' world,
the data is read in from a file with readu and is an array
of float variables. I want to perform the following maths
in double but with the cast to double I already introduce
'pretty random digits to the right', as you say.
I'ld really like to avoid calling something like double(string(a))
for some large array...

Any ideas?

Schï¿½nes langes Wochenende,
Dominic

[ Tuesday is a holiday, so I'll take Monday off :) ]
--
Dipl. Phys. Dominic R. Scales | Aero-Sensing Radarsysteme GmbH
Tel: +49 (0)8153-90 88 90 | c/o DLR Oberpfaffenhofen
Fax: +49 (0)8153-908 700 | 82234 Wessling, Germany
WWW: aerosensing.de | email: Dominic.Scales@aerosensing.de

Report message to a moderator

[Message index]

		Re: rounding errors By: thompson on Fri, 27 April 2001 11:34
		Re: rounding errors By: Liam E. Gumley on Fri, 27 April 2001 10:01
		Re: rounding errors By: Karl Schultz on Fri, 27 April 2001 09:07
		Re: rounding errors By: Paul van Delst on Fri, 27 April 2001 08:47
		Re: rounding errors By: Craig Markwardt on Fri, 27 April 2001 08:52
		Re: rounding errors By: John-David T. Smith on Fri, 27 April 2001 08:39
		Re: rounding errors By: thompson on Fri, 27 April 2001 08:23
		Re: rounding errors By: Dominic R. Scales on Fri, 27 April 2001 07:41
		Re: rounding errors By: Randall Skelton on Fri, 27 April 2001 07:51
		Re: rounding errors By: Liam E. Gumley on Fri, 27 April 2001 07:21
		Re: rounding errors By: Craig Markwardt on Fri, 27 April 2001 06:44
		Re: rounding errors By: Randall Skelton on Fri, 27 April 2001 06:17
		Re: rounding errors By: Dominic R. Scales on Fri, 27 April 2001 04:38
		Re: rounding errors By: Alex Schuster on Fri, 27 April 2001 03:52
		Re: rounding errors By: James Kuyper on Wed, 02 May 2001 08:06
		Re: rounding errors By: James Kuyper on Wed, 02 May 2001 08:07

Previous Topic:	Re: AAARGH-2 !!! CALL_EXTERNAL and IDL5.2 (the solution)
Next Topic:	TLB-Widget focus

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 08:34:56 PDT 2025

Total time taken to generate the page: 1.04100 seconds