comp.lang.idl-pvwave archive: archive » Re: double-single precision problem?????

Home » Public Forums » archive » Re: double-single precision problem?????

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: double-single precision problem????? [message #9408]

Mon, 30 June 1997 00:00

Peter Mason
Messages: 145
Registered: June 1996

Senior Member

On Fri, 27 Jun 1997, Eugenio Sansosti wrote:
> I'm using IDL. Version 4.0.1 (AIX ibmr2).
> Try to type
> print,double(13.1e0)-13.1d0
> you will get
> 3.8146973e-07......
> Is that normal?

Yes, that's par for the course with floating point arithmetic.

There isn't an exact binary floating-point representation for 13.1 - in the
same way that there isn't an exact decimal representation for 1/3.
Specifying "DOUBLE(13.1e0)" just appends zero digits onto the single-precision
value. This result is different to (less accurate than) what you get if you
express 13.1 in double precision from the start (as in "13.1D0"), and so you
don't get zero when you subtract the two values.

Peter Mason

Report message to a moderator

Previous Topic:	Lowest INTERNATIONAL Calling RATES Available !!
Next Topic:	Re: IDL 5.0 - call_external under Win95/NT

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Nov 29 18:19:10 PST 2025

Total time taken to generate the page: 1.60371 seconds