comp.lang.idl-pvwave archive: archive » fix(4.70*100) is... 469

Home » Public Forums » archive » fix(4.70*100) is... 469

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: fix(4.70*100) is... 469 [message #53611 is a reply to message #53578]

Fri, 20 April 2007 09:43

James Kuyper
Messages: 425
Registered: March 2000

Senior Member

Paolo Grigis wrote:
> David Fanning wrote:
...
>> I guess I would argue that what is being displayed is EXACTLY
>> what is being stored:
>
> Is it?
> I always wondered if the garbage after the 16th digit does
> correspond indeed to the exact decimal representation of the
> double stored in memory in binary format, or is just a meaningless
> side product of the binary to decimal conversion...
>
> IDL> print,!DPi,format='(f68.64)'
> 3.1415926535897931159979634685441851615905761718750000000000 000000

The fractional part of a floating point number represented in binary
format can always be written as an integer numerator divided by a
larger power of 2:

numerator
---------------
2^n

It can therefore be represented exactly by a decimal fraction as

numerator*5^n
---------------------
10^n

The are known algorithms for converting a floating point number to a
decimal string which will produce precisely that value, if you tell
them to use enough digits. I don't know any simple way of testing
this, but I would assume that IDL uses such an algorithm.

The extra digits won't make the decimal string a better representation
of pi, but they will make it an accurate representation of the
computer's best binary approximation to pi.

Report message to a moderator

[Message index]

		fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Wed, 18 April 2007 15:48
		Re: fix(4.70*100) is... 469 By: mmeron on Fri, 20 April 2007 10:30
		Re: fix(4.70*100) is... 469 By: Bruce Bowler on Fri, 20 April 2007 09:47
		Re: fix(4.70*100) is... 469 By: James Kuyper on Fri, 20 April 2007 09:43
		Re: fix(4.70*100) is... 469 By: James Kuyper on Fri, 20 April 2007 09:31
		Re: fix(4.70*100) is... 469 By: James Kuyper on Fri, 20 April 2007 09:26
		Re: fix(4.70*100) is... 469 By: Paul Van Delst[1] on Fri, 20 April 2007 07:27
		Re: fix(4.70*100) is... 469 By: Paul Van Delst[1] on Thu, 19 April 2007 13:44
		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 12:43

Previous Topic:	idl 6.1 crashes on linux
Next Topic:	HDF5 data

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 21:26:22 PDT 2025

Total time taken to generate the page: 1.84306 seconds