comp.lang.idl-pvwave archive: archive » Re: Numbers from nowhere?

Home » Public Forums » archive » Re: Numbers from nowhere?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Numbers from nowhere? [message #58780 is a reply to message #58725]

Thu, 21 February 2008 05:06

Sven Utcke
Messages: 10
Registered: October 2007

Junior Member

David Fanning <news@dfanning.com> writes:

> elwood writes:
>
>> But my question is more pointed: if you assign x=3.3 and you know
>> apriori that the floating point data type will not have enough bits
>> to store this number precisely, why does "print" show this number
>> as 3.3?
>
> I presume it is because whatever number *is* stored, when rounded to
> the 7-8 significant figures a float can accurately represent, comes
> out to 3.300000.

What number _is_ stored, actually? Assuming we are talking ieee, we
have one bit for the sign, 8 for the exponent, and 23 for the
mantissa. So what is 3.3?

3 = 11 = 1.1 * 2^1
0.3 = 0.0100110011001100110011001100110011001100...
which we see from

0.3*2 = _0_.6
0.6*2 = _1_.2
0.2*2 = _0_.4
0.4*2 = _0_.8
0.8*2 = _1_.6
0.6*2 = ...

so we get, combined,

3.3 = 1.10100110011001100110011 * 2^1

or

S | Exp + 127 | Mantissa without leading 1
0 | 1000000 | 1010011 00110011 00110011

which, if we recombine it, turns out to be 3.2999999523162841796875

We can actually see this in IDL too:

IDL> print, byte(3.3,0,4)
51 51 83 64
Which, if we rewrite it appropriately, turns out to be:

01000000 01010011 00110011 00110011

which, recombined differently, is the above number :-)

Sven
--
___ _ _____ ___ Dr.-Ing. Sven Utcke ___ ___ _____ __
/ __| |/ / __| __| phone: +49 40 8998-5317 | \| __/ __\ \ / /
| (_ | ' <\__ \__ \ fax : +49 40 8994-5317 (NEW) | |) | _|\__ \\ V /
\___|_|\_\___|___/ http://www.desy.de/~utcke (to come)|___/|___|___/ |_|

Report message to a moderator

[Message index]

		Re: Numbers from nowhere? By: David Fanning on Sun, 17 February 2008 13:53
		Re: Numbers from nowhere? By: elwood on Sun, 17 February 2008 12:19
		Re: Numbers from nowhere? By: David Fanning on Fri, 15 February 2008 14:51
		Re: Numbers from nowhere? By: Paul Van Delst[1] on Fri, 15 February 2008 13:31
		Re: Numbers from nowhere? By: bennette on Fri, 15 February 2008 13:12
		Re: Numbers from nowhere? By: Sven Utcke on Thu, 21 February 2008 05:06
		Re: Numbers from nowhere? By: elwood on Mon, 25 February 2008 09:57

Previous Topic:	Labeling the x-axis with a non standard time string
Next Topic:	idl format codes and output files

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 00:02:27 PDT 2025

Total time taken to generate the page: 0.01032 seconds