| Re: Numbers from nowhere? [message #58725] |
Sun, 17 February 2008 13:53  |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
elwood writes:
> But my question is more pointed:
> if you assign x=3.3
> and you know apriori that the floating point data type
> will not have enough bits to store this number precisely,
> why does "print" show this number as 3.3?
>
> Is it because it is rounding off with some kind of algorithm?
> Or am I misinterpreting how this number is being stored?
I presume it is because whatever number *is* stored,
when rounded to the 7-8 significant figures a float
can accurately represent, comes out to 3.300000.
IDL> x=3.3
IDL> print, x, format='(f0)'
3.300000
IDL> print, x, format='(f12.10)'
3.2999999523
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming (www.dfanning.com)
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Numbers from nowhere? [message #58726 is a reply to message #58725] |
Sun, 17 February 2008 12:19  |
elwood
Messages: 23
Registered: February 2007
|
Junior Member |
|
|
I'm glad that the Coyotes have something useful to do, besides keeping
me awake
all night in the Anza Borrego desert (it was wonderful to hear them
all night actually!)
But my question is more pointed:
if you assign x=3.3
and you know apriori that the floating point data type
will not have enough bits to store this number precisely,
why does "print" show this number as 3.3?
Is it because it is rounding off with some kind of algorithm?
Or am I misinterpreting how this number is being stored?
Curious,
Elisha
On Feb 15, 4:51 pm, David Fanning <n...@dfanning.com> wrote:
> Paul van Delst writes:
>> Cosmic rays randomly flipping the bits beyond the stated precision?
>
> Teams of coyotes, given jobs like this to satisfy
> their playful nature, but still keep them off the
> street, more likely.
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming (www.dfanning.com)
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
|
|
|
|
|
|
| Re: Numbers from nowhere? [message #58780 is a reply to message #58725] |
Thu, 21 February 2008 05:06 |
Sven Utcke
Messages: 10
Registered: October 2007
|
Junior Member |
|
|
David Fanning <news@dfanning.com> writes:
> elwood writes:
>
>> But my question is more pointed: if you assign x=3.3 and you know
>> apriori that the floating point data type will not have enough bits
>> to store this number precisely, why does "print" show this number
>> as 3.3?
>
> I presume it is because whatever number *is* stored, when rounded to
> the 7-8 significant figures a float can accurately represent, comes
> out to 3.300000.
What number _is_ stored, actually? Assuming we are talking ieee, we
have one bit for the sign, 8 for the exponent, and 23 for the
mantissa. So what is 3.3?
3 = 11 = 1.1 * 2^1
0.3 = 0.0100110011001100110011001100110011001100...
which we see from
0.3*2 = _0_.6
0.6*2 = _1_.2
0.2*2 = _0_.4
0.4*2 = _0_.8
0.8*2 = _1_.6
0.6*2 = ...
so we get, combined,
3.3 = 1.10100110011001100110011 * 2^1
or
S | Exp + 127 | Mantissa without leading 1
0 | 1000000 | 1010011 00110011 00110011
which, if we recombine it, turns out to be 3.2999999523162841796875
We can actually see this in IDL too:
IDL> print, byte(3.3,0,4)
51 51 83 64
Which, if we rewrite it appropriately, turns out to be:
01000000 01010011 00110011 00110011
which, recombined differently, is the above number :-)
Sven
--
___ _ _____ ___ Dr.-Ing. Sven Utcke ___ ___ _____ __
/ __| |/ / __| __| phone: +49 40 8998-5317 | \| __/ __\ \ / /
| (_ | ' <\__ \__ \ fax : +49 40 8994-5317 (NEW) | |) | _|\__ \\ V /
\___|_|\_\___|___/ http://www.desy.de/~utcke (to come)|___/|___|___/ |_|
|
|
|
|
| Re: Numbers from nowhere? [message #58920 is a reply to message #58780] |
Mon, 25 February 2008 09:57 |
elwood
Messages: 23
Registered: February 2007
|
Junior Member |
|
|
Thanks,
very succinct and clear explanation.
Fun with Binary and Computer Precision is what I summarize
the topic as ;-)
-Elisha
On Feb 21, 7:06 am, Sven Utcke <utcke+n...@informatik.uni-hamburg.de>
wrote:
> David Fanning <n...@dfanning.com> writes:
>> elwood writes:
>
>>> But my question is more pointed: if you assign x=3.3 and you know
>>> apriori that the floating point data type will not have enough bits
>>> to store this number precisely, why does "print" show this number
>>> as 3.3?
>
>> I presume it is because whatever number *is* stored, when rounded to
>> the 7-8 significant figures a float can accurately represent, comes
>> out to 3.300000.
>
> What number _is_ stored, actually? Assuming we are talking ieee, we
> have one bit for the sign, 8 for the exponent, and 23 for the
> mantissa. So what is 3.3?
>
> 3 = 11 = 1.1 * 2^1
> 0.3 = 0.0100110011001100110011001100110011001100...
> which we see from
>
> 0.3*2 = _0_.6
> 0.6*2 = _1_.2
> 0.2*2 = _0_.4
> 0.4*2 = _0_.8
> 0.8*2 = _1_.6
> 0.6*2 = ...
>
> so we get, combined,
>
> 3.3 = 1.10100110011001100110011 * 2^1
>
> or
>
> S | Exp + 127 | Mantissa without leading 1
> 0 | 1000000 | 1010011 00110011 00110011
>
> which, if we recombine it, turns out to be 3.2999999523162841796875
>
> We can actually see this in IDL too:
>
> IDL> print, byte(3.3,0,4)
> 51 51 83 64
> Which, if we rewrite it appropriately, turns out to be:
>
> 01000000 01010011 00110011 T
> which, recombined differently, is the above number :-)
>
> Sven
> --
> ___ _ _____ ___ Dr.-Ing. Sven Utcke ___ ___ _____ __
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
