| Numbers from nowhere? [message #58742] |
Fri, 15 February 2008 07:08  |
Conor
Messages: 138
Registered: February 2007
|
Senior Member |
|
|
I'm just curious:
print,3.3,format='(f30.28)'
3.2999999523162841796875000000
Obviously float is only good to ~6 decimal places, but IDL will still
print out additional digits. It seems that it does reach a point
where IDL will only print out zeroes. I'm just curious - where do
these extra digits come from? Does it come from rounding errors
converting binary to decimal? Or does it come from some other
nefarious location?
|
|
|
|
| Re: Numbers from nowhere? [message #58774 is a reply to message #58742] |
Thu, 21 February 2008 08:29  |
Norbert Hahn
Messages: 46
Registered: May 2003
|
Member |
|
|
Sven Utcke <utcke+news@informatik.uni-hamburg.de> wrote:
> David Fanning <news@dfanning.com> writes:
>
>> elwood writes:
>>
>>> But my question is more pointed: if you assign x=3.3 and you know
>>> apriori that the floating point data type will not have enough bits
>>> to store this number precisely, why does "print" show this number
>>> as 3.3?
>>
>> I presume it is because whatever number *is* stored, when rounded to
>> the 7-8 significant figures a float can accurately represent, comes
>> out to 3.300000.
>
> What number _is_ stored, actually? Assuming we are talking ieee, we
> have one bit for the sign, 8 for the exponent, and 23 for the
> mantissa. So what is 3.3?
>
> 3 = 11 = 1.1 * 2^1
> 0.3 = 0.0100110011001100110011001100110011001100...
Note that the above pattern repeats forever and *must* be truncated
in the real world (of computers) as much as 1/3 is not 0.3333 in
decimal.
[snip]
>
> S | Exp + 127 | Mantissa without leading 1
> 0 | 1000000 | 1010011 00110011 00110011
>
> which, if we recombine it, turns out to be 3.2999999523162841796875
... because we only store 24 binary mantissa digits. The formatting
routine in IDL most likely works with double precission and hence
appends binary zeroes rather than continuing to repeat the pattern 0011.
This results in a decimal number being slightly less than 3.30000000.
The problem arises from the conversion between binary to decimal numbers.
It is not a problem of IDL but a problem of working with numbers of
finite length.
Sven, thanks for you elaboration!
Norbert
|
|
|
|
| Re: NUMBERS [message #61171 is a reply to message #58742] |
Wed, 09 July 2008 22:02 |
d.poreh
Messages: 406
Registered: October 2007
|
Senior Member |
|
|
On 9 Jul., 05:33, Ingo von Borstel <newsgro...@planetmaker.de> wrote:
> Hi,
>
>> how i can fix it?(i used long-float and ULL but it is not work)
>> Any HELP????
>
> Either you use long, float or double. The first is an integer number,
> the latter floating point numbers. In IDL there's no such thing as long
> float, though you can convert one type into the other - which might then
> explain your results.
> In order to help you, you should describe *exactly* what you have done
> (e.g. post the necessary code).
>
> Regards,
> Ingo
>
> PS: it doesn't help either to post the same problem 3 times...
>
> --
> Ingo von Borstel <newsgro...@planetmaker.de>
> Public Key:http://www.planetmaker.de/ingo.asc
>
> If you need an urgent reply, replace newsgroups by vgap.
i am so sorry. i don't know what happend yeasterday but i posted one
time and did't see that an thought i made a mistake and post again.
sooo sorry
Cheers
|
|
|
|
| Re: NUMBERS [message #61195 is a reply to message #58742] |
Wed, 09 July 2008 05:33 |
Ingo von Borstel
Messages: 54
Registered: September 2006
|
Member |
|
|
Hi,
> how i can fix it?(i used long-float and ULL but it is not work)
> Any HELP????
Either you use long, float or double. The first is an integer number,
the latter floating point numbers. In IDL there's no such thing as long
float, though you can convert one type into the other - which might then
explain your results.
In order to help you, you should describe *exactly* what you have done
(e.g. post the necessary code).
Regards,
Ingo
PS: it doesn't help either to post the same problem 3 times...
--
Ingo von Borstel <newsgroups@planetmaker.de>
Public Key: http://www.planetmaker.de/ingo.asc
If you need an urgent reply, replace newsgroups by vgap.
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
