comp.lang.idl-pvwave archive
Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado

Home » Public Forums » archive » Re: fix(4.70*100) is... 469
Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend 
Return to the default flat view Create a new topic Submit Reply
Re: fix(4.70*100) is... 469 [message #53575 is a reply to message #53574] Wed, 18 April 2007 16:48 Go to previous messageGo to previous message
Jean H. is currently offline  Jean H.
Messages: 472
Registered: July 2006
Senior Member
> Hooboy! Haven't had one of these in awhile. :-)
>
> http://www.dfanning.com/math_tips/sky_is_falling.html
>
> Cheers,
>
> David

It seems a bit weirder though...

IDL> print, 470.0 - (4.70*100)
3.05176e-005

which is bigger than the smallest float

IDL> print, (machar()).eps
1.19209e-007

.... so how can it be the float accuracy problem if the difference
between the expected and the real value is 256 times bigger than the
float error?

Jean
[Message index]
 
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Previous Topic: sparsab, sparsax and complex arrays
Next Topic: Re: idl 6.1 crashes on linux

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Fri Oct 10 05:32:12 PDT 2025

Total time taken to generate the page: 1.68342 seconds