comp.lang.idl-pvwave archive: archive » Re: machine precision

Home » Public Forums » archive » Re: machine precision

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: machine precision [message #66495 is a reply to message #66494]

Tue, 19 May 2009 18:41

jameskuyper
Messages: 79
Registered: October 2007

Member

Jeff N. wrote:
...
> I think I have the beginnings of an understanding of this problem
> after having seen the topic pop up so much here in the newsgroup and
> after having read David's article several times. But now i'm
> wondering how other languages handle this situation? I've tried two
> other languages now, perl and VB, and they both seem to not give the
> "correct-but-not-expected" results that trip so many people up. For
> example, this snippet of perl code:
>
> $i = int(4.70*100);
> print "$i\n";
>
> prints 470 whereas the equivalent IDL code: print, fix(4.70*100)
> prints 469.

You've misunderstood the problem if you think that proves that the
problem doesn't occur in perl. Just because perl gave you the result you
expected in this case doesn't mean that it always will. When I typed
'man perlfun' on my home machine and searched for "int EXPR', I found:

"You should not use this function for rounding: one because it truncates
towards 0, and two because machine representations of floating point
numbers can sometimes produce counterintuitive results. For example,
"int(-6.725/0.025)" produces -268 rather than the correct -269; that’s
because it’s really more like -268.99999999999994315658 instead."

> Does anyone have any idea how other languages deal with the "sky is
> falling" problem?

The "sky is falling problem" is fundamentally a problem with
programmer's expectations being out of line with the way floating point
numbers actually work. The solution is eduction, there's really not a
lot that the language can do about it. For this specific kind of
problem, if you've got a value 'x' that should be a number close to an
integer, but which might be slightly more or slightly less than the
exact integer value, fix(x) is the wrong way to handle it. round() is
more appropriate.

Report message to a moderator

[Message index]

		Re: machine precision By: Wout De Nolf on Wed, 20 May 2009 06:08
		Re: machine precision By: Carsten Lechte on Wed, 20 May 2009 03:44
		Re: machine precision By: jameskuyper on Wed, 20 May 2009 03:28
		Re: machine precision By: Carsten Lechte on Wed, 20 May 2009 01:46
		Re: machine precision By: David Fanning on Tue, 19 May 2009 20:01
		Re: machine precision By: jameskuyper on Tue, 19 May 2009 18:41
		Re: machine precision By: Kenneth P. Bowman on Tue, 19 May 2009 11:15
		Re: machine precision By: jeffnettles4870 on Tue, 19 May 2009 08:16
		Re: machine precision By: jameskuyper on Mon, 18 May 2009 18:44
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 07:02
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 06:55
		Re: machine precision By: jameskuyper on Mon, 18 May 2009 06:33
		Re: machine precision By: David Fanning on Mon, 18 May 2009 06:30
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 06:04
		Re: machine precision By: David Fanning on Mon, 18 May 2009 04:53

Previous Topic:	Re: Simultaneous fitting in IDL
Next Topic:	Simultaneous fitting in IDL

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 08:56:49 PDT 2025

Total time taken to generate the page: 0.00700 seconds