comp.lang.idl-pvwave archive: archive » Re: machine precision

Home » Public Forums » archive » Re: machine precision

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: machine precision [message #66499 is a reply to message #66497]

Mon, 18 May 2009 18:44

jameskuyper
Messages: 79
Registered: October 2007

Member

Wox wrote:
> On Mon, 18 May 2009 13:33:48 GMT, James Kuyper
> <jameskuyper@verizon.net> wrote:
>
>> At a minimum, you should use pres*(f1^2+f2^2)^0.5 instead of pres for
>> the following comparison. eps gives you the relative precision, not the
>> absolute precision; it needs to be scaled by the numbers you're working
>> with.
>
> Now this I didn't realize! So it's the precision of the mantisse or
> something?

Yes. If you don't understand why, I recommend re-reading the "The Sky is
Falling!" website that David referred you to, and in particular the
paper it refers to titled "What Every Computer Scientist Should Know
about Floating Point Arithmetic".

Report message to a moderator

[Message index]

		Re: machine precision By: Wout De Nolf on Wed, 20 May 2009 06:08
		Re: machine precision By: Carsten Lechte on Wed, 20 May 2009 03:44
		Re: machine precision By: jameskuyper on Wed, 20 May 2009 03:28
		Re: machine precision By: Carsten Lechte on Wed, 20 May 2009 01:46
		Re: machine precision By: David Fanning on Tue, 19 May 2009 20:01
		Re: machine precision By: jameskuyper on Tue, 19 May 2009 18:41
		Re: machine precision By: Kenneth P. Bowman on Tue, 19 May 2009 11:15
		Re: machine precision By: jeffnettles4870 on Tue, 19 May 2009 08:16
		Re: machine precision By: jameskuyper on Mon, 18 May 2009 18:44
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 07:02
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 06:55
		Re: machine precision By: jameskuyper on Mon, 18 May 2009 06:33
		Re: machine precision By: David Fanning on Mon, 18 May 2009 06:30
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 06:04
		Re: machine precision By: David Fanning on Mon, 18 May 2009 04:53

Previous Topic:	Re: Simultaneous fitting in IDL
Next Topic:	Simultaneous fitting in IDL

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 15:32:40 PDT 2025

Total time taken to generate the page: 0.40171 seconds