comp.lang.idl-pvwave archive: archive » Re: bizarre number transformation

Home » Public Forums » archive » Re: bizarre number transformation

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: bizarre number transformation [message #31480 is a reply to message #31478]

Thu, 25 July 2002 12:22

Paul Van Delst[1]
Messages: 1157
Registered: April 2002

Senior Member

Michael Ganzer wrote:
>
> As plenty postings already were dealing about how to use a double precision
> number i wanted to ask u something different...
>
> Whatever you do with 443496.984 in multiplication or something else.....
> does it really matter at that number size if there is more than one digit
> exact after the digit separator???

My goodness. 443496.984 is not a "big" number. What if you have to add it to 0.004657?

<snip>

> And numeric approximations like in computer calculations often produce
> higher mistake percentages than u expect...

Usually due to people thinking computers can represent floating point numbers exactly. The
most important word in your sentence is "approximations"

> And your "almost half a million" for a calculation already is a big
> number...

If the number represented dollars (or euros...which are probably worth more now :o), I
agree. It's big.

If the number represents the number of water vapour molecules per metre^3 it's amazingly
teeny tiny.

paulv

--
Paul van Delst
CIMSS @ NOAA/NCEP/EMC Beer is good.
Ph: (301)763-8000 x7274 My wife.
Fax:(301)763-8545

Report message to a moderator

[Message index]

		Re: bizarre number transformation By: R.Bauer on Thu, 25 July 2002 23:58
		Re: bizarre number transformation By: Craig Markwardt on Thu, 25 July 2002 21:41
		Re: bizarre number transformation By: David Fanning on Thu, 25 July 2002 21:02
		Re: bizarre number transformation By: James Kuyper on Thu, 25 July 2002 12:58
		Re: bizarre number transformation By: Paul Van Delst[1] on Thu, 25 July 2002 12:40
		Re: bizarre number transformation By: David Fanning on Thu, 25 July 2002 12:40
		Re: bizarre number transformation By: Paul Van Delst[1] on Thu, 25 July 2002 12:22
		Re: bizarre number transformation By: Michael Ganzer on Thu, 25 July 2002 11:48
		Re: bizarre number transformation By: Paul Van Delst[1] on Thu, 25 July 2002 08:50
		Re: bizarre number transformation By: James Kuyper on Thu, 25 July 2002 06:51
		Re: bizarre number transformation By: wilms on Thu, 25 July 2002 05:11
		Re: bizarre number transformation By: Don J Lindler on Thu, 25 July 2002 05:25
		Re: bizarre number transformation By: James Kuyper on Fri, 26 July 2002 06:53
		Re: bizarre number transformation By: James Kuyper on Fri, 26 July 2002 06:47

Previous Topic:	activeX?
Next Topic:	Re: Trip to Ol' Stomping Grounds

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Thu Oct 09 19:00:24 PDT 2025

Total time taken to generate the page: 0.47969 seconds