comp.lang.idl-pvwave archive: archive » Re: fix(4.70*100) is... 469

Home » Public Forums » archive » Re: fix(4.70*100) is... 469

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: fix(4.70*100) is... 469 [message #53551 is a reply to message #53550]

Thu, 19 April 2007 09:54

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

Jean H. writes:

> What is being displayed is a bit different than what is being stored..
> IDL> print, 470.0 - (4.70*100)
> 3.05176e-005

I guess I would argue that what is being displayed is EXACTLY
what is being stored:

IDL> print, 470, format='(f18.14)'
470.00000000000000
IDL> print, 4.70*100, format='(f18.14)'
469.99996948242187
IDL> print, 470.00000000000000D - 469.99996948242187D, format='(f18.14)'
0.00003051757813

Cheers,

David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

[Message index]

		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 11:47
		Re: fix(4.70*100) is... 469 By: mmeron on Thu, 19 April 2007 11:15
		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 11:13
		Re: fix(4.70*100) is... 469 By: David Fanning on Thu, 19 April 2007 12:07
		Re: fix(4.70*100) is... 469 By: David Fanning on Thu, 19 April 2007 11:52
		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 10:44
		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 10:03
		Re: fix(4.70*100) is... 469 By: mmeron on Thu, 19 April 2007 10:00
		Re: fix(4.70*100) is... 469 By: mmeron on Thu, 19 April 2007 09:50
		Re: fix(4.70*100) is... 469 By: David Fanning on Thu, 19 April 2007 09:54
		Re: fix(4.70*100) is... 469 By: Paolo Grigis on Fri, 20 April 2007 01:04
		Re: fix(4.70*100) is... 469 By: Jean H. on Thu, 19 April 2007 08:35
		Re: fix(4.70*100) is... 469 By: David Fanning on Thu, 19 April 2007 09:35
		Re: fix(4.70*100) is... 469 By: Christopher Thom on Thu, 19 April 2007 08:21
		Re: fix(4.70*100) is... 469 By: eddedmondson on Thu, 19 April 2007 08:13
		Re: fix(4.70*100) is... 469 By: Paolo Grigis on Thu, 19 April 2007 08:02
		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 07:41
		Re: fix(4.70*100) is... 469 By: b_efremova@yahoo.com on Thu, 19 April 2007 07:32
		Re: fix(4.70*100) is... 469 By: David Fanning on Thu, 19 April 2007 07:48
		Re: fix(4.70*100) is... 469 By: David Fanning on Thu, 19 April 2007 07:42
		Re: fix(4.70*100) is... 469 By: KRDean on Thu, 19 April 2007 06:41
		Re: fix(4.70*100) is... 469 By: Paolo Grigis on Thu, 19 April 2007 01:13
		Re: fix(4.70*100) is... 469 By: mmeron on Wed, 18 April 2007 22:56
		Re: fix(4.70*100) is... 469 By: Qing on Wed, 18 April 2007 21:38
		Re: fix(4.70*100) is... 469 By: Qing on Wed, 18 April 2007 21:36
		Re: fix(4.70*100) is... 469 By: Qing on Wed, 18 April 2007 21:31
		Re: fix(4.70*100) is... 469 By: Jean H. on Wed, 18 April 2007 17:58
		Re: fix(4.70*100) is... 469 By: mmeron on Wed, 18 April 2007 17:14
		Re: fix(4.70*100) is... 469 By: Jean H. on Wed, 18 April 2007 16:48
		Re: fix(4.70*100) is... 469 By: David Fanning on Wed, 18 April 2007 17:12
		Re: fix(4.70*100) is... 469 By: mmeron on Fri, 20 April 2007 00:33
		Re: fix(4.70*100) is... 469 By: Qing on Thu, 19 April 2007 23:43
		Re: fix(4.70*100) is... 469 By: Paul Van Delst[1] on Thu, 19 April 2007 12:06

Previous Topic:	sparsab, sparsax and complex arrays
Next Topic:	Re: idl 6.1 crashes on linux

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:03:22 PDT 2025

Total time taken to generate the page: 0.00226 seconds