comp.lang.idl-pvwave archive: archive » Re: Numbers from nowhere?

Home » Public Forums » archive » Re: Numbers from nowhere?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Numbers from nowhere? [message #58738 is a reply to message #58737]

Fri, 15 February 2008 13:12

bennette
Messages: 1
Registered: February 2008

Junior Member

On Feb 15, 10:08 am, Conor <cmanc...@gmail.com> wrote:
> I'm just curious:
>
> print,3.3,format='(f30.28)'
> 3.2999999523162841796875000000
>
> Obviously float is only good to ~6 decimal places, but IDL will still
> print out additional digits. It seems that it does reach a point
> where IDL will only print out zeroes. I'm just curious - where do
> these extra digits come from? Does it come from rounding errors
> converting binary to decimal? Or does it come from some other
> nefarious location?

I think that you hit the nail on the head. There is a nice article on
decimal to binary conversion on David Fanning's sight.
http://www.dfanning.com/math_tips/sky_is_falling.html

Eric

----------------
Eric Bennett
National Institutes of Health
bennette at nih.nhlbi.gov

Report message to a moderator

[Message index]

		Re: Numbers from nowhere? By: David Fanning on Sun, 17 February 2008 13:53
		Re: Numbers from nowhere? By: elwood on Sun, 17 February 2008 12:19
		Re: Numbers from nowhere? By: David Fanning on Fri, 15 February 2008 14:51
		Re: Numbers from nowhere? By: Paul Van Delst[1] on Fri, 15 February 2008 13:31
		Re: Numbers from nowhere? By: bennette on Fri, 15 February 2008 13:12
		Re: Numbers from nowhere? By: Sven Utcke on Thu, 21 February 2008 05:06
		Re: Numbers from nowhere? By: elwood on Mon, 25 February 2008 09:57

Previous Topic:	Labeling the x-axis with a non standard time string
Next Topic:	idl format codes and output files

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Nov 30 07:14:19 PST 2025

Total time taken to generate the page: 1.60445 seconds