comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » double precision?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Double precision [message #65794 is a reply to message #49718]

Wed, 18 March 2009 22:31

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

plim.dreaming@gmail.com writes:

> Later in the program I calculate the separation between points (x1,y1)
> (x2,y2)
> And for some of those points the program says that the pairs are the
> same. But they are only the same if they are rounded off, the
> difference often only shows up in the last 2 decimal places.
> ya, i read that link, most of it at least.

I hope you aren't comparing floats with the EQ operator.
Maybe you should read that article again. All the way
to the end this time. :-)

You might try Floats_Equal. You will get better results,
probably:

http://www.dfanning.com/program/floats_equal.pro

> One other thing is: lets say the print out is the issue; a case like
> you pointed out above, then why is it that if I do:
> b=3Dstring(num)
> print,b will give me the rounded off number?

It isn't rounding off the number. The number is the
number. It is printing the number in 8 significant
figures, which is the default format. Give it another
format and it will do something else.

Cheers,

David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

[Message index]

		double precision? By: adisn123 on Thu, 10 August 2006 13:18
		Re: Double precision By: David Fanning on Thu, 19 March 2009 21:28
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 21:08
		Re: Double precision By: David Fanning on Thu, 19 March 2009 12:21
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 12:02
		Re: Double precision By: Jean H. on Thu, 19 March 2009 05:55
		Re: Double precision By: David Fanning on Thu, 19 March 2009 05:34
		Re: Double precision By: Allan Whiteford on Thu, 19 March 2009 03:51
		Re: Double precision By: Chris[6] on Thu, 19 March 2009 01:18
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 00:36
		Re: Double precision By: David Fanning on Wed, 18 March 2009 22:31
		Re: Double precision By: plim.dreaming on Wed, 18 March 2009 22:16
		Re: Double precision By: David Fanning on Wed, 18 March 2009 21:32

Previous Topic:	Re: Mac Widget Limitation?
Next Topic:	Re: Span a graph across 2 spaces in a !P.MULTI environment?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 09:10:29 PDT 2025

Total time taken to generate the page: 0.00489 seconds