comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » double precision?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Double precision [message #65793 is a reply to message #49718]

Thu, 19 March 2009 00:36

plim.dreaming
Messages: 22
Registered: February 2008

Junior Member

Well, I can't make sense of it.
I read the article and if my case is in there I can't find it.
Seems simple my problem.
I am reading in a bunch of numbers, the input file has those numbers
as
100.489418 10.512547
100.489718 10.512558
and so on
I read in those numbers with DOUBLE
i remove some pairs which don't interest me and the rest I print them
to an output file and they turn up as
100.48942 10.512547
100.48942 10.512558

i just want the precision to be the same in the output as in the
input. same number. that's all i ask. I notice that if I read in
those numbers as FLOAT instead of DOUBLE then it was even worse. So
it is true I am assuming that the DOUBLE is truncating/rounding-off/
whatever.

thanks for the help thus far.
P

On Mar 18, 10:31 pm, David Fanning <n...@dfanning.com> wrote:
> plim.dream...@gmail.com writes:
>> Later in the program I calculate the separation between points (x1,y1)
>> (x2,y2)
>> And for some of those points the program says that the pairs are the
>> same. But they are only the same if they are rounded off, the
>> difference often only shows up in the last 2 decimal places.
>> ya, i read that link, most of it at least.
>
> I hope you aren't comparing floats with the EQ operator.
> Maybe you should read that article again. All the way
> to the end this time. :-)
>
> You might try Floats_Equal. You will get better results,
> probably:
>
> http://www.dfanning.com/program/floats_equal.pro
>
>> One other thing is: lets say the print out is the issue; a case like
>> you pointed out above, then why is it that if I do:
>> b=3Dstring(num)
>> print,b will give me the rounded off number?
>
> It isn't rounding off the number. The number is the
> number. It is printing the number in 8 significant
> figures, which is the default format. Give it another
> format and it will do something else.
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

[Message index]

		double precision? By: adisn123 on Thu, 10 August 2006 13:18
		Re: Double precision By: David Fanning on Thu, 19 March 2009 21:28
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 21:08
		Re: Double precision By: David Fanning on Thu, 19 March 2009 12:21
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 12:02
		Re: Double precision By: Jean H. on Thu, 19 March 2009 05:55
		Re: Double precision By: David Fanning on Thu, 19 March 2009 05:34
		Re: Double precision By: Allan Whiteford on Thu, 19 March 2009 03:51
		Re: Double precision By: Chris[6] on Thu, 19 March 2009 01:18
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 00:36
		Re: Double precision By: David Fanning on Wed, 18 March 2009 22:31
		Re: Double precision By: plim.dreaming on Wed, 18 March 2009 22:16
		Re: Double precision By: David Fanning on Wed, 18 March 2009 21:32

Previous Topic:	Re: Mac Widget Limitation?
Next Topic:	Re: Span a graph across 2 spaces in a !P.MULTI environment?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 13:48:34 PDT 2025

Total time taken to generate the page: 0.23995 seconds