comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » double precision?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Double precision [message #65792 is a reply to message #49718]

Thu, 19 March 2009 01:18

Chris[6]
Messages: 84
Registered: July 2008

Member

On Mar 18, 9:36 pm, plim.dream...@gmail.com wrote:
> Well, I can't make sense of it.
> I read the article and if my case is in there I can't find it.
> Seems simple my problem.
> I am reading in a bunch of numbers, the input file has those numbers
> as
> 100.489418 10.512547
> 100.489718 10.512558
> and so on
> I read in those numbers with DOUBLE
> i remove some pairs which don't interest me and the rest I print them
> to an output file and they turn up as
> 100.48942 10.512547
> 100.48942 10.512558
>
> i just want the precision to be the same in the output as in the
> input. same number. that's all i ask. I notice that if I read in
> those numbers as FLOAT instead of DOUBLE then it was even worse. So
> it is true I am assuming that the DOUBLE is truncating/rounding-off/
> whatever.
>
> thanks for the help thus far.
> P
>
> On Mar 18, 10:31 pm, David Fanning <n...@dfanning.com> wrote:
>
>> plim.dream...@gmail.com writes:
>>> Later in the program I calculate the separation between points (x1,y1)
>>> (x2,y2)
>>> And for some of those points the program says that the pairs are the
>>> same. But they are only the same if they are rounded off, the
>>> difference often only shows up in the last 2 decimal places.
>>> ya, i read that link, most of it at least.
>
>> I hope you aren't comparing floats with the EQ operator.
>> Maybe you should read that article again. All the way
>> to the end this time. :-)
>
>> You might try Floats_Equal. You will get better results,
>> probably:
>
>> http://www.dfanning.com/program/floats_equal.pro
>
>>> One other thing is: lets say the print out is the issue; a case like
>>> you pointed out above, then why is it that if I do:
>>> b=3Dstring(num)
>>> print,b will give me the rounded off number?
>
>> It isn't rounding off the number. The number is the
>> number. It is printing the number in 8 significant
>> figures, which is the default format. Give it another
>> format and it will do something else.
>
>> Cheers,
>
>> David
>> --
>> David Fanning, Ph.D.
>> Fanning Software Consulting, Inc.
>> Coyote's Guide to IDL Programming:http://www.dfanning.com/
>> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
>
>

maybe include the line of code where you read in the file? I could see
accidentally converting the number to a float on input before
promoting it to a double, which would cause the roundoff.

chris

Report message to a moderator

[Message index]

		double precision? By: adisn123 on Thu, 10 August 2006 13:18
		Re: Double precision By: David Fanning on Thu, 19 March 2009 21:28
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 21:08
		Re: Double precision By: David Fanning on Thu, 19 March 2009 12:21
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 12:02
		Re: Double precision By: Jean H. on Thu, 19 March 2009 05:55
		Re: Double precision By: David Fanning on Thu, 19 March 2009 05:34
		Re: Double precision By: Allan Whiteford on Thu, 19 March 2009 03:51
		Re: Double precision By: Chris[6] on Thu, 19 March 2009 01:18
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 00:36
		Re: Double precision By: David Fanning on Wed, 18 March 2009 22:31
		Re: Double precision By: plim.dreaming on Wed, 18 March 2009 22:16
		Re: Double precision By: David Fanning on Wed, 18 March 2009 21:32

Previous Topic:	Re: Mac Widget Limitation?
Next Topic:	Re: Span a graph across 2 spaces in a !P.MULTI environment?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:00:16 PDT 2025

Total time taken to generate the page: 0.00569 seconds