comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » double precision?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Double precision [message #65777 is a reply to message #49718]

Thu, 19 March 2009 12:02

plim.dreaming
Messages: 22
Registered: February 2008

Junior Member

Thank you for all the useful comments.
My problem, you'll be excited to know, persists.

Allow me say more:

Firstly say what is not the problem:
the problem in not in comparing the numbers. I run one program which
reads in the file and does some stuff to the numbers and then outputs
to another file the selected array. Only after do I read in the
selected array and perform separation comparisons. The problem
already exists though in the output, since the numbers are truncated
or rounded off there.

And I really look at how I input the numbers and it seems fine. I
read them in as double and then I print the numbers with 2f12.7 for
example and they are still fully there, not truncated.

I think the problem is in the printing to the output file.

I have x,y and they are numbers such as 100.912498
then I do a=strcompress(x)
openw,1,'fds'
printf,1,a

so, i am confused because:

print,x(83220),format='(2f12.7)'
100.9126890 which is the full value, the one i want to be printed to
the output
but
a = string(x(83220))
print,a
100.91269 and here it is a string so it doesn't help to do
format='....'

And finally, you guys may be seeing it clearly and trying to explain
this to me so thanks for the patience.
P

On Mar 19, 5:55 am, "Jean H." <jghas...@DELTHIS.ucalgary.ANDTHIS.ca>
wrote:
> plim.dream...@gmail.com wrote:
>> Later in the program I calculate the separation between points (x1,y1)
>> (x2,y2)
>> And for some of those points the program says that the pairs are the
>> same. But they are only the same if they are rounded off, the
>> difference often only shows up in the last 2 decimal places.
>> ya, i read that link, most of it at least.
>> One other thing is: lets say the print out is the issue; a case like
>> you pointed out above, then why is it that if I do:
>> b=string(num)
>> print,b will give me the rounded off number?
>
> as David pointed it out, be careful on how you compare your numbers...
> a = 0.1234567890123456d
> b = 0.1234567890123456d
> print, a eq b
> 1
>
> b = 0.1234567890123456d + 0.1 - 0.1
> print, a eq b
> 0
>
> Also, I suggest you to print your number value a few times (after
> reading, after allocating to pixmag, before comparing it), with the
> proper format ...
> IDL> c = 100.489418d
> IDL> print, c
> 100.48942
> IDL> print, c, format = '(F15.10)'
> 100.4894180000
>
> Jean

Report message to a moderator

[Message index]

		double precision? By: adisn123 on Thu, 10 August 2006 13:18
		Re: Double precision By: David Fanning on Thu, 19 March 2009 21:28
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 21:08
		Re: Double precision By: David Fanning on Thu, 19 March 2009 12:21
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 12:02
		Re: Double precision By: Jean H. on Thu, 19 March 2009 05:55
		Re: Double precision By: David Fanning on Thu, 19 March 2009 05:34
		Re: Double precision By: Allan Whiteford on Thu, 19 March 2009 03:51
		Re: Double precision By: Chris[6] on Thu, 19 March 2009 01:18
		Re: Double precision By: plim.dreaming on Thu, 19 March 2009 00:36
		Re: Double precision By: David Fanning on Wed, 18 March 2009 22:31
		Re: Double precision By: plim.dreaming on Wed, 18 March 2009 22:16
		Re: Double precision By: David Fanning on Wed, 18 March 2009 21:32

Previous Topic:	Re: Mac Widget Limitation?
Next Topic:	Re: Span a graph across 2 spaces in a !P.MULTI environment?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 11:34:53 PDT 2025

Total time taken to generate the page: 0.00453 seconds