| double precision? [message #49718] |
Thu, 10 August 2006 13:18  |
adisn123
Messages: 44
Registered: July 2006
|
Member |
|
|
Hi,
The value that I want to keep in a variable, a, is
1.24500000476837.
After I define as
IDL> a = 1.24500000476837
then, print, a
IDL> print, a
IDL> 1.2450000
so, I tried
IDL> a = 1.24500000476837d ;; to make it as a double precision, but
IDL prints
IDL> help, a
A DOUBLE = 1.2450000
IDL> print, a
1.2450000
How can I keep all those decimal points?
I thought double precision can express up to 14 decimal places of
significant points?
|
|
|
|
| Re: Double precision [message #65762 is a reply to message #49718] |
Thu, 19 March 2009 21:28  |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
plim.dreaming@gmail.com writes:
> Ahhhhhhhhhh..... one more thing to add to that page. Muito obrigado!
> I do "need" to do it because i make a output with many columns and if
> it isn't all in string format then IDL (always IDL's fault) moves to
> the next line when it reaches the end of the page... as opposed to
> only moving to the next line at the end of each line of data...
Huh!? Not sure what you are talking about here, but I'm about
99.9% sure you are trying to solve a non-existent problem. :-)
(Maybe you need to know about the WIDTH keyword to the OpenW
command.)
> Anyway, i will go make the necessary changes and thanks again, i'm
> mentioning this IDL crew in my PhD.
Just get us a couple of cases of beer and we'll call it good. ;-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65764 is a reply to message #49718] |
Thu, 19 March 2009 21:08 |
plim.dreaming
Messages: 22
Registered: February 2008
|
Junior Member |
|
|
Ahhhhhhhhhh..... one more thing to add to that page. Muito obrigado!
I do "need" to do it because i make a output with many columns and if
it isn't all in string format then IDL (always IDL's fault) moves to
the next line when it reaches the end of the page... as opposed to
only moving to the next line at the end of each line of data...
Anyway, i will go make the necessary changes and thanks again, i'm
mentioning this IDL crew in my PhD.
P
On Mar 19, 12:21 pm, David Fanning <n...@dfanning.com> wrote:
> plim.dream...@gmail.com writes:
>> Firstly say what is not the problem:
>> the problem in not in comparing the numbers. I run one program which
>> reads in the file and does some stuff to the numbers and then outputs
>> to another file the selected array. Only after do I read in the
>> selected array and perform separation comparisons. The problem
>> already exists though in the output, since the numbers are truncated
>> or rounded off there.
>
>> And I really look at how I input the numbers and it seems fine. I
>> read them in as double and then I print the numbers with 2f12.7 for
>> example and they are still fully there, not truncated.
>
>> I think the problem is in the printing to the output file.
>
>> I have x,y and they are numbers such as 100.912498
>> then I do a=strcompress(x)
>> openw,1,'fds'
>> printf,1,a
>
> Ah, I see. Well, STRING (which is what StrCompress has
> to call) is like PRINT: they have default formatting
> rules. In particular, they format to eight significant
> digits by default, unless they are told something different.
>
> But, first of all. Why in the world are you converting these
> to strings before you save them in the file!? That is a sure
> way to get in trouble, as you have discovered. If you *had*
> to do it, you could do it like this:
>
> a = StrCompress(String(x, format='F0.10'), /Remove_all)
>
> But a much better way would just be to write them into the
> file without converting them to strings:
>
> printf, x, Format='(F0.10)'
>
> Or, whatever format it is you think you want.
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Coyote's Guide to IDL Programming (www.dfanning.com)
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65776 is a reply to message #49718] |
Thu, 19 March 2009 12:21 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
plim.dreaming@gmail.com writes:
> Firstly say what is not the problem:
> the problem in not in comparing the numbers. I run one program which
> reads in the file and does some stuff to the numbers and then outputs
> to another file the selected array. Only after do I read in the
> selected array and perform separation comparisons. The problem
> already exists though in the output, since the numbers are truncated
> or rounded off there.
>
> And I really look at how I input the numbers and it seems fine. I
> read them in as double and then I print the numbers with 2f12.7 for
> example and they are still fully there, not truncated.
>
> I think the problem is in the printing to the output file.
>
> I have x,y and they are numbers such as 100.912498
> then I do a=strcompress(x)
> openw,1,'fds'
> printf,1,a
Ah, I see. Well, STRING (which is what StrCompress has
to call) is like PRINT: they have default formatting
rules. In particular, they format to eight significant
digits by default, unless they are told something different.
But, first of all. Why in the world are you converting these
to strings before you save them in the file!? That is a sure
way to get in trouble, as you have discovered. If you *had*
to do it, you could do it like this:
a = StrCompress(String(x, format='F0.10'), /Remove_all)
But a much better way would just be to write them into the
file without converting them to strings:
printf, x, Format='(F0.10)'
Or, whatever format it is you think you want.
Cheers,
David
--
David Fanning, Ph.D.
Coyote's Guide to IDL Programming (www.dfanning.com)
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65777 is a reply to message #49718] |
Thu, 19 March 2009 12:02 |
plim.dreaming
Messages: 22
Registered: February 2008
|
Junior Member |
|
|
Thank you for all the useful comments.
My problem, you'll be excited to know, persists.
Allow me say more:
Firstly say what is not the problem:
the problem in not in comparing the numbers. I run one program which
reads in the file and does some stuff to the numbers and then outputs
to another file the selected array. Only after do I read in the
selected array and perform separation comparisons. The problem
already exists though in the output, since the numbers are truncated
or rounded off there.
And I really look at how I input the numbers and it seems fine. I
read them in as double and then I print the numbers with 2f12.7 for
example and they are still fully there, not truncated.
I think the problem is in the printing to the output file.
I have x,y and they are numbers such as 100.912498
then I do a=strcompress(x)
openw,1,'fds'
printf,1,a
so, i am confused because:
print,x(83220),format='(2f12.7)'
100.9126890 which is the full value, the one i want to be printed to
the output
but
a = string(x(83220))
print,a
100.91269 and here it is a string so it doesn't help to do
format='....'
And finally, you guys may be seeing it clearly and trying to explain
this to me so thanks for the patience.
P
On Mar 19, 5:55 am, "Jean H." <jghas...@DELTHIS.ucalgary.ANDTHIS.ca>
wrote:
> plim.dream...@gmail.com wrote:
>> Later in the program I calculate the separation between points (x1,y1)
>> (x2,y2)
>> And for some of those points the program says that the pairs are the
>> same. But they are only the same if they are rounded off, the
>> difference often only shows up in the last 2 decimal places.
>> ya, i read that link, most of it at least.
>> One other thing is: lets say the print out is the issue; a case like
>> you pointed out above, then why is it that if I do:
>> b=string(num)
>> print,b will give me the rounded off number?
>
> as David pointed it out, be careful on how you compare your numbers...
> a = 0.1234567890123456d
> b = 0.1234567890123456d
> print, a eq b
> 1
>
> b = 0.1234567890123456d + 0.1 - 0.1
> print, a eq b
> 0
>
> Also, I suggest you to print your number value a few times (after
> reading, after allocating to pixmag, before comparing it), with the
> proper format ...
> IDL> c = 100.489418d
> IDL> print, c
> 100.48942
> IDL> print, c, format = '(F15.10)'
> 100.4894180000
>
> Jean
|
|
|
|
| Re: Double precision [message #65789 is a reply to message #49718] |
Thu, 19 March 2009 05:55 |
Jean H.
Messages: 472
Registered: July 2006
|
Senior Member |
|
|
plim.dreaming@gmail.com wrote:
> Later in the program I calculate the separation between points (x1,y1)
> (x2,y2)
> And for some of those points the program says that the pairs are the
> same. But they are only the same if they are rounded off, the
> difference often only shows up in the last 2 decimal places.
> ya, i read that link, most of it at least.
> One other thing is: lets say the print out is the issue; a case like
> you pointed out above, then why is it that if I do:
> b=string(num)
> print,b will give me the rounded off number?
as David pointed it out, be careful on how you compare your numbers...
a = 0.1234567890123456d
b = 0.1234567890123456d
print, a eq b
1
b = 0.1234567890123456d + 0.1 - 0.1
print, a eq b
0
Also, I suggest you to print your number value a few times (after
reading, after allocating to pixmag, before comparing it), with the
proper format ...
IDL> c = 100.489418d
IDL> print, c
100.48942
IDL> print, c, format = '(F15.10)'
100.4894180000
Jean
|
|
|
|
| Re: Double precision [message #65790 is a reply to message #49718] |
Thu, 19 March 2009 05:34 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
plim.dreaming@gmail.com writes:
> I read in those numbers with DOUBLE
> i remove some pairs which don't interest me and the rest I print them
> to an output file and they turn up as
> 100.48942 10.512547
> 100.48942 10.512558
Well, I'd like to see that command where you print
them to an output file. Specifically, I would
like to see what format statement you are using.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65791 is a reply to message #49718] |
Thu, 19 March 2009 03:51 |
Allan Whiteford
Messages: 117
Registered: June 2006
|
Senior Member |
|
|
plim.dreaming@gmail.com wrote:
> Well, I can't make sense of it.
> I read the article and if my case is in there I can't find it.
> Seems simple my problem.
> I am reading in a bunch of numbers, the input file has those numbers
> as
> 100.489418 10.512547
> 100.489718 10.512558
> and so on
> I read in those numbers with DOUBLE
> i remove some pairs which don't interest me and the rest I print them
> to an output file and they turn up as
> 100.48942 10.512547
> 100.48942 10.512558
>
> i just want the precision to be the same in the output as in the
> input. same number. that's all i ask. I notice that if I read in
> those numbers as FLOAT instead of DOUBLE then it was even worse. So
> it is true I am assuming that the DOUBLE is truncating/rounding-off/
> whatever.
>
> thanks for the help thus far.
> P
>
Plim,
The numbers you give should be 'good enough' even in single precision,
given the following:
pres.txt:
100.489418 10.512547
100.489718 10.512558
and IDL commands of:
openr,unit,'pres.txt',/get_lun
readf,unit,a,b
readf,unit,c,d
free_lun,unit
print,a,b,format='(2F9.5)'
print,c,d,format='(2F9.5)'
I get:
100.48942 10.51255
100.48972 10.51256
which looks fine to me. You are working at the seventh significant
figure which is where you'd expect to move to having to use doubles but
my implementation of IDL doesn't have a problem with these particular
numbers - this issue shouldn't really be implementation dependent. Can
you try the above and see what you get?
Maybe the algorithm where you're removing some of the pairs isn't doing
exactly what you think it is?
As also pointed out by Chris, you need to be careful that you are
actually reading in the numbers as doubles rather than reading them as
floats then converting to doubles. Specifically:
readf,unit,a
a=double(a)
is different from:
a=0d0
readf,unit,a
(the second example is correct). This issue is covered in David's
article in the section headed "[Question:] But, why doesn't
x2=Double(443496.984) produce the correct result?" although he doesn't
phrase the question specifically in terms of reading from a file.
However, I don't think it's a precision thing causing your problem - try
to narrow the problem down by printing back the numbers as soon as
you've read them so see if it really is the reading step going wrong.
Thanks,
Allan
> On Mar 18, 10:31 pm, David Fanning <n...@dfanning.com> wrote:
>
>> plim.dream...@gmail.com writes:
>>
>>> Later in the program I calculate the separation between points (x1,y1)
>>> (x2,y2)
>>> And for some of those points the program says that the pairs are the
>>> same. But they are only the same if they are rounded off, the
>>> difference often only shows up in the last 2 decimal places.
>>> ya, i read that link, most of it at least.
>>
>> I hope you aren't comparing floats with the EQ operator.
>> Maybe you should read that article again. All the way
>> to the end this time. :-)
>>
>> You might try Floats_Equal. You will get better results,
>> probably:
>>
>> http://www.dfanning.com/program/floats_equal.pro
>>
>>
>>> One other thing is: lets say the print out is the issue; a case like
>>> you pointed out above, then why is it that if I do:
>>> b=3Dstring(num)
>>> print,b will give me the rounded off number?
>>
>> It isn't rounding off the number. The number is the
>> number. It is printing the number in 8 significant
>> figures, which is the default format. Give it another
>> format and it will do something else.
>>
>> Cheers,
>>
>> David
>> --
>> David Fanning, Ph.D.
>> Fanning Software Consulting, Inc.
>> Coyote's Guide to IDL Programming:http://www.dfanning.com/
>> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
>
>
|
|
|
|
| Re: Double precision [message #65792 is a reply to message #49718] |
Thu, 19 March 2009 01:18 |
Chris[6]
Messages: 84
Registered: July 2008
|
Member |
|
|
On Mar 18, 9:36 pm, plim.dream...@gmail.com wrote:
> Well, I can't make sense of it.
> I read the article and if my case is in there I can't find it.
> Seems simple my problem.
> I am reading in a bunch of numbers, the input file has those numbers
> as
> 100.489418 10.512547
> 100.489718 10.512558
> and so on
> I read in those numbers with DOUBLE
> i remove some pairs which don't interest me and the rest I print them
> to an output file and they turn up as
> 100.48942 10.512547
> 100.48942 10.512558
>
> i just want the precision to be the same in the output as in the
> input. same number. that's all i ask. I notice that if I read in
> those numbers as FLOAT instead of DOUBLE then it was even worse. So
> it is true I am assuming that the DOUBLE is truncating/rounding-off/
> whatever.
>
> thanks for the help thus far.
> P
>
> On Mar 18, 10:31 pm, David Fanning <n...@dfanning.com> wrote:
>
>> plim.dream...@gmail.com writes:
>>> Later in the program I calculate the separation between points (x1,y1)
>>> (x2,y2)
>>> And for some of those points the program says that the pairs are the
>>> same. But they are only the same if they are rounded off, the
>>> difference often only shows up in the last 2 decimal places.
>>> ya, i read that link, most of it at least.
>
>> I hope you aren't comparing floats with the EQ operator.
>> Maybe you should read that article again. All the way
>> to the end this time. :-)
>
>> You might try Floats_Equal. You will get better results,
>> probably:
>
>> http://www.dfanning.com/program/floats_equal.pro
>
>>> One other thing is: lets say the print out is the issue; a case like
>>> you pointed out above, then why is it that if I do:
>>> b=3Dstring(num)
>>> print,b will give me the rounded off number?
>
>> It isn't rounding off the number. The number is the
>> number. It is printing the number in 8 significant
>> figures, which is the default format. Give it another
>> format and it will do something else.
>
>> Cheers,
>
>> David
>> --
>> David Fanning, Ph.D.
>> Fanning Software Consulting, Inc.
>> Coyote's Guide to IDL Programming:http://www.dfanning.com/
>> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
>
>
maybe include the line of code where you read in the file? I could see
accidentally converting the number to a float on input before
promoting it to a double, which would cause the roundoff.
chris
|
|
|
|
| Re: Double precision [message #65793 is a reply to message #49718] |
Thu, 19 March 2009 00:36 |
plim.dreaming
Messages: 22
Registered: February 2008
|
Junior Member |
|
|
Well, I can't make sense of it.
I read the article and if my case is in there I can't find it.
Seems simple my problem.
I am reading in a bunch of numbers, the input file has those numbers
as
100.489418 10.512547
100.489718 10.512558
and so on
I read in those numbers with DOUBLE
i remove some pairs which don't interest me and the rest I print them
to an output file and they turn up as
100.48942 10.512547
100.48942 10.512558
i just want the precision to be the same in the output as in the
input. same number. that's all i ask. I notice that if I read in
those numbers as FLOAT instead of DOUBLE then it was even worse. So
it is true I am assuming that the DOUBLE is truncating/rounding-off/
whatever.
thanks for the help thus far.
P
On Mar 18, 10:31 pm, David Fanning <n...@dfanning.com> wrote:
> plim.dream...@gmail.com writes:
>> Later in the program I calculate the separation between points (x1,y1)
>> (x2,y2)
>> And for some of those points the program says that the pairs are the
>> same. But they are only the same if they are rounded off, the
>> difference often only shows up in the last 2 decimal places.
>> ya, i read that link, most of it at least.
>
> I hope you aren't comparing floats with the EQ operator.
> Maybe you should read that article again. All the way
> to the end this time. :-)
>
> You might try Floats_Equal. You will get better results,
> probably:
>
> http://www.dfanning.com/program/floats_equal.pro
>
>> One other thing is: lets say the print out is the issue; a case like
>> you pointed out above, then why is it that if I do:
>> b=3Dstring(num)
>> print,b will give me the rounded off number?
>
> It isn't rounding off the number. The number is the
> number. It is printing the number in 8 significant
> figures, which is the default format. Give it another
> format and it will do something else.
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65794 is a reply to message #49718] |
Wed, 18 March 2009 22:31 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
plim.dreaming@gmail.com writes:
> Later in the program I calculate the separation between points (x1,y1)
> (x2,y2)
> And for some of those points the program says that the pairs are the
> same. But they are only the same if they are rounded off, the
> difference often only shows up in the last 2 decimal places.
> ya, i read that link, most of it at least.
I hope you aren't comparing floats with the EQ operator.
Maybe you should read that article again. All the way
to the end this time. :-)
You might try Floats_Equal. You will get better results,
probably:
http://www.dfanning.com/program/floats_equal.pro
> One other thing is: lets say the print out is the issue; a case like
> you pointed out above, then why is it that if I do:
> b=3Dstring(num)
> print,b will give me the rounded off number?
It isn't rounding off the number. The number is the
number. It is printing the number in 8 significant
figures, which is the default format. Give it another
format and it will do something else.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65795 is a reply to message #49718] |
Wed, 18 March 2009 22:16 |
plim.dreaming
Messages: 22
Registered: February 2008
|
Junior Member |
|
|
Later in the program I calculate the separation between points (x1,y1)
(x2,y2)
And for some of those points the program says that the pairs are the
same. But they are only the same if they are rounded off, the
difference often only shows up in the last 2 decimal places.
ya, i read that link, most of it at least.
One other thing is: lets say the print out is the issue; a case like
you pointed out above, then why is it that if I do:
b=string(num)
print,b will give me the rounded off number?
On Mar 18, 9:32 pm, David Fanning <n...@dfanning.com> wrote:
> plim.dream...@gmail.com writes:
>> Ok, this is probably dumb but I need help with it.
>> So i'm reading in some data which was in string format, several
>> columns of data, and the data in one of the columns is getting
>> truncated, rounded off. I want more significant figures.
>> I have:
>> pixmag = reform(double(dbwork[1,*]))
>> dbwork is the original data, it has outputs such as 100.261787
>> pixmag, however, rounds that off to 100.26179 which is causing errors.
>> So how do I bump this up to 14 decimal places? Where do I put the
>> "D"??
>
> What evidence do you have that the numbers are being
> "rounded off"?
>
> IDL> str = '6.1234567890'
> IDL> num = double(str)
> IDL> print, num, format='(F12.10)'
> 6.1234567890
>
> However,
>
> IDL> print, num
> 6.1234568
>
> Is the default format of the PRINT command your problem?
>
> This is always a useful article to read:
>
> http://www.dfanning.com/math_tips/sky_is_falling.html
>
> Cheers,
>
> David
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Double precision [message #65796 is a reply to message #49718] |
Wed, 18 March 2009 21:32 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
plim.dreaming@gmail.com writes:
> Ok, this is probably dumb but I need help with it.
> So i'm reading in some data which was in string format, several
> columns of data, and the data in one of the columns is getting
> truncated, rounded off. I want more significant figures.
> I have:
> pixmag = reform(double(dbwork[1,*]))
> dbwork is the original data, it has outputs such as 100.261787
> pixmag, however, rounds that off to 100.26179 which is causing errors.
> So how do I bump this up to 14 decimal places? Where do I put the
> "D"??
What evidence do you have that the numbers are being
"rounded off"?
IDL> str = '6.1234567890'
IDL> num = double(str)
IDL> print, num, format='(F12.10)'
6.1234567890
However,
IDL> print, num
6.1234568
Is the default format of the PRINT command your problem?
This is always a useful article to read:
http://www.dfanning.com/math_tips/sky_is_falling.html
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
