| Why does 0.8 = 0.80000001 ? [message #31903] |
Mon, 26 August 2002 05:34 |
Benjamin Panter
Messages: 6
Registered: August 2002
|
Junior Member |
|
|
Hi Chaps,
I've just found something quite interesting and I wonder if anyone had
any clues as to why it's happening
I have a rather huge program which spits out a set of numbers in an
array called str1, declared as a double. When I try to set one of the
elements to 0.8, it actually sets to 0.80000001... this is below the
accuracy that I worry about, but I'm just rather interested as to why it
happens. Some output is below
Many Thanks!
Ben
-----Using IDLDE 5.4 on a Linux machine -----
IDL> print, transpose(str1)
9.3810789e-07
3.9122462e-07
5.3666963e-07
1.0000000e-07
0.079999998
1.0000000e-07
2.4324324
1.0000000e-07
0.62996066
0.053492580
0.013443811
0.57078505
1.5306662
0.019947363
3.0532773
2.0000000
0.039999999
IDL> str1[6]=0.8
IDL> print, transpose(str1)
9.3810789e-07
3.9122462e-07
5.3666963e-07
1.0000000e-07
0.079999998
1.0000000e-07
0.80000001
1.0000000e-07
0.62996066
0.053492580
0.013443811
0.57078505
1.5306662
0.019947363
3.0532773
2.0000000
0.039999999
IDL> print, str1[6]
0.80000001
IDL> help, str1
STR1 DOUBLE = Array[17]
--
Ben Panter, Royal Observatory, Edinburgh
benpanterREMOVE@bigfoot.com
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
