Home »
Public Forums »
archive »
Problems with double precision in IDL
| Problems with double precision in IDL [message #1251] |
Wed, 16 June 1993 09:12 |
isaacman
Messages: 20
Registered: June 1992
|
Junior Member |
|
|
We have a potentially VERY serious problem with the COBE data analysis
because of the way IDL seems to be (mis?)handling double precision
numbers. Here is an example of how IDL treats floating point numbers
when converting them to double precision. The operations were performed
on a DECstation.
z=.32
print,f2,double(z) ; If a variable is declared DOUBLE this is what happens.
0.319999992847
print,f2,.32d ; If the "d" notation is used instead it's accurate.
0.320000000000
print,double(z)-.32D
-7.15255737e-09
print,[double(z)-.32D]/.32D
-2.23517418e-08
An illustration of the difference between FORTRAN and IDL follows. Two
parallel programs were written, one in each language, which do the following:
1. Read variables declared as single precision from a file (typically
2-5 numbers).
2. Convert the variables to double precision
3. Perform various computations on the double precision numbers (the
computations involved only a few addition/subtraction operations and some
divison).
4. Convert the answers to single precision and write to a file.
The difference in the answers shows that IDL only gives accuracy to four
significant digits (or less) under these circumstances. That's really not
acceptable for many applications.
COBE Variable Name FORTRAN IDL
.coad_spec_data.XCAL_SIGMA 8.096988e-02 8.096990e-02
.coad_spec_data.ICAL_SIGMA 7.917210e-05 7.912043e-05
.coad_spec_data.SKYHORN_SIGMA 4.511252e-04 4.511047e-04
.coad_spec_data.REFHORN_SIGMA 6.387254e-05 6.390238e-05
.coad_spec_data.DIHEDRAL_SIGMA 9.627582e-02 9.627583e-02
.coad_spec_data.COLLIMATOR_SIGMA 1.346936e-02 1.346943e-02
.coad_spec_data.BOL_ASSEM_SIGMA 4.545850e-03 4.545857e-03
.coad_spec_data.BOL_ASSEM_SIGMA 4.577306e-03 4.577292e-03
.coad_spec_data.BOL_ASSEM_SIGMA 4.641583e-03 4.641529e-03
.coad_spec_data.BOL_ASSEM_SIGMA 6.197933e-03 6.197947e-03
.spec_data.RESP_SIGMA 3.442960e-03 3.442946e-03
.spec_data.TC_SIGMA 1.057141e-06 1.057107e-06
.spec_data.PC_SIGMA 8.645144e-05 8.645145e-05
.spec_data.QRAD_SIGMA 8.201977e-11 8.201978e-11
.spec_data.IR_POWER_SIGMA 4.887270e-14 4.888122e-14
========================================
This is potentially very serious for us. Has anyone out there
encountered the problem or figured out a solution?
Thanks,
Rich Isaacman
COBE Project
NASA/Goddard Space Flight Center
|
|
|
|
Current Time: Wed Oct 08 19:12:43 PDT 2025
Total time taken to generate the page: 0.00595 seconds
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
