| Strange floating-point precision behavior [message #33926] |
Sat, 08 February 2003 15:33  |
lloyd
Messages: 16
Registered: February 2003
|
Junior Member |
|
|
I have written a routine that converts Earth-Centered Inertial
coordinates in x/y/z to geodetic latitude/longitude/altitude using the
WGS84 standard. I have one issue, however, that I believe is
affecting my calculations of altitude so that they are accurate only
to 1-meter resolution. I am defining the ECI coordinates as
double-precision:
IDL> boulder={x:-1283388.8693d0, $
y:-4713016.9053d0, $
z:4090191.0471d0} ;Boulder, CO, GPS station
and yet IDL seems to be storing the data incorrectly:
IDL> print,boulder,format='(3f20.10)'
-1283388.8692999999 -4713016.9052999998 4090191.0471000001
What am I doing wrong? I am fairly certain that this behavior is
responsible for my calculations yielding 1674.6658 m as the altitude
of the Boulder GPS station, and not 1674.7428 m (the actual altitude).
This is on IDL 5.6 for Mac OS X.
Thanks,
Tim Lloyd
Laboratory for Atmospheric & Space Physics
|
|
|
|
| Re: Strange floating-point precision behavior [message #33993 is a reply to message #33926] |
Tue, 11 February 2003 09:49  |
tim
Messages: 9
Registered: July 1994
|
Junior Member |
|
|
On Mon, 10 Feb 2003, James Kuyper wrote:
> Tim Lloyd wrote:
>>
>> I have written a routine that converts Earth-Centered Inertial
>> coordinates in x/y/z to geodetic latitude/longitude/altitude using the
>> WGS84 standard. I have one issue, however, that I believe is
>> affecting my calculations of altitude so that they are accurate only
>> to 1-meter resolution. I am defining the ECI coordinates as
>> double-precision:
>>
>> IDL> boulder={x:-1283388.8693d0, $
>> y:-4713016.9053d0, $
>> z:4090191.0471d0} ;Boulder, CO, GPS station
>
> Are you sure those are ECI coordinates? Interpreted as ECR
> (Earth-Centered Rotating) coordinates, they correspond pretty closely to
> Boulder CO. Interpreted as ECI coordinates, you'd need a fourth value,
> the precise time at which the conversion from ECI to ECR should occur.
> ECI and ECR coordinates match only once each day, so it would be quite a
> coincidence if those ECI coordinates happened to match the ECR
> coordinates for Boulder.
My bad, those are actually ECR. Guess I use too many TLA's to keep them
all straight.
>> and yet IDL seems to be storing the data incorrectly:
>>
>> IDL> print,boulder,format='(3f20.10)'
>> -1283388.8692999999 -4713016.9052999998 4090191.0471000001
> ...
>> What am I doing wrong? I am fairly certain that this behavior is
>> responsible for my calculations yielding 1674.6658 m as the altitude
>> of the Boulder GPS station, and not 1674.7428 m (the actual altitude).
>
> No, that isn't the cause of your problem. Floating point roundoff has
> introduced errors of only about 10^-10 meters into your calculations;
> that can't be the cause of a 0.123 meter error in the altitude. I have
> access to a C routine which performs this same conversion, and it
> produces the same result as your routine. Is it possible that it's not
> the routine that's at fault, but the data?
That's helpful, actually. Looks like I got a false positive on my
debugging. Time to dive back in...
Thanks,
Tim Lloyd, lloyd@lasp.colorado.edu
SNOE Mission Operations Lead Flight Controller
Laboratory for Atmospheric and Space Physics
"The eyes of the world now look into space, to the moon and to the
planets beyond, and we have vowed that we shall not see it governed
by a hostile flag of conquest, but by a banner of freedom and peace."
-- John F. Kennedy
|
|
|
|
| Re: Strange floating-point precision behavior [message #34010 is a reply to message #33926] |
Mon, 10 February 2003 19:57 |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
James Kuyper <kuyper@saicmodis.com> writes:
>
> Are you sure those are ECI coordinates? Interpreted as ECR
> (Earth-Centered Rotating) coordinates, they correspond pretty closely to
> Boulder CO. Interpreted as ECI coordinates, you'd need a fourth value,
> the precise time at which the conversion from ECI to ECR should occur.
> ECI and ECR coordinates match only once each day, so it would be quite a
> coincidence if those ECI coordinates happened to match the ECR
> coordinates for Boulder.
And, if 12 cm precision is really desired, then earth precession,
nutation, and polar wander are of concern, in which case, ECI and ECR
match each other... pretty much never! :-)
Craig
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
| Re: Strange floating-point precision behavior [message #34013 is a reply to message #33926] |
Mon, 10 February 2003 12:44 |
James Kuyper
Messages: 425
Registered: March 2000
|
Senior Member |
|
|
Tim Lloyd wrote:
>
> I have written a routine that converts Earth-Centered Inertial
> coordinates in x/y/z to geodetic latitude/longitude/altitude using the
> WGS84 standard. I have one issue, however, that I believe is
> affecting my calculations of altitude so that they are accurate only
> to 1-meter resolution. I am defining the ECI coordinates as
> double-precision:
>
> IDL> boulder={x:-1283388.8693d0, $
> y:-4713016.9053d0, $
> z:4090191.0471d0} ;Boulder, CO, GPS station
Are you sure those are ECI coordinates? Interpreted as ECR
(Earth-Centered Rotating) coordinates, they correspond pretty closely to
Boulder CO. Interpreted as ECI coordinates, you'd need a fourth value,
the precise time at which the conversion from ECI to ECR should occur.
ECI and ECR coordinates match only once each day, so it would be quite a
coincidence if those ECI coordinates happened to match the ECR
coordinates for Boulder.
> and yet IDL seems to be storing the data incorrectly:
>
> IDL> print,boulder,format='(3f20.10)'
> -1283388.8692999999 -4713016.9052999998 4090191.0471000001
...
> What am I doing wrong? I am fairly certain that this behavior is
> responsible for my calculations yielding 1674.6658 m as the altitude
> of the Boulder GPS station, and not 1674.7428 m (the actual altitude).
No, that isn't the cause of your problem. Floating point roundoff has
introduced errors of only about 10^-10 meters into your calculations;
that can't be the cause of a 0.123 meter error in the altitude. I have
access to a C routine which performs this same conversion, and it
produces the same result as your routine. Is it possible that it's not
the routine that's at fault, but the data?
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
