comp.lang.idl-pvwave archive: archive » Re: WRONG RESULTS WITH IDL

Home » Public Forums » archive » Re: WRONG RESULTS WITH IDL

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: WRONG RESULTS WITH IDL [message #11712 is a reply to message #11708]

Tue, 05 May 1998 00:00

Andy Loughe
Messages: 174
Registered: November 1995

Senior Member

Maybe this helps to explain the "sin()" difference...

print, 12345678901.12345678901D, format='(f30.15)'
12345678901.123456954956055

print, double(12345678901.12345678901), format='(f30.15)'
12345678848.000000000000000

Apparently when converting the real to double in the second case,
there is a great loss in precision since the original real number
culdn't properly store a number that large.

Caesar E. Ordonez wrote:
>
> I did the following:
>
> IDL> x = 12345678901.12345678901D
> IDL> print,sin(x)
> -0.37327885

--
Andrew F. Loughe |
afl@cdc.noaa.gov
University of Colorado, CIRES Box 449 |
http://cdc.noaa.gov/~afl
Boulder, CO 80309-0449 | phn:(303)492-0707
fax:(303)497-7013
------------------------------------------------------------ ---------------
"I do not feel obliged to believe that the same God who has endowed us
with
sense, reason, and intellect has intended us to forego their use."
-Galileo

Report message to a moderator

[Message index]

		Re: WRONG RESULTS WITH IDL By: Richard G. French on Tue, 05 May 1998 00:00
		Re: WRONG RESULTS WITH IDL By: Thomas A. McGlynn on Tue, 05 May 1998 00:00
		Re: WRONG RESULTS WITH IDL By: Andy Loughe on Tue, 05 May 1998 00:00
		Re: WRONG RESULTS WITH IDL By: Caesar E. Ordonez on Tue, 05 May 1998 00:00

Previous Topic:	[Q] if-else statement
Next Topic:	5.0 XMANAGER problem

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Apr 26 12:20:52 PDT 2026

Total time taken to generate the page: 1.03920 seconds