comp.lang.idl-pvwave archive: archive » Re: WRONG RESULTS WITH IDL

Home » Public Forums » archive » Re: WRONG RESULTS WITH IDL

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: WRONG RESULTS WITH IDL [message #11708]

Tue, 05 May 1998 00:00

Richard G. French
Messages: 65
Registered: June 1997

Member

>> sin(12345678901.12345678901) = -03727 9004 9960 8007
>>
>> where IDL get:
>>
>> IDL> x = double(12345678901.12345678901)

************^^^^^^^ This line converts a number with single
precision to a double precision value, but it does not keep
all of the digits listed. You need to make the number double
precision explicitly by adding a D0 at the end:
x=12345678901.12345678901D0

If you print x instead of sin(x), you will see that you have not
retained all of the listed digits of precision

Report message to a moderator

[Message index]

		Re: WRONG RESULTS WITH IDL By: Richard G. French on Tue, 05 May 1998 00:00
		Re: WRONG RESULTS WITH IDL By: Thomas A. McGlynn on Tue, 05 May 1998 00:00
		Re: WRONG RESULTS WITH IDL By: Andy Loughe on Tue, 05 May 1998 00:00
		Re: WRONG RESULTS WITH IDL By: Caesar E. Ordonez on Tue, 05 May 1998 00:00

Previous Topic:	[Q] if-else statement
Next Topic:	5.0 XMANAGER problem

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Nov 28 16:20:18 PST 2025

Total time taken to generate the page: 0.02222 seconds