comp.lang.idl-pvwave archive: archive » Re: DOUBLE precision no precise??

Home » Public Forums » archive » Re: DOUBLE precision no precise??

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: DOUBLE precision no precise?? [message #29617 is a reply to message #29613]

Tue, 05 March 2002 06:07

R.Bauer
Messages: 1424
Registered: November 1998

Senior Member

James Tappin wrote:
>
> David Williams wrote:
>
>>
>> I've always had heaps of help from the inhabitants of this newsgroup --
>> for which I am eternally grateful -- despite my often stupid questions.
>> So, when a mate of mine came across this `quirk' yesterday, and I wasn't
>> sure how to help him out, I thought I'd ask this group.
>>
>> He has an array of numbers that he wants to apply a user-defined
>> function to, but we're both a little disturbed by the fact that if you
>> do the calculations with a pocket calculator, you get different numbers
>> than if you perform the same calculation in IDL.
>>
>> To try and find where the problem is, we tried the following lines...
>>
>> IDL> a = DOUBLE(42766.080001)
>> IDL> print,a,FORMAT='(F24.17)'
>>
>> 42766.07812500000000000
>>
>> As you see, the number we get out isn't the same as the number we
>> entered. I'm guessing it's to do with the way IDL stores numbers in
>> memory, but my understanding of low-level computational processes isn't
>> great.
>>
>> Can anybody help me understand what's going on, and/or if there's a way
>> around? I'd really appreciate whatever help is on offer, so thanks in
>> advance.
>
> The problem is that 42766.080001 is a single precision constant, so what's
> happening is that you are storing the single-precision approximation to
> 42766.080001 in some scratch location, then converting that to double.
>
> What you actually want is:
> a=42766.080001D0
>
> --
> +------------------------+-------------------------------+-- -------+
> | James Tappin | School of Physics & Astronomy | O__ |
> | sjt@star.sr.bham.ac.uk | University of Birmingham | -- \/` |
> | Ph: 0121-414-6462. Fax: 0121-414-3722 | |
> +--------------------------------------------------------+-- -------+

My vote to this answer!

Reimar

--
Reimar Bauer

Institut fuer Stratosphaerische Chemie (ICG-I)
Forschungszentrum Juelich
email: R.Bauer@fz-juelich.de
------------------------------------------------------------ -------
a IDL library at ForschungsZentrum Juelich
http://www.fz-juelich.de/icg/icg1/idl_icglib/idl_lib_intro.h tml
============================================================ =======

Report message to a moderator

[Message index]

		Re: DOUBLE precision no precise?? By: Pete[1] on Tue, 05 March 2002 13:11
		Re: DOUBLE precision no precise?? By: dave_r_williams on Tue, 05 March 2002 09:20
		Re: DOUBLE precision no precise?? By: James Kuyper on Tue, 05 March 2002 08:29
		Re: DOUBLE precision no precise?? By: William Clodius on Tue, 05 March 2002 08:11
		Re: DOUBLE precision no precise?? By: Robert Stockwell on Tue, 05 March 2002 07:32
		Re: DOUBLE precision no precise?? By: Martin Downing on Tue, 05 March 2002 06:30
		Re: DOUBLE precision no precise?? By: Vincent Schut on Tue, 05 March 2002 07:07
		Re: DOUBLE precision no precise?? By: R.Bauer on Tue, 05 March 2002 06:07
		Re: DOUBLE precision no precise?? By: Vincent Schut on Tue, 05 March 2002 05:01
		Re: DOUBLE precision no precise?? By: James Tappin on Tue, 05 March 2002 04:14

Previous Topic:	bad magic number for shared library when using CALL_EXTERNAL
Next Topic:	CALL_EXTERNAL using C/Fortran shared library on HPUX11

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 20:07:15 PDT 2025

Total time taken to generate the page: 0.00396 seconds