comp.lang.idl-pvwave archive: archive » Re: DOUBLE precision no precise??

Home » Public Forums » archive » Re: DOUBLE precision no precise??

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: DOUBLE precision no precise?? [message #29619 is a reply to message #29617]

Tue, 05 March 2002 05:01

Vincent Schut
Messages: 8
Registered: February 2002

Junior Member

David Williams wrote:
> I've always had heaps of help from the inhabitants of this newsgroup --
> for which I am eternally grateful -- despite my often stupid questions.
> So, when a mate of mine came across this `quirk' yesterday, and I wasn't
> sure how to help him out, I thought I'd ask this group.
>
> He has an array of numbers that he wants to apply a user-defined
> function to, but we're both a little disturbed by the fact that if you
> do the calculations with a pocket calculator, you get different numbers
> than if you perform the same calculation in IDL.
>
> To try and find where the problem is, we tried the following lines...
>
> IDL> a = DOUBLE(42766.080001)
> IDL> print,a,FORMAT='(F24.17)'
>
> 42766.07812500000000000
>
> As you see, the number we get out isn't the same as the number we
> entered. I'm guessing it's to do with the way IDL stores numbers in
> memory, but my understanding of low-level computational processes isn't
> great.
>
> Can anybody help me understand what's going on, and/or if there's a way
> around? I'd really appreciate whatever help is on offer, so thanks in
> advance.
>
>
> Dave
>
> ------------------------------------------------------------ ---------
> David R. Williams, | BT7 1NN, Northern Ireland.
> Astrophysics & Planetary Science, | d.williams@qub.ac.uk
> Queen's University, Belfast, | http://star.pst.qub.ac.uk/~drw/
> ============================================================ =========

I'm no expert on math precision, but I think that this is a known
'problem', caused by the different possible ways to store floating point
precision data. (For example, different C++ compilers also give
different values for a double precision float constant of pi, nice eh?
:-)) It might help to read the idl help section called 'accuracy &
floating point operations', as a start. Maybe others in the group can
give you a more specific answer, though.

cheers,
Vincent.

Report message to a moderator

[Message index]

		Re: DOUBLE precision no precise?? By: Pete[1] on Tue, 05 March 2002 13:11
		Re: DOUBLE precision no precise?? By: dave_r_williams on Tue, 05 March 2002 09:20
		Re: DOUBLE precision no precise?? By: James Kuyper on Tue, 05 March 2002 08:29
		Re: DOUBLE precision no precise?? By: William Clodius on Tue, 05 March 2002 08:11
		Re: DOUBLE precision no precise?? By: Robert Stockwell on Tue, 05 March 2002 07:32
		Re: DOUBLE precision no precise?? By: Martin Downing on Tue, 05 March 2002 06:30
		Re: DOUBLE precision no precise?? By: Vincent Schut on Tue, 05 March 2002 07:07
		Re: DOUBLE precision no precise?? By: R.Bauer on Tue, 05 March 2002 06:07
		Re: DOUBLE precision no precise?? By: Vincent Schut on Tue, 05 March 2002 05:01
		Re: DOUBLE precision no precise?? By: James Tappin on Tue, 05 March 2002 04:14

Previous Topic:	bad magic number for shared library when using CALL_EXTERNAL
Next Topic:	CALL_EXTERNAL using C/Fortran shared library on HPUX11

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 13:58:42 PDT 2025

Total time taken to generate the page: 0.00451 seconds