comp.lang.idl-pvwave archive: archive » Re: DOUBLE precision no precise??

Home » Public Forums » archive » Re: DOUBLE precision no precise??

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: DOUBLE precision no precise?? [message #29613 is a reply to message #29610]

Tue, 05 March 2002 06:30

Martin Downing
Messages: 136
Registered: September 1998

Senior Member

Hi David,

James is correct. You are attempting to assign a double constant
incorrectly, since the default for a floating point constant is a single
precision (FLOAT) number. Append with the letter "d" and all will be well.
You will find many recent discussions in the group on the consequences of
floating point arithmetic with inadequate precision, in the mean time I hope
the following helps you see where your precision was lost!

cheers

Martin
=======================

IDL> a = 42766.080001

IDL> help , a

A FLOAT = 42766.1

IDL> print, a, format = '(f30.20)'

42766.07812500000000000000

IDL> a = double(a)

IDL> print, a, format = '(f30.20)'

42766.07812500000000000000

IDL> a = 42766.080001d

IDL> help , a

A DOUBLE = 42766.080

IDL> print, a, format = '(f30.20)'

42766.08000100000200000000

=====================
--
----------------------------------------
Martin Downing,
Clinical Research Physicist,
Grampian Orthopaedic RSA Research Centre,
Woodend Hospital, Aberdeen, AB15 6LS.
Tel. 01224 556055 / 07903901612
Fax. 01224 556662

m.downing@abdn.ac.uk

"David Williams" <d.williams@qub.ac.uk> wrote in message
news:3C84B20D.57963F41@qub.ac.uk...
>
> I've always had heaps of help from the inhabitants of this newsgroup --
> for which I am eternally grateful -- despite my often stupid questions.
> So, when a mate of mine came across this `quirk' yesterday, and I wasn't
> sure how to help him out, I thought I'd ask this group.
>
> He has an array of numbers that he wants to apply a user-defined
> function to, but we're both a little disturbed by the fact that if you
> do the calculations with a pocket calculator, you get different numbers
> than if you perform the same calculation in IDL.
>
> To try and find where the problem is, we tried the following lines...
>
> IDL> a = DOUBLE(42766.080001)
> IDL> print,a,FORMAT='(F24.17)'
>
> 42766.07812500000000000
>
> As you see, the number we get out isn't the same as the number we
> entered. I'm guessing it's to do with the way IDL stores numbers in
> memory, but my understanding of low-level computational processes isn't
> great.
>
> Can anybody help me understand what's going on, and/or if there's a way
> around? I'd really appreciate whatever help is on offer, so thanks in
> advance.
>
>
> Dave
>
> ------------------------------------------------------------ ---------
> David R. Williams, | BT7 1NN, Northern Ireland.
> Astrophysics & Planetary Science, | d.williams@qub.ac.uk
> Queen's University, Belfast, | http://star.pst.qub.ac.uk/~drw/
> ============================================================ =========

Report message to a moderator

[Message index]

		Re: DOUBLE precision no precise?? By: Pete[1] on Tue, 05 March 2002 13:11
		Re: DOUBLE precision no precise?? By: dave_r_williams on Tue, 05 March 2002 09:20
		Re: DOUBLE precision no precise?? By: James Kuyper on Tue, 05 March 2002 08:29
		Re: DOUBLE precision no precise?? By: William Clodius on Tue, 05 March 2002 08:11
		Re: DOUBLE precision no precise?? By: Robert Stockwell on Tue, 05 March 2002 07:32
		Re: DOUBLE precision no precise?? By: Martin Downing on Tue, 05 March 2002 06:30
		Re: DOUBLE precision no precise?? By: Vincent Schut on Tue, 05 March 2002 07:07
		Re: DOUBLE precision no precise?? By: R.Bauer on Tue, 05 March 2002 06:07
		Re: DOUBLE precision no precise?? By: Vincent Schut on Tue, 05 March 2002 05:01
		Re: DOUBLE precision no precise?? By: James Tappin on Tue, 05 March 2002 04:14

Previous Topic:	bad magic number for shared library when using CALL_EXTERNAL
Next Topic:	CALL_EXTERNAL using C/Fortran shared library on HPUX11

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 06:42:38 PDT 2025

Total time taken to generate the page: 0.39968 seconds