comp.lang.idl-pvwave archive: archive » Re: Floating-point accuracy problem

Home » Public Forums » archive » Re: Floating-point accuracy problem

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Floating-point accuracy problem [message #72108]

Mon, 09 August 2010 09:09

Michael Galloy
Messages: 1114
Registered: April 2006

Senior Member

On 8/9/10 9:30 AM, Ramy wrote:
> I have a problem related to float-point accuracy
> If I type in: 50d - 1d-9, I get 50.000000
>
> And here lies my problem, I'm doing a numerical simulation where such
> an arithmetic is common place, and as a result i get a lot or errors.
>
> I know for example, that if i simply type
> print, 50d - 1d-9, format = '(f.20.10)' , i'll get:
> 49.9999999990
>
> But how can I convince IDL to do it on its own during computations?
>
> Any suggestions will be massively welcomed!

You are confusing the computations with the printed output. IDL will do
double precision arithmetic if one of the arguments is a double. When
you print it out, you can choose the format you want to print it out in:

IDL> d = 50d - 1d-9
IDL> print, d
50.000000
IDL> print, d, format='(F20.10)'
49.9999999990

Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation

Report message to a moderator

[Message index]

Re: Floating-point accuracy problem

By: Michael Galloy on Mon, 09 August 2010 09:09

Previous Topic:	Floating-point accuracy problem
Next Topic:	Re: IDL_Container and garbage collection

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Mon Dec 01 09:35:39 PST 2025

Total time taken to generate the page: 1.28024 seconds