comp.lang.idl-pvwave archive: archive » Re: Problem with FLOAT in CW

Home » Public Forums » archive » Re: Problem with FLOAT in CW_FORM

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Problem with FLOAT in CW_FORM [message #16018 is a reply to message #16010]

Tue, 22 June 1999 00:00

Axel vom Endt
Messages: 9
Registered: March 1999

Junior Member

Steffen Keitel wrote:
>
> Hello!
> Building a widget based fit program (IDL 5.1 / Linux) I encountered the
> following problem: I use the CW_FORM widget to enter my initial values
> of the fit parameter. If I type e.g. "0.350" in a FLOAT field, IDL will
> work with "0.3499999940" or something like that. How can I teach IDL to
> use the _exact_ value?
>
> Thanks in advance, Steffen

Hi steffen,

that's a more general problem: How do you tell a computer to work with
0.350 ?

short answer: .35 cannot be represented by a finite binary fraction.

Try the same in c:

#include <stdio.h>
int main(){
float x = 0.35;
printf("x = %12.10f\n",x);
return 0;
}

For a more detailed answer, refer to any textbook on numerical methods.

Axel

Report message to a moderator

[Message index]

		Re: Problem with FLOAT in CW_FORM By: Martin Schultz on Tue, 22 June 1999 00:00
		Re: Problem with FLOAT in CW_FORM By: Axel vom Endt on Tue, 22 June 1999 00:00

Previous Topic:	stereo display with IDL
Next Topic:	Re: For which versions of RH Linux does IDL 4.01b work?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 16:24:41 PDT 2025

Total time taken to generate the page: 0.83151 seconds