comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: rounding errors

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: rounding errors [message #24956 is a reply to message #24907]

Wed, 02 May 2001 08:07

James Kuyper
Messages: 425
Registered: March 2000

Senior Member

Paul van Delst wrote:
>
> Randall Skelton wrote:
>>
>> On Fri, 27 Apr 2001, Liam E. Gumley wrote:
>>
>>> This is a subtle but important point. DOUBLE() is a type conversion
>>> function, and
>>>
>>> a = double(2.348339)
>>>
>>> shows a FLOAT argument being converted to a DOUBLE. The safest way to
>>> 'cast' a double variable is
>>>
>>> a = 2.348339d
>> [snip]
>>
>> Wow... I am glad that I have now learned that particular 'IDL feature'
>> early on in my PhD. Just yesterday, I convinced the department that we
>> really need a few good IDL programming books as the current
>> 'learning-by-fire' approach could have some unfortunate consequences ;)
>
> This "feature" has absolutely *nothing* to do with IDL. The same thing occurs in other
> languages, e.g. Fortran, C, etc. Floating point numbers, in general, cannot be represented
> exactly and you have to keep that in mind when writing code

I think you're misunderstanding the "feature" of IDL that surprised me
as much as it surprised Randall and Liam. This has everything to do with
IDL, and nothing to do with expecting exact representation of a
finite-length decimal fraction. By default, in C 2.348339 represents a
double precision number, not a single precision one, and I'd never
realized that the IDL convention was different. That probably explains a
number of the wierd results I've seen.

Report message to a moderator

[Message index]

		Re: rounding errors By: thompson on Fri, 27 April 2001 11:34
		Re: rounding errors By: Liam E. Gumley on Fri, 27 April 2001 10:01
		Re: rounding errors By: Karl Schultz on Fri, 27 April 2001 09:07
		Re: rounding errors By: Paul van Delst on Fri, 27 April 2001 08:47
		Re: rounding errors By: Craig Markwardt on Fri, 27 April 2001 08:52
		Re: rounding errors By: John-David T. Smith on Fri, 27 April 2001 08:39
		Re: rounding errors By: thompson on Fri, 27 April 2001 08:23
		Re: rounding errors By: Dominic R. Scales on Fri, 27 April 2001 07:41
		Re: rounding errors By: Randall Skelton on Fri, 27 April 2001 07:51
		Re: rounding errors By: Liam E. Gumley on Fri, 27 April 2001 07:21
		Re: rounding errors By: Craig Markwardt on Fri, 27 April 2001 06:44
		Re: rounding errors By: Randall Skelton on Fri, 27 April 2001 06:17
		Re: rounding errors By: Dominic R. Scales on Fri, 27 April 2001 04:38
		Re: rounding errors By: Alex Schuster on Fri, 27 April 2001 03:52
		Re: rounding errors By: James Kuyper on Wed, 02 May 2001 08:06
		Re: rounding errors By: James Kuyper on Wed, 02 May 2001 08:07

Previous Topic:	Re: AAARGH-2 !!! CALL_EXTERNAL and IDL5.2 (the solution)
Next Topic:	TLB-Widget focus

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 12:43:24 PDT 2025

Total time taken to generate the page: 1.28371 seconds