comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Inaccuracies

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Inaccuracies [message #5295 is a reply to message #5294]

Wed, 15 November 1995 00:00

Andy Loughe
Messages: 174
Registered: November 1995

Senior Member

William Thompson wrote:

>
> I believe that I was that user. Specifically, I reported that the expected

No, it was someone else who emailed me about this issue.

> answer of 0 was seen when the calculation was done in single precision.
> However, the same was not true in double precision. The only thing this
> demonstrates is that the problems of computer round-off error shows up in
> different ways in VMS than on other computers, and is simply because of the
> difference between the VAX floating point representation and the more standard
> IEEE one. In other words, it's a hardware difference, not a software one.

That is what I have learned as well.

>
> I think the discussion of how computer round-off error manifests itself is
> interesting. However, it's important to remember that this has nothing to do
> with IDL. Round-off errors are determined solely by the floating point
> processing done by the CPU. It was stated earlier, without testing, that IDL
> was deficient in this repect relative to Fortran or C--I showed in an earlier
> message that this was not the case.
>
> William Thompson

Agreed.

--
Andrew F. Loughe (afl@cdc.noaa.gov)
University of Colorado, CIRES * Campus Box 449 * Boulder, CO 80309
phone: (303) 492-0707 fax: (303) 497-7013

Report message to a moderator

[Message index]

		Re: Inaccuracies By: C R Shaw on Thu, 16 November 1995 00:00
		Re: Inaccuracies By: Hermann Mannstein on Thu, 16 November 1995 00:00
		Re: Inaccuracies By: wclodius on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: Andy Loughe on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: thompson on Wed, 15 November 1995 00:00
		Re: Inaccuracies By: hahn on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: lpmix on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: Andy Loughe on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: wclodius on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: thompson on Tue, 14 November 1995 00:00
		Re: Inaccuracies By: Jackel on Tue, 14 November 1995 00:00

Previous Topic:	Re: How can I add parameter to curvefit?
Next Topic:	XLoadCT in a procedure ?????

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 19:20:09 PDT 2025

Total time taken to generate the page: 0.00402 seconds