comp.lang.idl-pvwave archive: archive » Histogram indeterminate results

Home » Public Forums » archive » Histogram indeterminate results

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Histogram indeterminate results [message #33248 is a reply to message #33214]

Sat, 14 December 2002 09:03

James Kuyper
Messages: 425
Registered: March 2000

Senior Member

swisswuff wrote:
...
> Now know that mathematical results can be *simply* numerically unstable
> if calculated in the wrong data type on IDL.

Keep in mind that there's nothing magical about double precision vs.
single precision. For a sufficiently nasty situation, even double
precision is inadequate; even quad precision can fail. Sometimes the
best approach is not to use higher precision, but to find an entirely
different approach that's not as much of a strain on the available
precision. Unfortunately, it takes a lot of mathematical sophistication
to identify such a technique.

Report message to a moderator

[Message index]

		Histogram indeterminate results By: K. Bowman on Wed, 11 December 2002 09:29
		Re: Histogram indeterminate results By: James Kuyper on Sat, 14 December 2002 09:03

Previous Topic:	Re: Histogram indeterminate results
Next Topic:	Re: Read Total lines in an ASCII file

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 07:50:08 PDT 2025

Total time taken to generate the page: 0.38712 seconds