comp.lang.idl-pvwave archive
Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado

Home » Public Forums » archive » Re: Negative Variance?
Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend 
Return to the default flat view Create a new topic Submit Reply
Re: Negative Variance? [message #65768 is a reply to message #65767] Thu, 19 March 2009 19:43 Go to previous messageGo to previous message
wlandsman is currently offline  wlandsman
Messages: 743
Registered: June 2000
Senior Member
On Mar 19, 6:00 pm, "anniebry...@gmail.com" <anniebry...@gmail.com>
wrote:
> I am using the VARIANCE comand in IDL and am getting negative values.

I would check that you not getting round-off problems by using double
precision or the /DOUBLE keyword to VARIANCE. As discussed in
http://en.wikipedia.org/wiki/Algorithms_for_calculating_vari ance
round-off error can accumulate in the two-pass variance calculation
(used by IDL) when most values are similar but some are wildly
different.

I had a similar headscratcher a few years back when I found that
SMOOTHing a non-negative array could yield negative numbers. This
was a roundoff problem that was fixed by using double precision.

--Wayne
[Message index]
 
Read Message
Read Message
Read Message
Read Message
Read Message
Read Message
Previous Topic: Re: Make easy and nice your IDL documentation
Next Topic: Span a graph across 2 spaces in a !P.MULTI environment?

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Wed Oct 08 19:21:02 PDT 2025

Total time taken to generate the page: 0.00461 seconds