comp.lang.idl-pvwave archive
Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado

Home » Public Forums » archive » Incorrect behavior of /NAN
Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend 
Switch to threaded view of this topic Create a new topic Submit Reply
Incorrect behavior of /NAN [message #30033] Thu, 04 April 2002 07:40 Go to next message
K. Bowman is currently offline  K. Bowman
Messages: 330
Registered: May 2000
Senior Member
I tried to incite some interest in this last week, but had no takers,
so I'll try again. ;-)

I believe that the behavior of the TOTAL function is incorrect in
the case where all of the data are NaN and the /NAN keyword is set.

Here are several possible cases:

IDL> a = !values.f_nan
IDL> print, total([a, 1.0])
NaN
IDL> print, total([a, 1.0], /nan)
1.00000
IDL> print, total([a, a])
NaN
IDL> print, total([a, a], /nan)
0.00000

I believe that the last case is incorrect.

The documentation for /NAN says "Elements with the value NaN are treated
as missing data." In the last case there are no valid data, so how can
their sum be zero?

I think this is probably a simple error in the TOTAL algorithm.
Rather than computing the "sum of all the non-NaN values", it is
probably doing "set sum to zero and then add all non-Nan values".


For comparison, the MEAN function does not behave this way.

IDL> print, mean([a, 1.0])
NaN
IDL> print, mean([a, 1.0], /nan)
1.00000
IDL> print, mean([a, a])
NaN
IDL> print, mean([a, a], /nan)
NaN


Before I submit something (at least a question) to RSI, does anyone
have comments?

Ken Bowman
Re: Incorrect behavior of /NAN [message #30134 is a reply to message #30033] Sat, 06 April 2002 07:46 Go to previous messageGo to next message
Craig Markwardt is currently offline  Craig Markwardt
Messages: 1869
Registered: November 1996
Senior Member
Kenneth Bowman <k-bowman@null.tamu.edu> writes:

> In article <onu1qpzubn.fsf@cow.physics.wisc.edu>,
> Craig Markwardt <craigmnet@cow.physics.wisc.edu> wrote:
>
>> I could easily argue the other way, in terms of reasonableness. The
>> sum of "no" numbers is zero.
>
> Zero is a number. How can the sum of no numbers be a number?
> If there are no numbers how can there even be a sum?

I think either value is a "reasonable" return from TOTAL, zero or NaN,
in the case you describe. The real issue is that it's not documented.

Now where did I put that box of cupcakes...

Craig

--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
Re: Incorrect behavior of /NAN [message #30137 is a reply to message #30033] Sat, 06 April 2002 06:28 Go to previous messageGo to next message
Kenneth P. Bowman is currently offline  Kenneth P. Bowman
Messages: 585
Registered: May 2000
Senior Member
In article <MPG.171804089a266d298986e@news.frii.com>,
David Fanning <david@dfanning.com> wrote:

> "How many marbles are in the box?"
>
> "None. The box is full of cupcakes."
>
> Then the sum of marbles in the box is 0.

You English language statement is almost correct, but it
should be "the number of marbles in the box is 0".

Translating to IDL

i = WHERE(thingInBox EQ marble, numberOfMarbles)

not

numberOfMarbles = TOTAL(thingInBox, /NaMarble)

;-)

Ken
Re: Incorrect behavior of /NAN [message #30141 is a reply to message #30033] Fri, 05 April 2002 17:57 Go to previous messageGo to next message
David Fanning is currently offline  David Fanning
Messages: 11724
Registered: August 2001
Senior Member
James Kuyper (kuyper@gscmail.gsfc.nasa.gov) writes:

> The number of ball bearings produced in a given city during a specified
> time period can be expressed as a sum over the amounts produced by each
> ball bearing plant in that city. What should the value of that sum be,
> for a city that has no ball bearing plants? I'd say that it's definitely
> a number, and definitely 0. I think that this is the most reasonable
> value for the sum of almost any variable-length list, when that list
> happens to have a length of 0. It's also the natural result of the most
> obvious algorithm for calculating the sum.

I agree. I was thinking about this while I was
feeding balls to my tennis team. (Don't ask,
please.) A sum is a counting operation.

"How many marbles are in the box?"

"None. The box is full of cupcakes."

Then the sum of marbles in the box is 0.

Seems pretty straightforward to me. But then
I've never understood the whole Christian/Muslim
thing either. :-(

Cheers,

David

--
David W. Fanning, Ph.D.
Fanning Software Consulting
Phone: 970-221-0438, E-mail: david@dfanning.com
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Toll-Free IDL Book Orders: 1-888-461-0155
Re: Incorrect behavior of /NAN [message #30143 is a reply to message #30033] Fri, 05 April 2002 14:53 Go to previous messageGo to next message
James Kuyper is currently offline  James Kuyper
Messages: 425
Registered: March 2000
Senior Member
Kenneth Bowman wrote:

> In article <onu1qpzubn.fsf@cow.physics.wisc.edu>,
> Craig Markwardt <craigmnet@cow.physics.wisc.edu> wrote:
>
>
>> I could easily argue the other way, in terms of reasonableness. The
>> sum of "no" numbers is zero.
>
>
> Zero is a number. How can the sum of no numbers be a number?
> If there are no numbers how can there even be a sum?
>
> Ken

The number of ball bearings produced in a given city during a specified
time period can be expressed as a sum over the amounts produced by each
ball bearing plant in that city. What should the value of that sum be,
for a city that has no ball bearing plants? I'd say that it's definitely
a number, and definitely 0. I think that this is the most reasonable
value for the sum of almost any variable-length list, when that list
happens to have a length of 0. It's also the natural result of the most
obvious algorithm for calculating the sum.
Re: Incorrect behavior of /NAN [message #30144 is a reply to message #30033] Fri, 05 April 2002 14:25 Go to previous messageGo to next message
K. Bowman is currently offline  K. Bowman
Messages: 330
Registered: May 2000
Senior Member
In article <onu1qpzubn.fsf@cow.physics.wisc.edu>,
Craig Markwardt <craigmnet@cow.physics.wisc.edu> wrote:

> I could easily argue the other way, in terms of reasonableness. The
> sum of "no" numbers is zero.

Zero is a number. How can the sum of no numbers be a number?
If there are no numbers how can there even be a sum?

Ken
Re: Incorrect behavior of /NAN [message #30145 is a reply to message #30033] Fri, 05 April 2002 13:53 Go to previous messageGo to next message
Craig Markwardt is currently offline  Craig Markwardt
Messages: 1869
Registered: November 1996
Senior Member
Kenneth Bowman <k-bowman@null.tamu.edu> writes:

> In article <onofgz74jo.fsf@cow.physics.wisc.edu>,
> Craig Markwardt <craigmnet@cow.physics.wisc.edu> wrote:
>
>> If all the data are missing, how can the sum be NaN? Neither a return
>> value of 0, nor a return value of NaN, seems to be appropriate to
>> me. This appears to be an undefined case.
>
> If NaN represents missing data, and all of the data are missing, then the
> total is missing also, so it seems reasonable to me to return NaN.

:-)

I could easily argue the other way, in terms of reasonableness. The
sum of "no" numbers is zero.

Craig

--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
Re: Incorrect behavior of /NAN [message #30157 is a reply to message #30033] Fri, 05 April 2002 07:35 Go to previous messageGo to next message
Paul van Delst is currently offline  Paul van Delst
Messages: 364
Registered: March 1997
Senior Member
Craig Markwardt wrote:
>
> Paul van Delst <paul.vandelst@noaa.gov> writes:
>>
>> Maybe a holistic division operator is required in these sorts of cases. :o)
>
> Or should IDL apply L'Hopital's rule? :-)

That would be good - I'd never have to fret over calculating sinc functions again.....

paulv

--
Paul van Delst Religious and cultural
CIMSS @ NOAA/NCEP purity is a fundamentalist
Ph: (301)763-8000 x7274 fantasy
Fax:(301)763-8545 V.S.Naipaul
Re: Incorrect behavior of /NAN [message #30167 is a reply to message #30033] Thu, 04 April 2002 13:52 Go to previous messageGo to next message
Paul van Delst is currently offline  Paul van Delst
Messages: 364
Registered: March 1997
Senior Member
Mark Hadfield wrote:
>
> "Paul van Delst" <paul.vandelst@noaa.gov> wrote in message
> news:3CAC9914.B52D1669@noaa.gov...
>> Seems reasonable to me. It only sums the data not flagged as missing. If
>> all the data is missing, the sum of nothing is, well, nothing.
>
> And IDL normally uses NaN ("Not a Number") to represent nothing.

Huh. I though that's what zero was for. Or is a throwback from Roman times:

... -V -IV -III -II -I NaN I II III IV V ......

paulv

--
Paul van Delst Religious and cultural
CIMSS @ NOAA/NCEP purity is a fundamentalist
Ph: (301)763-8000 x7274 fantasy
Fax:(301)763-8545 V.S.Naipaul
Re: Incorrect behavior of /NAN [message #30238 is a reply to message #30033] Mon, 15 April 2002 06:47 Go to previous message
K. Bowman is currently offline  K. Bowman
Messages: 330
Registered: May 2000
Senior Member
In article <yx6hvgatn3vp.fsf@socrates.Berkeley.EDU>,
noymer@socrates.Berkeley.EDU wrote:

> NaN is an IEEE standard, is it not?
>
> Does the IEEE define what to do in such cases?
>
> It seems to me that IDL should conform to such a standard, if it exists,
> even if it goes against our intuition.

As far as I know, IDL correctly does arithmetic involving NaNs
(i.e., it returns NaNs, etc.).

But IDL also uses NaNs as "missing values", which is not an IEEE issue,
and the behavior here is not clearly defined in some cases.

Ken
Re: Incorrect behavior of /NAN [message #30243 is a reply to message #30033] Sun, 14 April 2002 18:32 Go to previous message
noymer is currently offline  noymer
Messages: 65
Registered: June 1999
Member
NaN is an IEEE standard, is it not?

Does the IEEE define what to do in such cases?

It seems to me that IDL should conform to such a standard, if it exists,
even if it goes against our intuition.

That said, doing a google on IEEE 754 and NaN and sum didn't turn up a
definitive answer (but I didn't look at all of the over 1,000 pages
google found).

Just my $0.02

Andrew
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: Vector Field Plot on Irregular Grid
Next Topic: Vector Field Plot on Irregular Grid

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Wed Oct 08 13:42:35 PDT 2025

Total time taken to generate the page: 0.00795 seconds