| Re: Roundoff error in SMOOTH [message #15480] |
Wed, 26 May 1999 00:00 |
thompson
Messages: 584
Registered: August 1991
|
Senior Member |
|
|
landsman@stars.gsfc.nasa.gov (Wayne Landsman) writes:
> I was recently surprised to discover that applying the SMOOTH function to a
> non-negative array could yield an array with negative numbers. I give an
> example below. This problem is evidently due to some sort of roundoff error,
> since it does not occur when using double precision. But it is not obvious to
> me how averaging 9 non-negative numbers (for a 3x3 box smooth) could yield a
> negative number, even allowing for roundoff error.
> Although not obvious from my simple example, this has nothing to do with edge
> effects -- I originally found the problem when 3x3 smoothing a 1024 x 1024
> array.
(rest deleted)
My theory is that the SMOOTH function probably uses Fourier transforms to speed
up the convolution, and this is where the round-off error is sneaking in.
Bill Thompson
|
|
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
