SMOOTH [message #9360] |
Thu, 19 June 1997 00:00 |
J.D. Smith
Messages: 214 Registered: August 1996
|
Senior Member |
|
|
Hi Folks,
I've encountered what may be a serious error in the SMOOTH algorithm.
It appears SMOOTH suffers catastrophic roundoff errors with boxsizes as
small as 3x3 on some floating point data. What's more, the error is
position dependent inside the array, making strange "band" structures in
the smoothed image. In my case, I was smoothing a positive definite
image, and the smoothed image contained bands of negative values! I
think arrays which contain a wide dynamic range of values are most
susceptible. Converting to double precision seems to solve the problem,
although I cannot see why double precision would be needed for what
seems naively to be a simple problem. I am using SunOs 4.1.3 on a Sparc
IPC. Has anyone else encountered this error? Does anyone know what
tricks IDL uses to speed the SMOOTh process?
Thanks,
JD
|
|
|