| TRIGRID precision [message #4647] |
Tue, 27 June 1995 00:00 |
Serdar Manizade
Messages: 15
Registered: December 1994
|
Junior Member |
|
|
I encountered unsatisfactory behavior from TRIGRID relating to the
precision of the (x,y) locations of the grid which it produces. I
am using double precision x,y,z input data. The documentation states
that TRIGRID handles double precision z data, but there is no mention
of the precision of the x,y data. I suspect TRIGRID is always using
float precision to compute the x,y grid locations.
The problem showed up when I was changing the units on my data. By
scaling the x,y data, the limits and the gridspacing all together, I
was getting an output grid whose dimensions were changing by one
(it changed from 601 to 602: an error of 1/601=.0017 for double
precision input data!). The consequences of this precision
is that the interpolated result has an error greater than my intended
tolerance. I suppose I can work around this by scaling all the (x,y)
variables (i.e. the input data, the gridspacing, and the limits) to
a range where the errors are minimized, and then scaling the result
back to the original units. This ranks up there with the worst
kludges on which I've had to rely.
Anyone have an alternative to TRIGRID? Any way to get TRIGRID to use
DOUBLEs instead of FLOATs for its intermediate calculations?
Even simple commiseration will be appreciated.
I am running IDL 3.6.1a under Solaris2.3
Serdar Manizade
EG&G - NASA Wallops Flight Facility
http://aol16.wff.nasa.gov/
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
