comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: TRIGRID precision

TRIGRID precision
type: item
description: I encountered unsatisfactory behavior from TRIGRID relating to the precision of the (x,y) locations of the grid which it produces. I am using double precision x,y,z input data. The documentation states that TRIGRID handles double precision z data, but there is no mention of the precision of the x,y data. I suspect TRIGRID is always using float precision to compute the x,y grid locations. The problem showed up when I was changing the units on my data. By scaling the x,y data, the limits and the gridspacing all together, I was getting an output grid whose dimensions were changing by one (it changed from 601 to 602: an error of 1/601=.0017 for double precision input data!). The consequences of this precision is that the interpolated result has an error greater than my intended tolerance. I suppose I can work around this by scaling all the (x,y) variables (i.e. the input data, the gridspacing, and the limits) to a range where the errors are minimized, and then scaling the result back to the original units. This ranks up there with the worst kludges on which I've had to rely. Anyone have an alternative to TRIGRID? Any way to get TRIGRID to use DOUBLEs instead of FLOATs for its intermediate calculations? Even simple commiseration will be appreciated. I am running IDL 3.6.1a under Solaris2.3 Serdar Manizade EG&G - NASA Wallops Flight Facility http://aol16.wff.nasa.gov/
creator: Serdar Manizade
date: 1995-06-27T07:00:00-00:00