| Precision Problem [message #49403] |
Mon, 24 July 2006 09:51  |
DMac
Messages: 6
Registered: May 2006
|
Junior Member |
|
|
Hello All,
Using IDL version 6.2 on a Windows XP machine I am reading in a set of
coordinates from a binary file (it is LiDAR data stored in LAS Version
1.0 format) stored in UTM NAD83 Zone 11 and these points are stored as
4 byte long integers in the binary file . The coordinates are stored
in the binary file with an offset such that they need to be divided by
100 to obtain the actual coordinates. The binary file is read into a
heap variable (Data below). The X and Y coordinates are than written
from the heap variable into a double precision array as follows:
x = dblarr(Num_pts)
x = TEMPORARY(data.x*1.000e-002)
y = dblarr(Num_pts)
y = TEMPORARY(data.y*1.000e-002)
Eventually these points are output to an ESRI shapefile. My problem is
that the y coordinates are rounded to the nearest integer value in the
shapefile. The X coordinates are fine. The X coordinates are 6 digits
to the left of the decimal place while the Y coordinates are 7 digits
to the left of the decimal place
Can anyone shed any light on what is going on here or what I might be
doing wrong?
Thanks.
Derek M.
|
|
|
|
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
