comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Precision Problem

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Precision Problem [message #49403]

Mon, 24 July 2006 09:51

DMac
Messages: 6
Registered: May 2006

Junior Member

Hello All,
Using IDL version 6.2 on a Windows XP machine I am reading in a set of
coordinates from a binary file (it is LiDAR data stored in LAS Version
1.0 format) stored in UTM NAD83 Zone 11 and these points are stored as
4 byte long integers in the binary file . The coordinates are stored
in the binary file with an offset such that they need to be divided by
100 to obtain the actual coordinates. The binary file is read into a
heap variable (Data below). The X and Y coordinates are than written
from the heap variable into a double precision array as follows:
x = dblarr(Num_pts)
x = TEMPORARY(data.x*1.000e-002)

y = dblarr(Num_pts)
y = TEMPORARY(data.y*1.000e-002)

Eventually these points are output to an ESRI shapefile. My problem is
that the y coordinates are rounded to the nearest integer value in the
shapefile. The X coordinates are fine. The X coordinates are 6 digits
to the left of the decimal place while the Y coordinates are 7 digits
to the left of the decimal place

Can anyone shed any light on what is going on here or what I might be
doing wrong?

Thanks.

Derek M.

Report message to a moderator

[Message index]

		Precision Problem By: DMac on Mon, 24 July 2006 09:51
		Re: Precision Problem By: Mark Hadfield on Mon, 24 July 2006 15:10
		Re: precision problem By: David Fanning on Tue, 18 August 2009 05:58

Previous Topic:	precision problem
Next Topic:	Read BUFR Meteorological Data in IDL?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Fri Oct 10 05:29:42 PDT 2025

Total time taken to generate the page: 0.16002 seconds