| Re: inexplicable LONG() - behaviour [message #2870] |
Wed, 07 September 1994 06:51  |
thompson
Messages: 584
Registered: August 1991
|
Senior Member |
|
|
frank@chaos.uni-frankfurt.dbp.de (Frank Hoffsuemmer) writes:
> Hello,
> I'm using IDL 3.1.1 (no update in sight :-( ) under HP-UX.
> And there are some strange things happening....
> Of course, these are just things that I understand wrong :), so could someone
> please explain this behaviour:
> IDL. Version 3.1.1 (hp-ux hp_pa).
> Copyright 1989-1993, Research Systems, Inc.
> All rights reserved. Unauthorized reproduction prohibited.
> Installation number: 3063.
> Licensed for use by: Johann Wolfgang Goethe-Universitaet, HRZ
>
> X-IDL> print, long(1231231434.1)
> 1231231488
This behavior exists in IDL 3.5 as well.
The reason for this is quite simple. The argument to the LONG function is a
floating point number. The way you've phrased the statement, it's a *single*
*precision* floating point number. Thus, it's already lost accuracy before you
even get to converting it to a long integer. You can see this by entering the
following command:
IDL> print, 1231231434.1, format='(f20.1)'
1231231488.0
On the other hand, if you define the floating point constant to be double
precision, then it works fine, i.e.
IDL> print, long(1231231434.1d0)
1231231434
Bill Thompson
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
