| Re: Set Precision??????????? [message #15787 is a reply to message #15786] |
Mon, 14 June 1999 00:00  |
Liam Gumley
Messages: 473
Registered: November 1994
|
Senior Member |
|
|
Anil Kochhar wrote:
> I'd like create a varible which always holds numbers out to 6 places
> passed the decimal point, and 4 numbers before the decimal point (e.g.
> 1999.123456). I tried double
> precision but this seems to only hold 8 digits total for decimal numbers.
> However I would like to create a variable corresonding to a fraction of a
> Year (e.g 1995.123456). I have not been able to find a procedure allowing
> me to create 10 digit decimal number , without the number being rounded
> off to 8 digits.
> Does anyone know of a way to declare a variable to hold a 10 digit
> decimal number?
I think you are confusing internal machine precision with print
formatting precision. For example, to print the double precision PI
system variable:
IDL> print, !dpi
3.1415927
IDL> print, !dpi, format='(e20.10)'
3.1415926536e+00
The default print format only prints 8 digits, but you can change the
print format to print more digits. On machines with IEEE arithmetic, 64
bit double precision stores about 15 digits of information. For a nice
discussion of how floating point numbers are represented, see 'Section 4
- Floating-point numbers' at
http://metalab.unc.edu/pub/languages/fortran/unfp.html
Cheers,
Liam.
--
Liam E. Gumley
Space Science and Engineering Center, UW-Madison
http://cimss.ssec.wisc.edu/~gumley
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
