| Re: Set Precision??????????? [message #15838 is a reply to message #15790] |
Wed, 16 June 1999 00:00   |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
wbiagiot@suffolk.lib.ny.us wrote:
>
>> However I would like to create a variable corresonding to a fraction
> of a
>> Year (e.g 1995.123456). I have not been able to find a procedure
> allowing
>> me to create 10 digit decimal number , without the number being
> rounded
>> off to 8 digits.
>> Does anyone know of a way to declare a variable to hold a 10 digit
>> decimal number?
>
> The other replies were on the money with the correct answer, but also
> consider this: there are only 365 days in a year. I'm pretty sure you
> could get that resolute with even a "float" (4 byte decimal).
>
> - Bill B.
>
Bill,
you had better not hit the send button for this message!! A reply
like yours is what caused the Y2K bug (admitted: it has created many
jobs). Sure, if you are only after "daily" resolution, then a simple
float is fine, but as soon as you want to extract seconds, you get
terrible roundoff errors. And believe it or not: there are datasets in
1-second resolution (or finer) that are stored with the day of the year.
More systematically:
1 year ~ 3e7 secs ~ 3e10 msecs ~ 3e13 usecs
i.e. with 7 digits you can get to about 10 secs, with 15 digits you
achieve better than microsecond resolution.
Regards,
Martin
--
|||||||||||||||\\\\\\\\\\\\\-------------------///////////// //|||||||||||||||
Martin Schultz, DEAS, Harvard University, 29 Oxford St., Pierce 109,
Cambridge, MA 02138 phone (617) 496 8318 fax (617) 495 4551
e-mail mgs@io.harvard.edu web http://www-as/people/staff/mgs/
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
