comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: DOUBLE trouble

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: DOUBLE trouble [message #14555]

Tue, 09 March 1999 00:00

Axel vom Endt
Messages: 9
Registered: March 1999

Junior Member

Hi Jacco,

there are two things going on:

IDL like most programming languages uses hardware floating point
numbers. On all machines I've ever used that means using binary
fractions. 3.0 becomes 11.0 (which then will be stored as .11 in the
mantissa). 3.1 becomes a periodic binary fraction 11.0001100110011 which
cannot be stored without roundoff error. I hope I got the numbers right,
but this is explained in any textbook on numerical methods.

The second issue is printing in IDL. Try using an explicit FORMAT
statement

IDL> print, 3.1d0, format='(f60.55)'
3.1000000000000000888178419700125232338905334472656250000
IDL> print, 3.1e0, format='(f60.55)'
3.0999999046325683593750000000000000000000000000000000000

and you'll see that neither double nor single precision will be exactly
3.1

Axel

Report message to a moderator

[Message index]

Re: DOUBLE trouble

By: Axel vom Endt on Tue, 09 March 1999 00:00

Previous Topic:	voxel_proj images
Next Topic:	DOUBLE trouble

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 07:40:46 PDT 2025

Total time taken to generate the page: 0.87010 seconds