comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Rounding off

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Rounding off [message #70607 is a reply to message #70591]

Wed, 21 April 2010 14:52

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

Carsten Lechte writes:

> It should also be noted, that this method, or indeed anything that
> involves rounding, does not work if the number to round does not fit
> into a 32bit or 64bit integer (you will get an exception and the
> result will be garbage). If that happens, you are probably trying to
> round at a position waaaaay past the last significant digit anyway,
> so the whole rounding step should be skipped. But you have to test for
> it.

If you are looking for people with recent experience
to testify to how much trouble you can get into by
not testing data type bounds, I'm available at
my professional rate of $0.25/hour.

Cheers,

David

--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")

Report message to a moderator

[Message index]

		Re: Rounding off By: Bob[3] on Thu, 22 April 2010 09:59
		Re: Rounding off By: Paul Van Delst[1] on Thu, 22 April 2010 09:37
		Re: Rounding off By: David Fanning on Wed, 21 April 2010 14:52
		Re: Rounding off By: David Fanning on Wed, 21 April 2010 14:48
		Re: Rounding off By: Carsten Lechte on Wed, 21 April 2010 14:39
		Re: Rounding off By: cgguido on Wed, 21 April 2010 14:20
		Re: Rounding off By: David Fanning on Wed, 21 April 2010 13:50
		Re: Rounding off By: David Fanning on Wed, 21 April 2010 13:49

Previous Topic:	Re: isurface madness?
Next Topic:	baffling syntax error

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Nov 29 22:11:54 PST 2025

Total time taken to generate the page: 0.00264 seconds