comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Floats

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Floats [message #47964 is a reply to message #47862]

Fri, 10 March 2006 01:29

Liberum
Messages: 48
Registered: September 2005

Member

Boy, you guys really went off on this issue. But it is good.
Much is revealed about IDL, the art of programming, and, an added
extra, the personalities of the writers :)
To answer an earlier question of why I asked. Well, maybe I am just too
curious about things :)
I used Matlab before and it had the possibility to limit the precision.
But I could not control it too much.
I am working with a large number of arrays and averaging millions of
pixel values. As a result I am getting rounding errors.
I stop the program here and there and check the data at different
points. As such I only want a precision to the nearest hundreth.
It makes for quick assessment. Silly but,I am still learning :)
I need to understand more fundemental things about ROUND, FIX, FLOAT,
and LONG so as to eliminate some of these annoying errors. (Thanks
David for the info)
Using double precision seem to be the next step for me.

/Sheldon

Report message to a moderator

[Message index]

		Floats By: Liberum on Wed, 08 March 2006 12:52
		Re: Floats By: Liberum on Sun, 12 March 2006 00:32
		Re: Floats By: Liberum on Sun, 12 March 2006 00:22
		Re: Floats By: R.Bauer on Sat, 11 March 2006 04:08
		Re: Floats By: Paul Van Delst[1] on Fri, 10 March 2006 08:45
		Re: Floats By: Kenneth P. Bowman on Fri, 10 March 2006 06:27
		Re: Floats By: Liberum on Fri, 10 March 2006 01:29
		Re: Floats By: Mark Hadfield on Thu, 09 March 2006 13:10
		Re: Floats By: David Fanning on Thu, 09 March 2006 06:19

Previous Topic:	accuracy problems
Next Topic:	Solaris 10 compatibility

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Tue Dec 02 06:12:03 PST 2025

Total time taken to generate the page: 1.52028 seconds