comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » Re: Funny math?

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Funny math? [message #20726]

Mon, 24 July 2000 00:00

Bruce Bowler
Messages: 128
Registered: September 1998

Senior Member

And of course there's the issue of not being able to represent certain
decimal numbers _exactly_ in the binary number system...

Simon de Vet wrote:
>
> I've got some arrays with data. None extend past 3 decimal places.
>
> 'print, filterfunc(0,0,half1), filterfunc(0,0,half2+200)'
> produces:
> '537.100 528.300'
>
> 'print, filterfunc(0,0,half1) - filterfunc(0,0,half2+200)'
> produces:
> '8.79999'
>
> What's up with that?
>
> Simon

Report message to a moderator

[Message index]

		Re: Funny math? By: Bruce Bowler on Mon, 24 July 2000 00:00
		Re: Funny math? By: Alex Schuster on Thu, 20 July 2000 00:00

Previous Topic:	Re: launching web browser?
Next Topic:	Re: Hints for optical flow analysis anyone?

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 12:46:50 PDT 2025

Total time taken to generate the page: 0.40262 seconds