comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » modulo operator

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: modulo operator [message #86447 is a reply to message #86446]

Sat, 09 November 2013 07:53

spluque
Messages: 33
Registered: September 2013

Member

On Friday, November 8, 2013 9:36:39 PM UTC-6, Craig Markwardt wrote:
> On Friday, November 8, 2013 5:54:30 PM UTC-5, spl...@gmail.com wrote:
>
>> Hi,
>
>>
>
>>
>
>>
>
>> I'm a little confused by the following:
>
>>
>
>>
>
>>
>
>> IDL> print, 1200 MOD 0.1
>
>>
>
>> 0.0999821
>
>>
>
>> IDL> print, (1200 * 10.0) MOD (0.1 * 10)
>
>>
>
>> 0.00000
>
>>
>
>>
>
>>
>
>> I expected this to be 0 either way. What am I missing?
>
>
>
> See David's post.
>
> Then try the same experiment with double precision everywhere instead.
>
>
>
> IDL> print, 1200d MOD 0.1d
>
>
>
> IDL> print, (1200d * 10d) MOD (0.1d * 10d)
>

Thank you both for these pointers. I didn't expect the difference to be so large due to these numerical representation issues. So what is the canonical way to guard against this?

Seb

Report message to a moderator

[Message index]

		modulo operator By: spluque on Fri, 08 November 2013 14:54
		Re: modulo operator By: David Fanning on Fri, 08 November 2013 15:09
		Re: modulo operator By: Craig Markwardt on Fri, 08 November 2013 19:36
		Re: modulo operator By: spluque on Sat, 09 November 2013 07:53
		Re: modulo operator By: tom.grydeland on Sun, 10 November 2013 14:05

Previous Topic:	strange !P.multi[0] behavior??
Next Topic:	Feature request for DILATE function

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Thu Oct 09 20:38:21 PDT 2025

Total time taken to generate the page: 1.19987 seconds