comp.lang.idl-pvwave archive: archive

Home » Public Forums » archive » modulo operator

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

modulo operator [message #86444]

Fri, 08 November 2013 14:54

spluque
Messages: 33
Registered: September 2013

Member

Hi,

I'm a little confused by the following:

IDL> print, 1200 MOD 0.1
0.0999821
IDL> print, (1200 * 10.0) MOD (0.1 * 10)
0.00000

I expected this to be 0 either way. What am I missing?

Thanks,
Seb

Report message to a moderator

Re: modulo operator [message #86445 is a reply to message #86444]

Fri, 08 November 2013 15:09

David Fanning
Messages: 11724
Registered: August 2001

Senior Member

spluque@gmail.com writes:

> I'm a little confused by the following:
>
> IDL> print, 1200 MOD 0.1
> 0.0999821
> IDL> print, (1200 * 10.0) MOD (0.1 * 10)
> 0.00000
>
> I expected this to be 0 either way. What am I missing?

Almost certainly the way numbers are represented on computers:

http://www.idlcoyote.com/math_tips/sky_is_falling.html

Cheers,

David

--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")

Report message to a moderator

Re: modulo operator [message #86446 is a reply to message #86444]

Fri, 08 November 2013 19:36

Craig Markwardt
Messages: 1869
Registered: November 1996

Senior Member

On Friday, November 8, 2013 5:54:30 PM UTC-5, spl...@gmail.com wrote:
> Hi,
>
>
>
> I'm a little confused by the following:
>
>
>
> IDL> print, 1200 MOD 0.1
>
> 0.0999821
>
> IDL> print, (1200 * 10.0) MOD (0.1 * 10)
>
> 0.00000
>
>
>
> I expected this to be 0 either way. What am I missing?

See David's post.
Then try the same experiment with double precision everywhere instead.

IDL> print, 1200d MOD 0.1d

IDL> print, (1200d * 10d) MOD (0.1d * 10d)

Craig

Report message to a moderator

Re: modulo operator [message #86447 is a reply to message #86446]

Sat, 09 November 2013 07:53

spluque
Messages: 33
Registered: September 2013

Member

On Friday, November 8, 2013 9:36:39 PM UTC-6, Craig Markwardt wrote:
> On Friday, November 8, 2013 5:54:30 PM UTC-5, spl...@gmail.com wrote:
>
>> Hi,
>
>>
>
>>
>
>>
>
>> I'm a little confused by the following:
>
>>
>
>>
>
>>
>
>> IDL> print, 1200 MOD 0.1
>
>>
>
>> 0.0999821
>
>>
>
>> IDL> print, (1200 * 10.0) MOD (0.1 * 10)
>
>>
>
>> 0.00000
>
>>
>
>>
>
>>
>
>> I expected this to be 0 either way. What am I missing?
>
>
>
> See David's post.
>
> Then try the same experiment with double precision everywhere instead.
>
>
>
> IDL> print, 1200d MOD 0.1d
>
>
>
> IDL> print, (1200d * 10d) MOD (0.1d * 10d)
>

Thank you both for these pointers. I didn't expect the difference to be so large due to these numerical representation issues. So what is the canonical way to guard against this?

Seb

Report message to a moderator

Re: modulo operator [message #86448 is a reply to message #86447]

Sun, 10 November 2013 14:05

tom.grydeland
Messages: 51
Registered: September 2012

Member

On Saturday, November 9, 2013 4:53:14 PM UTC+1, Sebastian Luque wrote:

> Thank you both for these pointers. I didn't expect the difference to be so large due to these numerical representation issues. So what is the canonical way to guard against this?

If you want to work with an integral number of tenths, multiply by 10 and do your mod's in integer math before dividing by 10 again.

> Seb

--T

Report message to a moderator

Previous Topic:	strange !P.multi[0] behavior??
Next Topic:	Feature request for DILATE function

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 13:44:04 PDT 2025

Total time taken to generate the page: 0.00683 seconds