comp.lang.idl-pvwave archive: archive » Re: machine precision

Home » Public Forums » archive » Re: machine precision

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: machine precision [message #66496 is a reply to message #66495]

Tue, 19 May 2009 11:15

Kenneth P. Bowman
Messages: 585
Registered: May 2000

Senior Member

In article
<6c259101-aa56-4964-b01f-cd1c96bdcfc3@m24g2000vbp.googlegroups.com>,
"Jeff N." <jeffnettles4870@gmail.com> wrote:

> $i = int(4.70*100);
> print "$i\n";
>
> prints 470 whereas the equivalent IDL code: print, fix(4.70*100)
> prints 469.

Have you looked to see exactly what the perl fix function does?
Perhaps it rounds.

In IDL you might want to try ROUND, CEIL, or FLOOR, depending
on exactly what you are trying to accomplish.

People often use LONG when what they really want is ROUND.

Ken Bowman

Report message to a moderator

[Message index]

		Re: machine precision By: Wout De Nolf on Wed, 20 May 2009 06:08
		Re: machine precision By: Carsten Lechte on Wed, 20 May 2009 03:44
		Re: machine precision By: jameskuyper on Wed, 20 May 2009 03:28
		Re: machine precision By: Carsten Lechte on Wed, 20 May 2009 01:46
		Re: machine precision By: David Fanning on Tue, 19 May 2009 20:01
		Re: machine precision By: jameskuyper on Tue, 19 May 2009 18:41
		Re: machine precision By: Kenneth P. Bowman on Tue, 19 May 2009 11:15
		Re: machine precision By: jeffnettles4870 on Tue, 19 May 2009 08:16
		Re: machine precision By: jameskuyper on Mon, 18 May 2009 18:44
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 07:02
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 06:55
		Re: machine precision By: jameskuyper on Mon, 18 May 2009 06:33
		Re: machine precision By: David Fanning on Mon, 18 May 2009 06:30
		Re: machine precision By: Wout De Nolf on Mon, 18 May 2009 06:04
		Re: machine precision By: David Fanning on Mon, 18 May 2009 04:53

Previous Topic:	Re: Simultaneous fitting in IDL
Next Topic:	Simultaneous fitting in IDL

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sat Oct 11 09:04:40 PDT 2025

Total time taken to generate the page: 0.48051 seconds