comp.lang.idl-pvwave archive: archive » Re: HISTOGRAM and the Razor's Edge.

Home » Public Forums » archive » Re: HISTOGRAM and the Razor's Edge.

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: HISTOGRAM and the Razor's Edge. [message #35436 is a reply to message #35435]

Thu, 12 June 2003 14:54

timrobishaw
Messages: 16
Registered: June 2003

Junior Member

Thanks a bunch, folks. Your responses were very helpful. Much
appreciated.

-Tim.

Paul van Delst wrote:

> The result isn't wrong. Your assumptions about the numerical accuracy are.
> I'm amazed you're surprised at getting a result of 1.9999980926513671875 for
> the expression (-5.40-(-5.50))/0.05, Even double precision won't help you
> here:
>
> IDL> print, (-5.40d0-(-5.50d0))/0.05d0, format='(f21.19)'
> 1.9999999999999928946

Paul:

First of all, let me thank you for your amazement at my lack of
understanding. That's not a very friendly way to encourage people to
post questions to your group. Secondly, I was not surprised that your
example wasn't an integer; rather, I discovered this was exactly the
reason HISTOGRAM *wasn't* working as I expected it to. I was surprised
that HISTOGRAM was subtracting the MIN and then dividing by the
BINSIZE when this is bound to goof up values at the hairy edges of
bins. My question was whether or not there was a way to get around
this problem should you be expecting it.

Report message to a moderator

[Message index]

		Re: HISTOGRAM and the Razor's Edge. By: Paul Van Delst[1] on Fri, 13 June 2003 06:58
		Re: HISTOGRAM and the Razor's Edge. By: JD Smith on Thu, 12 June 2003 15:25
		Re: HISTOGRAM and the Razor's Edge. By: timrobishaw on Thu, 12 June 2003 14:54
		Re: HISTOGRAM and the Razor's Edge. By: meinel on Thu, 12 June 2003 08:03
		Re: HISTOGRAM and the Razor's Edge. By: Paul Van Delst[1] on Thu, 12 June 2003 07:40
		Re: HISTOGRAM and the Razor's Edge. By: David Fanning on Thu, 12 June 2003 07:30
		Re: HISTOGRAM and the Razor's Edge. By: James Kuyper on Thu, 12 June 2003 06:53
		Re: HISTOGRAM and the Razor's Edge. By: R.G. Stockwell on Thu, 12 June 2003 04:33
		Re: HISTOGRAM and the Razor's Edge. By: Craig Markwardt on Thu, 12 June 2003 00:04

Previous Topic:	how did this happen?
Next Topic:	Re: Redirect STDOUT

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 10:01:51 PDT 2025

Total time taken to generate the page: 0.88214 seconds