comp.lang.idl-pvwave archive: archive » Chunk Array Decimation

Home » Public Forums » archive » Chunk Array Decimation

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: Chunk Array Decimation [message #32514 is a reply to message #32420]

Thu, 10 October 2002 07:26

Jaco van Gorkom
Messages: 97
Registered: November 2000

Member

"JD Smith" <jdsmith@as.arizona.edu> wrote in message
news:pan.2002.10.10.00.18.45.172062.5209@as.arizona.edu...
>
> Aha, a very interesting and compact submission. I think there's one
> error there. Should it not be data[ri[nh+1:*]] without the inds? I
> translated as:
>
> h = histogram(inds, REVERSE_INDICES=ri)
> nh = n_elements(h)
> tdata = [0.,total(data[ri[nh+1:*]],/CUMULATIVE)]
> vec9 = tdata[ri[1:nh]-nh-1]-tdata[ri[0:nh-1]-nh-1]

Ok, this is how I had coded it originally. I tested it on some ten-element
sample vectors, got confused, and convinced myself that the extra inds had
to be in. Reading the original problem again, I suppose you are right.

> The biggest problem with this method, though, is roundoff error, which
> accumulates like mad in a cumulative total of any length. If you do
> it in floating point, as you've written, I find unacceptably large
> roundoff errors for as few as 500 individual indices.

Oops. Ok, so we should use /DOUBLE, indeed. But a quick test of
TOTAL(REPLICATE(1., 30000000), /CUMULATIVE) reveals roundoff errors only
after roughly 17,000,000 elements on my machine. If you tested the routine
with a data vector data = FINDGEN(100000) then the average value for each data element would be 50,000 , so
after 500 indices the cumulative total is
already 25,000,000. Realistic data might well have a lower average value,
or even zero average for data spread around zero, so that the cumulative
total of 25,000,000 is only reached after many more elements or never.

And what exactly do you mean by 'accumulates like mad in a cumulative
total'? Suffers a cumulative total more from roundoff error than a single
total? That should not be, should it? The way I see it, not having any
computer science background, the error in a cumulative total would be the
(very small) error in each sum (accumulating many, many times), plus the
(much larger) roundoff errors that occur when we get to very high values.
So as long as we do not get to high values we should be ok, especially
since we take differences out of subelements of the total, which will not
be very far apart, probably.

Just my thoughts,
Jaco

P.S. The (floating-point) roundoff error in TOTAL seems to come in
integer steps, how strange! Maybe more later in a new thread...

Report message to a moderator

[Message index]

		Chunk Array Decimation By: JD Smith on Tue, 01 October 2002 12:32
		Re: Chunk Array Decimation By: JD Smith on Fri, 04 October 2002 15:07
		Re: Chunk Array Decimation By: Jaco van Gorkom on Mon, 07 October 2002 08:51
		Re: Chunk Array Decimation By: JD Smith on Wed, 09 October 2002 17:18
		Re: Chunk Array Decimation By: JD Smith on Wed, 09 October 2002 17:19
		Re: Chunk Array Decimation By: Jaco van Gorkom on Thu, 10 October 2002 07:26
		Re: Chunk Array Decimation By: JD Smith on Thu, 10 October 2002 11:51
		Re: Chunk Array Decimation By: Keflavich on Tue, 18 November 2008 07:21
		Re: Chunk Array Decimation By: Wout De Nolf on Tue, 18 November 2008 02:52
		Re: Chunk Array Decimation By: Keflavich on Mon, 17 November 2008 13:40

Previous Topic:	ASTER Data
Next Topic:	reading slices of FITS files

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 20:13:47 PDT 2025

Total time taken to generate the page: 0.00642 seconds