comp.lang.idl-pvwave archive: archive » How to speed up kernel density smoothing for many data points

Home » Public Forums » archive » How to speed up kernel density smoothing for many data points

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Re: How to speed up kernel density smoothing for many data points [message #86136 is a reply to message #86133]

Thu, 10 October 2013 06:48

Moritz Fischer
Messages: 32
Registered: June 2013

Member

as allways: time vs memory. try below.
I guess one could also specialize matrix_eucl... for this very case,
where n=1 and k=2, especially removing the make_array part and the loop.

PRO test_kernel

N = 1000000
r = 2*randomn(seed,N)
v = 2*randomu(seed,N)
x = [[r],[v]]

; Just a smoothing parameter, unimportant...
hopt = 6.24/(N^(1./6.))*sqrt((stddev(x(*,0))^2+stddev(x(*,1))^2)/2. )

; Slow loop, where I need help
f1 = fltarr(N)
for i=0L,N-1 do begin
mat = matrix_euclidean_distance( x[i,*], x) ; line by line...
f1(i) = 1./float(N) * total(1./(hopt^2)*K(mat,hopt))
endfor

END

FUNCTION K, t ,h
aa = where(t ge 1., n0, COMPLEMENT = bb, nCOMPLEMENT=nt)
if n0 ne 0 then t(aa) = 0.
if nt ne 0 then t(bb) = 4./!Pi*(1.-t(bb)^2)^3
RETURN,t
END

Am 10.10.2013 15:29, schrieb jacobsvensmark@gmail.com:
> On Thursday, October 10, 2013 3:03:15 PM UTC+2, Moritz Fischer
> wrote:
>> ...and taking a look at K:
>>
>> check out the COMPLEMENT keyword to WHERE!
>>
>> Note that RES[aa] = 0. is redundant (it gets initialized with
>> zeros)!
>>
>> --m
>>
>>
>>
>>
>>
>> Am 10.10.2013 14:22, schrieb :
>>
>>> Okay so I have a long array of N 2D points:
>
> Hey,
>
> Thanks, I removed the RES[aa] = 0, good point. That will give some
> speed. And thanks for your help with the matrix_euclidean_distance
> program - I tested it out, and for N=1000 it runs instantly, but for
> N=10000, its very slow and becomes unresponsive, and for N=100000 it
> just spits out "% Array has too many elements". Makes sense because I
> guess your program effectively makes a NxN matrix from the N
> points...
>

Report message to a moderator

[Message index]

		How to speed up kernel density smoothing for many data points By: jacobsvensmark on Thu, 10 October 2013 05:22
		Re: How to speed up kernel density smoothing for many data points By: Moritz Fischer on Thu, 10 October 2013 05:55
		Re: How to speed up kernel density smoothing for many data points By: Moritz Fischer on Thu, 10 October 2013 06:03
		Re: How to speed up kernel density smoothing for many data points By: jacobsvensmark on Thu, 10 October 2013 06:29
		Re: How to speed up kernel density smoothing for many data points By: Moritz Fischer on Thu, 10 October 2013 06:48

Previous Topic:	Newline in WIDGET_LABEL
Next Topic:	CMIP5 files - time

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Sun Oct 12 02:59:33 PDT 2025

Total time taken to generate the page: 1.68204 seconds