| Re: Can IDL calculate the confidence level about correlation [message #56882] |
Fri, 23 November 2007 17:30  |
wanglin1981
Messages: 10
Registered: August 2007
|
Junior Member |
|
|
On 11月24日, 上午5时02分, Vince Hradil <hrad...@yahoo.com> wrote:
> On Nov 22, 7:51 pm, Lin Wang <wanglin1...@gmail.com> wrote:
>
>> I found a code which can calculate the confidence level using pvalue
>> method (see below). It works well, but usually I use the Student's t-
>> test method. So can anyone help?
>
>> Thanks!
>
>> r=correlate(x,y)
>> var=1/(n-3.0)
>> zvalue = 0.5*alog((1+r)/(1-r))/sqrt(var)
>> if (zvalue lt 0) then pvalue = 2*(gauss_pdf(zvalue)) $
>> else pvalue = 2*(1-gauss_pdf(zvalue))
>
> I think THIS is the way to do it. The t-test is irrelevant.
Vince,
Yes, p-value test can test the significance of correlatioins, but t-
test is also widely used in meteorological studies, even more popular
than p-value test I think.
The probability density fuction for correlation coefficient r is:
f(r)=gama((n-1)/2)*(1-r*r)**(n/2-2)/sqrt(pi)/gama((n-2)/2)
Set r=t/sqrt(n-2)/sqrt(1+t*t/(n-2)), and v=n-2, then
f(r)dr=(after some transformation)=[1/sqrt(v*pi)]*[gama((v+1)/2)/
gama(v/2)]*[(1+t*t/2)**(-(v+1)/2)]*dt
this is the probability density function for t distribution, so the
significance of r can be evaluated by the t-test.
Lin
|
|
|
|
|
|
|
|
|
|
| Re: Can IDL calculate the confidence level about correlation [message #56888 is a reply to message #56885] |
Fri, 23 November 2007 04:06  |
yp
Messages: 42
Registered: February 2005
|
Member |
|
|
Not exactly what you want... but this might give you some idea.
;+
; PRO: sig_test
; Significance Test of correlation for a sample size = N
;
; N: Number of observations
; r: Correlation coefficient
; Pr: Probaility that random noise could produce the result
(correlation) with N samples
; Pr=ERFC(r*sqrt(N/2))
; ERFC: Complementary Error Function
; rsig: At which we have 100*(1-limit) chance that random data would
produce this result (r)
; rsig=INVERF(limit)*sqrt(2/N)
; Interpretation:
; Any "r" value greater than "rsig" are significant at "limit*100"
level
; Modification: Yaswant Pradhan 10/6/05
;-
PRO sig_test
r=0D
N=0D
Pr=0D
rsig=0D
i=0.99
input: read,'Input Correlation coefficint and Number of samples [r,
N]',r,N
if (r gt 1.0 or r lt -1.0) then begin
err=widget_message('r value should be between -1.0 and 1.0. Input
r, N again!',/Error)
goto, input
endif
r=abs(r)
Pr= ERFC(r*sqrt(N/2.))
while (inverf(i)*sqrt(2./N) gt r) do begin
i=i-0.01
rsig=inverf(i)*sqrt(2./N)
endwhile
print,'Correlation Significance Test Result:'
print,'====================================='
print,'Correlation coefficient: ',r
print,'Number of samples: ',N
print,'Confidence Limit: '+string(9b)+string(i*100,format='(i2.2)')
+'%'
print,'Probability (Pr): ',Pr
print,'====================================='
END
On Nov 23, 1:51 am, Lin Wang <wanglin1...@gmail.com> wrote:
> I found a code which can calculate the confidence level using pvalue
> method (see below). It works well, but usually I use the Student's t-
> test method. So can anyone help?
>
> Thanks!
>
> r=correlate(x,y)
> var=1/(n-3.0)
> zvalue = 0.5*alog((1+r)/(1-r))/sqrt(var)
> if (zvalue lt 0) then pvalue = 2*(gauss_pdf(zvalue)) $
> else pvalue = 2*(1-gauss_pdf(zvalue))
|
|
|
|
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
