|
|
|
|
| Re: correlation coefficient and confidence level [message #87501 is a reply to message #87500] |
Sun, 09 February 2014 17:44   |
gunvicsin11
Messages: 93
Registered: November 2012
|
Member |
|
|
Hello sir,
I tried idl routines correlate and coyote's cgscatter2d, but both doesnt give, confidence level for the correlation coefficient.
Then I tried r_correlate, which gives significance level, but I guess significance level is different from confidence level.
So I have no idea how to do this.
thanking you
sid
On Monday, February 10, 2014 6:38:17 AM UTC+5:30, David Fanning wrote:
> sid writes:
>
>
>
>> Is it possible to find confidence level for the correlation coefficient. Please let me know if you have any idea.
>
>
>
> I have an idea. What have you tried that didn't work?
>
>
>
> Cheers,
>
>
>
> David
>
>
>
>
>
>
>
> --
>
> David Fanning, Ph.D.
>
> Fanning Software Consulting, Inc.
>
> Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
>
> Sepore ma de ni thue. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: correlation coefficient and confidence level [message #87503 is a reply to message #87501] |
Sun, 09 February 2014 18:35 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
sid writes:
> I tried idl routines correlate and coyote's cgscatter2d, but both doesnt give, confidence level for the correlation coefficient.
> Then I tried r_correlate, which gives significance level, but I guess significance level is different from confidence level.
> So I have no idea how to do this.
The first article that appears when I use Google to search "confidence
correlation coefficient" is this article:
http://www2.sas.com/proceedings/sugi31/170-31.pdf
Low and behold, there is code in there that calculates the confidence
level of the correlation coefficient! Lucky! But, you know, sometimes it
happens. :-)
Simply translating the SAS code to IDL (I'm pretty sure you could have
done this, if I managed it), I get this:
n = 101
data_1 = cgDemoData(1)+ RandomU(-3L, n) * 10
data_2 = cgDemoData(1)+ RandomU(-5L, n) * 10
cgScatter2D, data_1, data_2
rho = Correlate(data_1, data_2)
fisherTransform = 0.5*(alog(1+rho) - alog(1-rho))
sigmaz = 1.0 / Sqrt(n-3)
l_95 = fisherTransform - (1.96 * sigmaz)
h_95 = fisherTransform + (1.96 * sigmaz)
lo95 = (exp(2*l_95)-1)/(exp(2*l_95)+1)
hi95 = (exp(2*h_95)-1)/(exp(2*h_95)+1)
Print, 'Low 95% Confidence Level: ', lo95
Print, 'High 95% Confidence Level: ', hi95
END
For a sanity check, I used another page from my Google search to find a
confidence level calculator. I plugged my numbers in and they matched
more or less:
http://vassarstats.net/rho.html?
I spent about 45 minutes on this, if you feel like you want to make a
contribution to the work.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: correlation coefficient and confidence level [message #87506 is a reply to message #87503] |
Sun, 09 February 2014 22:09 |
gunvicsin11
Messages: 93
Registered: November 2012
|
Member |
|
|
On Monday, February 10, 2014 8:05:57 AM UTC+5:30, David Fanning wrote:
> sid writes:
>
>
>
>> I tried idl routines correlate and coyote's cgscatter2d, but both doesnt give, confidence level for the correlation coefficient.
>
>> Then I tried r_correlate, which gives significance level, but I guess significance level is different from confidence level.
>
>> So I have no idea how to do this.
>
>
>
> The first article that appears when I use Google to search "confidence
>
> correlation coefficient" is this article:
>
>
>
> http://www2.sas.com/proceedings/sugi31/170-31.pdf
>
>
>
> Low and behold, there is code in there that calculates the confidence
>
> level of the correlation coefficient! Lucky! But, you know, sometimes it
>
> happens. :-)
>
>
>
> Simply translating the SAS code to IDL (I'm pretty sure you could have
>
> done this, if I managed it), I get this:
>
>
>
> n = 101
>
> data_1 = cgDemoData(1)+ RandomU(-3L, n) * 10
>
> data_2 = cgDemoData(1)+ RandomU(-5L, n) * 10
>
> cgScatter2D, data_1, data_2
>
>
>
> rho = Correlate(data_1, data_2)
>
> fisherTransform = 0.5*(alog(1+rho) - alog(1-rho))
>
> sigmaz = 1.0 / Sqrt(n-3)
>
> l_95 = fisherTransform - (1.96 * sigmaz)
>
> h_95 = fisherTransform + (1.96 * sigmaz)
>
>
>
> lo95 = (exp(2*l_95)-1)/(exp(2*l_95)+1)
>
> hi95 = (exp(2*h_95)-1)/(exp(2*h_95)+1)
>
> Print, 'Low 95% Confidence Level: ', lo95
>
> Print, 'High 95% Confidence Level: ', hi95
>
> END
>
>
>
> For a sanity check, I used another page from my Google search to find a
>
> confidence level calculator. I plugged my numbers in and they matched
>
> more or less:
>
>
>
>
>
>
>
> I spent about 45 minutes on this, if you feel like you want to make a
>
> contribution to the work.
>
>
>
> Cheers,
>
>
>
> David
>
> --
>
> David Fanning, Ph.D.
>
> Fanning Software Consulting, Inc.
>
> Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
>
> Sepore ma de ni thue. ("Perhaps thou speakest truth.")
Respected sir,
I have tried the website "http://vassarstats.net/rho.html?" and the one above also. But what I want to do is to get the confidence level of how well the two data sets are correlating.
How much % confidence level the correlation coefficient indicates.
thanking you
sid
|
|
|
|
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
