| t-testing in IDL [message #58240] |
Wed, 23 January 2008 07:52  |
mariamalene
Messages: 2
Registered: January 2008
|
Junior Member |
|
|
I have two sample data sets with 20 years of 12 months simulations in
a lon/lat grid of 128 x 64. , i.e. 2x(20,12,128,64). I would like to
find the difference between the two simulations and find out which
differences that are significant at a 95% level. I have calculated the
annual average of my 20-year simulation, and also the standard-
deviation, but how do I perfom the t-test? I have looked at the
tm_test but haven't figured out how to apply it on my 4-D approach..
Thanks
|
|
|
|
|
|
| Re: t-testing in IDL [message #58300 is a reply to message #58240] |
Thu, 24 January 2008 08:03  |
Brian Larsen
Messages: 270
Registered: June 2006
|
Senior Member |
|
|
> If you are going to do stats, I'd suggest you use a statistics
> package, such as R (www.r-project.org) instead of IDL. For
> example, if a t-test is to be useful, your data must be normally
> distributed. That is very easy to test with R, and relatively
> easy to test with IDL, but once you get beyond very basic
> statistics, you'll get beyond IDL's build-in capabilities. There
> are relatively easy to use mixed effects modeling tools in R.
I agree with all of this but my issue always is that as soon as I
start mixing tools (IDL and R) things start to get confusing as well
as I seem to spend all my time doing file io instead of analysis. To
this end I started making some stats routines in IDL, some are general
purpose and some are really specific.
For this example you want to know if the data is normally distributed,
for this I like normal probability plots and the Blom Position test,
both of which I implemented in IDL (and would love fixes, feedback,
etc). There are other IDL stats resources on the web too, it would be
good to pull those together into an IDLstats package...
See:
http://people.bu.edu/balarsen/IDLdoc/
blom_position_test.pro
normal_prob_plot.pro
Cheers,
Brian
------------------------------------------------------------ --------------
Brian Larsen
Boston University
Center for Space Physics
|
|
|
|
| Re: t-testing in IDL [message #58301 is a reply to message #58240] |
Thu, 24 January 2008 07:31 |
mariamalene
Messages: 2
Registered: January 2008
|
Junior Member |
|
|
On Jan 24, 3:52 pm, mmill...@iupui.edu (Michael A. Miller) wrote:
>>>> >> "mariamalene" == mariamalene <mariamal...@hotmail.com> writes:
>
>> I have two sample data sets with 20 years of 12 months
>> simulations in a lon/lat grid of 128 x 64. ,
>> i.e. 2x(20,12,128,64). I would like to find the difference
>> between the two simulations and find out which differences
>> that are significant at a 95% level. I have calculated the
>> annual average of my 20-year simulation, and also the
>> standard- deviation, but how do I perfom the t-test? I have
>> looked at the tm_test but haven't figured out how to apply
>> it on my 4-D approach..
>
> I'm not entirely clear on what you want to do, but I'll take a
> shot.
>
> If you have two data sets, A and B, and you want to test the
> hypothesis that the means are the same, you can use tm_test as in
> tm_test(A,B). It the data sets are paired, meaning A[i] is
> correlated with B[i], you can use tm_test(A,B,/paired) to test
> the hypothesis that the mean difference is zero. I suspect that
> this might be what you want. If you want to test the hypothesis
> that the mean difference is zero and that the difference does not
> have a spatial dependence, you could use a mixed effect model.
>
> If you are going to do stats, I'd suggest you use a statistics
> package, such as R (www.r-project.org) instead of IDL. For
> example, if a t-test is to be useful, your data must be normally
> distributed. That is very easy to test with R, and relatively
> easy to test with IDL, but once you get beyond very basic
> statistics, you'll get beyond IDL's build-in capabilities. There
> are relatively easy to use mixed effects modeling tools in R.
>
> Mike
>
> P.S. For the record: this posting should not be taken as an
> opinion against IDL, which I love/hate and use constantly. It's
> just that R, which I love/hate and use constantly, has a
> different set of strenghts and weaknesses. Give me grep, awk,
> python, IDL and R and an emacs to run it all in and I can shoot
> myself in the other foot every time!
>
> P.P.S. Getting even more off topic: this list needs to be
> updated:http://burks.bton.ac.uk/burks/language/shoot.htm
You are guessing right about my reasons for doing this. I'll check out
R, although I hope I don't have to go further into statistics than
this! Thank you.
|
|
|
|
| Re: t-testing in IDL [message #58303 is a reply to message #58240] |
Thu, 24 January 2008 06:52 |
mmiller3
Messages: 81
Registered: January 2002
|
Member |
|
|
>>>> > "mariamalene" == mariamalene <mariamalene@hotmail.com> writes:
> I have two sample data sets with 20 years of 12 months
> simulations in a lon/lat grid of 128 x 64. ,
> i.e. 2x(20,12,128,64). I would like to find the difference
> between the two simulations and find out which differences
> that are significant at a 95% level. I have calculated the
> annual average of my 20-year simulation, and also the
> standard- deviation, but how do I perfom the t-test? I have
> looked at the tm_test but haven't figured out how to apply
> it on my 4-D approach..
I'm not entirely clear on what you want to do, but I'll take a
shot.
If you have two data sets, A and B, and you want to test the
hypothesis that the means are the same, you can use tm_test as in
tm_test(A,B). It the data sets are paired, meaning A[i] is
correlated with B[i], you can use tm_test(A,B,/paired) to test
the hypothesis that the mean difference is zero. I suspect that
this might be what you want. If you want to test the hypothesis
that the mean difference is zero and that the difference does not
have a spatial dependence, you could use a mixed effect model.
If you are going to do stats, I'd suggest you use a statistics
package, such as R (www.r-project.org) instead of IDL. For
example, if a t-test is to be useful, your data must be normally
distributed. That is very easy to test with R, and relatively
easy to test with IDL, but once you get beyond very basic
statistics, you'll get beyond IDL's build-in capabilities. There
are relatively easy to use mixed effects modeling tools in R.
Mike
P.S. For the record: this posting should not be taken as an
opinion against IDL, which I love/hate and use constantly. It's
just that R, which I love/hate and use constantly, has a
different set of strenghts and weaknesses. Give me grep, awk,
python, IDL and R and an emacs to run it all in and I can shoot
myself in the other foot every time!
P.P.S. Getting even more off topic: this list needs to be
updated: http://burks.bton.ac.uk/burks/language/shoot.htm
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
