| Re: error estimates (a little off-topic maybe) [message #32397] |
Fri, 04 October 2002 03:22 |
wmconnolley
Messages: 106
Registered: November 2000
|
Senior Member |
|
|
Joern Wilms <wilms@astro.uni-tuebingen.de> wrote:
> Also, note that if you're analyzing a time series, then the uncertainty of
> the power-spectrum is well known through its chi^2 properties (the
> power spectrum is the square of the fourier transform of a time series, and
> it is easy to show that its uncertainty is a chi^2 distribution with 2
> degrees of freedom, i.e., the uncertainty of each value of the power spectrum
> is as large as the value itself).
You can reduce the errors by averaging, of course.
-W
ps: the book I had in mind is "spectral analysis for physical applications",
percival_db and walden_at. But its not a quick fix.
--
William M Connolley | wmc@bas.ac.uk | http://www.nerc-bas.ac.uk/icd/wmc/
Climate Modeller, British Antarctic Survey | Disclaimer: I speak for myself
I'm a .signature virus! copy me into your .signature file & help me spread!
|
|
|
|
| Re: error estimates (a little off-topic maybe) [message #32403 is a reply to message #32397] |
Thu, 03 October 2002 13:14  |
wilms
Messages: 5
Registered: January 2001
|
Junior Member |
|
|
In article <Pine.LNX.4.44.0210031925160.1364-100000@umzog.ulo.ucl.ac.uk>,
src <src@umzog.ulo.ucl.ac.uk> wrote:
>
> This question might be a little off-topic, but since a lot of
> scientists/engineers use IDL, someone may have some ideas.
>
> What I'd like to do is measure the amplitude (and estimate the error) of a
> peak in a Fourier transform. Now I could fit the peak with a Gaussian
> function, and if I use CurveFit, it will give me an error estimate too.
> The problem is, the error estimate is dependent on on the number of points
> fitted, as one might expect with such routines. Since I wish to perform a
> hypothesis test later, I could get the outcome I want just be increasing
> the number of frequency points in the Fourier transform, so that the error
> is small and my hypothesis is accepted.
I am not entirely sure what you mean by increasing the number of
frequency points. If you're using a Fourier transform and have
equally spaced numbers, then the number of frequency points is fixed
by the number of Fourier frequencies and it is quite difficult to
simply increase the number of points...
Also, note that if you're analyzing a time series, then the uncertainty of
the power-spectrum is well known through its chi^2 properties (the
power spectrum is the square of the fourier transform of a time series, and
it is easy to show that its uncertainty is a chi^2 distribution with 2
degrees of freedom, i.e., the uncertainty of each value of the power spectrum
is as large as the value itself). Similar arguments also apply to any
fourier transform of real measured data, which I assume you're performing.
Thus, although you might not be aware of it, you most probably can compute
real error bars from your data, and should do so :-).
Unrelated to this: please note that the "errors" returned from curvefit
are normally NOT the formal errors of the fit parameters. This is only
true if the fit parameters you use in your fit are statistically independent
from each other (i.e. if the Hessian matrix at the minimum of teh chi^2
is diagonal). In general, this is not the case and you need to determine
the uncertainty of your fitting parameters from the shape of the chi^2
contour. A nice discussion of this can be found, e.g., in Bevington &
Robinson. Some routines to perform such an error determination either
for the case of one or two parameters (the latter for the case that
two parameters are correlated) can be found at
http://astro.uni-tuebingen.de/software/idl/aitlib/fitting/
(if I am allowed to enter this shameless plug for our software here,
that is ;-) ). These routines are to be used in conjunction with
Craig Marquardt's great IDL fitting routines, which I'd recommend
as a replacement of mpfitfun.
>
> Does anyone know of a way of estimating such an error which is not
> dependent on the number of points fitted?
nope, you will always need to assign an uncertainty to your points, and
there is no such thing as data without errors...
Cheers,
Joern
>
> cheers,
> S
>
--
Dr. Joern Wilms Linsenbergstr. 33
Univ. Tuebingen, Institute for Astronomy D-72074 Tuebingen
Sand 1 (phone) +49 7071 29-76128
D-72076 Tuebingen, Germany wilms@astro.uni-tuebingen.de
|
|
|
|
| Re: error estimates (a little off-topic maybe) [message #32404 is a reply to message #32403] |
Thu, 03 October 2002 13:16 |
wmconnolley
Messages: 106
Registered: November 2000
|
Senior Member |
|
|
src <src@umzog.ulo.ucl.ac.uk> wrote:
> What I'd like to do is measure the amplitude (and estimate the error) of a
> peak in a Fourier transform. Now I could fit the peak with a Gaussian
> function, and if I use CurveFit, it will give me an error estimate too.
> The problem is, the error estimate is dependent on on the number of points
> fitted, as one might expect with such routines. Since I wish to perform a
> hypothesis test later, I could get the outcome I want just be increasing
> the number of frequency points in the Fourier transform, so that the error
> is small and my hypothesis is accepted.
Well, I've always measured the height of my peaks by how high the peaks are
(I'm being serious here). It's not clear you're justified in fitting curves
through them (this is in datasets with, say, a few hundred points: perhaps
if you had thousands and lots of detail in the peaks it would be
different).
The error est you'd get from curvefit would bear no resemblance to the
"true" statistical error estimates you could do, I think. You can do
naive error ests by, say, cutting the data into 2 halves and repeating the
analysis. Or you can do the full multi-taper spectral analysis - I used to
be vaguely familiar with that. I could point you to a book, if interested.
-W.
--
William M Connolley | wmc@bas.ac.uk | http://www.nerc-bas.ac.uk/icd/wmc/
Climate Modeller, British Antarctic Survey | Disclaimer: I speak for myself
I'm a .signature virus! copy me into your .signature file & help me spread!
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
