| Re: Time series with 75% missing observations [message #27344] |
Thu, 18 October 2001 09:00  |
Joe Means
Messages: 44
Registered: November 1996
|
Member |
|
|
Thanks much Craig. This should work great!
Cheers,
Joe
Craig Markwardt wrote:
> Joe Means <joe.means@orst.edu> writes:
>
>> I have long data series with many random, missing values. These series
>> each have only one or two frequencies. As I read the IDL 5.4
>> documentation, time series analysis routines require all values to be
>> evenly spaced. The periodicities are not sinusoidal, but such an
>> assumption might well find them. How can I find these periodicities?
>
>
> Greetings!
>
> If your time series are regularly sampled, but have missing values,
> then it is relatively straightforward to proceed with the FFT.
>
> Theoretically, zeroes do not actually contribute to the FFT power, but
> in practice you get aliases of the DC power which contaminate all the
> other frequencies. Contamination is bad.
>
> The solution is to subtract off the mean value of the signal *from the
> non-missing values* before doing the FFT. Say you have a variable Y
> which contains the signal, and missing values are set to -1. First
> you would transform to a new variable, YP, which has the average value
> subtracted.
>
> yp = y*0
> wh = where(y NE -1, ct)
> if ct EQ 0 then message, 'ERROR: no valid points!'
> yp(wh) = y(wh) - avg(y(wh))
>
> Note that the original missing values are converted to zeroes, so
> overall YP should have a zero mean value itself. Then you just do
> your FFT as normal.
>
> ***
>
> The other possibility, is if you have irregularly sampled points.
> Then you are better off with something like the Lomb Scargle
> periodogram. It's in Numerical Recipes. I have a crude routine which
> does this, available by request.
>
> Craig
>
>
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
