| Re: Fourier analysis of the Data with some gaps [message #66027] |
Tue, 14 April 2009 09:39  |
R.G. Stockwell
Messages: 363
Registered: July 1999
|
Senior Member |
|
|
<duxiyu@gmail.com> wrote in message
news:55d7a5d9-0fc3-462d-a680-3be22f337271@k19g2000prh.google groups.com...
Thank you for your explanation.
I am very intertesting in the local spectral technique you mentioned.
Could you recommend some references about it?
Best regards,
jdu
**********************
There is a lot. A straight forward Short Time Fourier Transform is
simple and easy to understand. Simply parse the time series into small
overlapping regions and FFT it.
There are wigner type transforms, that in my opinion are very useful
for real data with noise.
There are wavelets (two disctinct branches, you'd want the CWT with morlet
wavelets or perhaps mexican hat wavelets if you are interested in local
spectra.
Orthogonal wavelets are not directly related to the fourier spectrum)
And there is the S-transform, which is combination of STFT and Wavelets,
and has several advantageous characteristics.
http://www.cora.nwra.com/stransform/
cheers,
bob
PS disclaimer, the ST stuff is mine, so I may be a bit biased when I
say it is superior to all other methods for the analysis of transient
quasimonochromatic
signals in geophysical data. :)
Source code is available on that website.
|
|
|
|
| Re: Fourier analysis of the Data with some gaps [message #66036 is a reply to message #66027] |
Mon, 13 April 2009 23:37  |
duxiyu@gmail.com
Messages: 88
Registered: March 2007
|
Member |
|
|
Thank you for your explanation.
I am very intertesting in the local spectral technique you mentioned.
Could you recommend some references about it?
Best regards,
jdu
On Apr 14, 12:10 am, "R.G. Stockwell" <noemai...@please.com> wrote:
> "Kenneth P. Bowman" <k-bow...@null.edu> wrote in messagenews:k-bowman-5F7B9B.10442113042009@news.tamu.edu...
>
>
>
>> In article
>> < 7cfa22f4-1133-48a1-9562-a84a0d932...@i28g2000prd.googlegroup s.com >,
>> "dux...@gmail.com" <dux...@gmail.com> wrote:
>
>>> Dear all,
>
>>> I want to take FFT on the data.
>>> But there are some shorts data gaps during this data interval.
>>> How should I deal with these gaps?
>
>>> Best regards,
>
>>> jdu
>
>> This is a very general question and there is no unique answer. You
>> need to be aware of the characteristics of the data.
>
> I agree, and would go a bit further. There is no unique answer, and
> no good answer.
>
>> You can interpolate to fill the gaps. (Many methods.)
>
> This is fine if the gaps are not too common, or too large.
>
>> You can use least-squares instead of FFT.
>
> The Lomb Scargle technique is often misused in this case.
> It does a fit of a _single_ sinusoid, and calculates the significance
> of it. It should not be used to calculate the spectrum (which of course
> is exactly what the Numerical Recipe book does).
> An actual least squares fit to all the fourier components, where there
> is gappy data, is almost always an ill posed matrix. The sinusoids are
> orthogonal
> with regular sampling, but when you remove a point in the time series, those
> sinusoids are no longer orthogonal.
>
> Perhaps a local spectral technique would be appropriate, which gives
> one the spectrum where there is data, and gaps where there are gaps.
>
> Cheers,
> bob
|
|
|
|
| Re: Fourier analysis of the Data with some gaps [message #66044 is a reply to message #66036] |
Mon, 13 April 2009 09:10 |
R.G. Stockwell
Messages: 363
Registered: July 1999
|
Senior Member |
|
|
"Kenneth P. Bowman" <k-bowman@null.edu> wrote in message
news:k-bowman-5F7B9B.10442113042009@news.tamu.edu...
> In article
> <7cfa22f4-1133-48a1-9562-a84a0d932a6e@i28g2000prd.googlegroups.com>,
> "duxiyu@gmail.com" <duxiyu@gmail.com> wrote:
>
>> Dear all,
>>
>> I want to take FFT on the data.
>> But there are some shorts data gaps during this data interval.
>> How should I deal with these gaps?
>>
>> Best regards,
>>
>> jdu
>
> This is a very general question and there is no unique answer. You
> need to be aware of the characteristics of the data.
I agree, and would go a bit further. There is no unique answer, and
no good answer.
> You can interpolate to fill the gaps. (Many methods.)
This is fine if the gaps are not too common, or too large.
> You can use least-squares instead of FFT.
The Lomb Scargle technique is often misused in this case.
It does a fit of a _single_ sinusoid, and calculates the significance
of it. It should not be used to calculate the spectrum (which of course
is exactly what the Numerical Recipe book does).
An actual least squares fit to all the fourier components, where there
is gappy data, is almost always an ill posed matrix. The sinusoids are
orthogonal
with regular sampling, but when you remove a point in the time series, those
sinusoids are no longer orthogonal.
Perhaps a local spectral technique would be appropriate, which gives
one the spectrum where there is data, and gaps where there are gaps.
Cheers,
bob
|
|
|
|
|
|
| Re: Fourier analysis of the Data with some gaps [message #66168 is a reply to message #66027] |
Wed, 15 April 2009 00:36 |
duxiyu@gmail.com
Messages: 88
Registered: March 2007
|
Member |
|
|
Thank you very much.
It is very helpful.
Best wishes,
jdu
On Apr 15, 12:39 am, "R.G. Stockwell" <noemai...@please.com> wrote:
> <dux...@gmail.com> wrote in message
>
> news:55d7a5d9-0fc3-462d-a680-3be22f337271@k19g2000prh.google groups.com...
> Thank you for your explanation.
> I am very intertesting in the local spectral technique you mentioned.
> Could you recommend some references about it?
>
> Best regards,
> jdu
> **********************
>
> There is a lot. A straight forward Short Time Fourier Transform is
> simple and easy to understand. Simply parse the time series into small
> overlapping regions and FFT it.
>
> There are wigner type transforms, that in my opinion are very useful
> for real data with noise.
>
> There are wavelets (two disctinct branches, you'd want the CWT with morlet
> wavelets or perhaps mexican hat wavelets if you are interested in local
> spectra.
> Orthogonal wavelets are not directly related to the fourier spectrum)
>
> And there is the S-transform, which is combination of STFT and Wavelets,
> and has several advantageous characteristics.
>
> http://www.cora.nwra.com/stransform/
>
> cheers,
> bob
>
> PS disclaimer, the ST stuff is mine, so I may be a bit biased when I
> say it is superior to all other methods for the analysis of transient
> quasimonochromatic
> signals in geophysical data. :)
> Source code is available on that website.
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
