| Re: detrend [message #71210] |
Sun, 06 June 2010 08:16 |
envi35@yahoo.ca
Messages: 48
Registered: March 2005
|
Member |
|
|
Thanks for the reply, Craig. I'll try to remove a linear trend first,
hopefully that is all I need to do with my data.
Jenny
On Jun 6, 1:19 am, Craig Markwardt <craig.markwa...@gmail.com> wrote:
> On Jun 5, 10:50 pm, Jenny <env...@yahoo.ca> wrote:
>
>> Dear All,
>
>> I'm looking for an IDL routine to apply detrend on time series
>> data. Any ideas?
>
> It really matters a lot what kind of detrending you want to do, and
> how you expect it to affect your data. There is a fine line between
> "detrending" your data, and removing the real signal. Basically any
> type of detrending will involve a filtering step and a removal step.
> The filtering step is key, because it determines how much signal is
> removed.
>
> For reasonably regularly sampled data where you are interested in the
> high frequency variations, you could do a high-pass filter,
> Y = ... original data ...
> Y_SMOOTH = smooth(Y,10,/edge) ;; Smoothed version (smoothed over 10
> samples)
> Y_DETREND = Y - Y_SMOOTH ;; High-pass version
>
> If you want to remove a linear trend, the filtering step could be a
> linear least squares fit,
> AB = LINFIT(X,Y) ;; Fit linear trend
> Y_DETREND = Y - AB[0] - AB[1]*X ;; Remove linear trend
> More sophisticated analysis would add error bars; or a higher order
> polynomial fit with POLY_FIT() if appropriate.
>
> Good luck,
> Craig
|
|
|
|
| Re: detrend [message #71212 is a reply to message #71210] |
Sat, 05 June 2010 22:19  |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
On Jun 5, 10:50 pm, Jenny <env...@yahoo.ca> wrote:
> Dear All,
>
> I'm looking for an IDL routine to apply detrend on time series
> data. Any ideas?
It really matters a lot what kind of detrending you want to do, and
how you expect it to affect your data. There is a fine line between
"detrending" your data, and removing the real signal. Basically any
type of detrending will involve a filtering step and a removal step.
The filtering step is key, because it determines how much signal is
removed.
For reasonably regularly sampled data where you are interested in the
high frequency variations, you could do a high-pass filter,
Y = ... original data ...
Y_SMOOTH = smooth(Y,10,/edge) ;; Smoothed version (smoothed over 10
samples)
Y_DETREND = Y - Y_SMOOTH ;; High-pass version
If you want to remove a linear trend, the filtering step could be a
linear least squares fit,
AB = LINFIT(X,Y) ;; Fit linear trend
Y_DETREND = Y - AB[0] - AB[1]*X ;; Remove linear trend
More sophisticated analysis would add error bars; or a higher order
polynomial fit with POLY_FIT() if appropriate.
Good luck,
Craig
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
