# 4.4: Linear Filtering

$$\newcommand{\vecs}{\overset { \rightharpoonup} {\mathbf{#1}} }$$ $$\newcommand{\vecd}{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$$$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}{\| #1 \|}$$ $$\newcommand{\inner}{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}{\| #1 \|}$$ $$\newcommand{\inner}{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$$$\newcommand{\AA}{\unicode[.8,0]{x212B}}$$

A linear filter uses specific coefficients $$(\psi_s\colon s\in\mathbb{Z})$$, called the impulse response function, to transform a weakly stationary input series $$(X_t\colon t\in\mathbb{Z})$$ into an output series $$(Y_t\colon t\in\mathbb{Z})$$ via

$Y_t=\sum_{s=-\infty}^\infty\psi_sX_{t-s}, \qquad t\in\mathbb{Z}, \nonumber$

where $$\sum_{s=-\infty}^\infty|\psi_s|<\infty$$. Then, the frequency response function

$\Psi(\omega)=\sum_{s=-\infty}^\infty\psi_s\exp(-2\pi i\omega s) \nonumber$

is well defined. Note that the two-point moving average of Example 4.2.2 and the differenced sequence $$\nabla X_t$$ are examples of linear filters. On the other hand, any \/ causal ARMA process can be identified as a linear filter applied to a white noise sequence. Implicitly this concept was already used to compute the spectral densities in Exampels 4.2.2 and 4.2.3. To investigate this in further detail, let $$\gamma_X(h)$$ and $$\gamma_Y(h)$$ denote the ACVF of the input process $$(X_t\colon t\in\mathbb{Z})$$ and the output process $$(Y_t\colon t\in\mathbb{Z})$$, respectively, and denote by $$f_X(\omega)$$ and $$f_Y(\omega)$$ the corresponding spectral densities. The following is the main result in this section.

Theorem 4.4.1.

Under the assumptions made in this section, it holds that $$f_Y(\omega)=|\Psi(\omega)|^2f_X(\omega)$$.

Proof. First note that

$$\gamma_Y(h)=E\big[(Y_{t+h}-\mu_Y)(Y_t-\mu_Y)]\$.2cm]$$ $$=\sum_{r=-\infty}^\infty\sum_{s=-\infty}^\infty\psi_r\psi_s\gamma(h-r+s)\\[.2cm]$$ $$=\sum_{r=-\infty}^\infty\sum_{s=-\infty}^\infty\psi_r\psi_s\int_{-1/2}^{1/2}\exp(2\pi i\omega(h-r+s))f_X(\omega)d\omega\\[.2cm]$$ $$=\int_{-1/2}^{1/2}\Big(\sum_{r=-\infty}^\infty\psi_r\exp(-2\pi i\omega r)\Big)\Big(\sum_{s=-\infty}^\infty\psi_s\exp(2\pi i\omega s)\Big)\exp(2\pi i\omega h)f_X(\omega)d\omega\\[.2cm]$$ $$=\int_{-1/2}^{1/2}\exp(2\pi i\omega h)|\Psi(\omega)|^2f_X(\omega)d\omega.$$ Now identify $$f_Y(\omega)=|\Psi(\omega)|^2f_X(\omega)$$, which is the assertion of the theorem. Theorem 4.4.1 suggests a way to compute the spectral density of a causal ARMA process. To this end, let $$(Y_t\colon t\in\mathbb{Z})$$ be such a causal ARMA(p,q) process satisfying $$Y_t=\psi(B)Z_t$$, where $$(Z_t\colon t\in\mathbb{Z})\sim\mbox{WN}(0,\sigma^2)$$ and \[ \psi(z)=\dfrac{\theta(z)}{\phi(z)}=\sum_{s=0}^\infty\psi_sz^s,\qquad |z|\leq 1. \nonumber$

with $$\theta(z)$$ and $$\phi(z)$$ being the moving average and autoregressive polynomial, respectively. Note that the $$(\psi_s\colon s\in\mathbb{N}_0)$$ can be viewed as a special impulse response function.

Corollary 4.4.1.

If $$(Y_t\colon t\in\mathbb{Z})$$ be a causal ARMA(p,q)\) process. Then, its spectral density is given by

$f_Y(\omega)=\sigma^2\dfrac{|\theta(e^{-2\pi i\omega})|^2}{|\phi(e^{-2\pi i\omega})|^2}. \nonumber$

Proof. Apply Theorem 4.4.1 with input sequence $$(Z_t\colon t\in\mathbb{Z})$$. Then $$f_Z(\omega)=\sigma^2$$, and moreover the frequency response function is

$\Psi(\omega)=\sum_{s=0}^\infty\psi_s\exp(-2\pi i\omega s)=\psi(e^{-2\pi i\omega})=\dfrac{\theta(e^{-2\pi i\omega})}{\phi(e^{-2\pi i\omega})}. \nonumber$

Since $$f_Y(\omega)=|\Psi(\omega)|^2f_X(\omega)$$, the proof is complete.

Corollary 4.4.1 gives an easy approach to define parametric spectral density estimates for causal ARMA(p,q) processes by simply replacing the population quantities by appropriate sample counterparts. This gives the spectral density estimator

$\hat f(\omega)=\hat\sigma_n^2\dfrac{|\hat\theta(e^{-2\pi i\omega})|^2}{|\hat\phi(e^{-2\pi i\omega})|^2}. \nonumber$

Now any of the estimation techniques discussed in Section 3.5 may be applied when computing $$\hat f(\omega)$$.

## Contributers

This page titled 4.4: Linear Filtering is shared under a not declared license and was authored, remixed, and/or curated by Alexander Aue.