General Linear Test

Last updated
Save as PDF

Page ID: 231

$ \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $ $ \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} $$\newcommand{\id}{\mathrm{id}}$ $ \newcommand{\Span}{\mathrm{span}}$ $ \newcommand{\kernel}{\mathrm{null}\,}$ $ \newcommand{\range}{\mathrm{range}\,}$ $ \newcommand{\RealPart}{\mathrm{Re}}$ $ \newcommand{\ImaginaryPart}{\mathrm{Im}}$ $ \newcommand{\Argument}{\mathrm{Arg}}$ $ \newcommand{\norm}[1]{\| #1 \|}$ $ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $ \newcommand{\Span}{\mathrm{span}}$ $\newcommand{\id}{\mathrm{id}}$ $ \newcommand{\Span}{\mathrm{span}}$ $ \newcommand{\kernel}{\mathrm{null}\,}$ $ \newcommand{\range}{\mathrm{range}\,}$ $ \newcommand{\RealPart}{\mathrm{Re}}$ $ \newcommand{\ImaginaryPart}{\mathrm{Im}}$ $ \newcommand{\Argument}{\mathrm{Arg}}$ $ \newcommand{\norm}[1]{\| #1 \|}$ $ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $ \newcommand{\Span}{\mathrm{span}}$$\newcommand{\AA}{\unicode[.8,0]{x212B}}$

We are interested in testing for the dependence on the predictor variable from a different viewpoint. We call the model

$Y_i = \beta_0+\beta_1X_i+\epsilon_i$

the full model. We want to test $H_0 : \beta_1 = 0$ against $H_1 : \beta_1 \neq 0$. Under $H_0 : \beta_1 = 0$, we have the reduced model:

$Y_i = \beta_0 + \epsilon_i$

Under the full model,

\[SSE_{full} = \sum_i(Y_i - \hat{Y_i})^2 = SSE .\]

Under the reduced model

\[SSE_{red} = \sum_i(Y_i - \hat{Y_i})^2 = SSTO\]

General Structure of Test Statistic

Observe that d.f.$(SSE_{full}) = n-2$, d.f.$(SSE_{red}) = n-1$ and $SSE_{red} - SSE_{full} = SSR$.

$$F^\ast =\dfrac{\dfrac{SSE_{red}-SSE_{full}}{d.f.(SSE_{red})-d.f.(SSE_{full})}}{\dfrac{SSE_{full}}{d.f.(SSE_{full})}} = \dfrac{\dfrac{SSR}{d.f.(SSR)}}{\dfrac{SSE}{d.f.(SSE)}}=\dfrac{MSR}{MSE}$$

Under normal error model, and under $H_0 : \beta_1 = 0, F^\ast$ has the $F$ distribution with (paired) degress of freedom $(d.f.(SSE_{red}) - d.f.(SSE_{full}), d.f.(SSE_{full}))$.

Descriptive Measure of Association Between $X$ and $Y$

Define the coefficient of determination:

$$R^2 = \dfrac{SSR}{SSTO}= 1-\dfrac{SSE}{SSTO}$$

Observe that $0\leq R^2\leq 1$, and the correlation coefficient, Corr$(X, Y)$ between $X$ and $Y$ is the (signed) square root of $R^2$. That is (Corr$(X, Y))^2 = R^2$. Larger value of $R^2$ generally indicates higher degree of linear association between $X$ and $Y$. Another (and considered better) measure of association is the adjusted coefficient of determination:

$$R^2_{ad} =1- \dfrac{MSE}{MSTO}$$

$R^2$ is the proportion of variability in $Y$ explained by its regression on $X$. Also, $R^2$ is unit free, i.e. does not depend on the units of measurements of the variables $X$ and $Y$.

For the housing price data, $SSR = 352.91, SSTO = 556.08, n = 19$, and hence $SSE = 203.17$, $d.f.(SSE) = 17$, $d.f.(SSTO) = 18$. So, $R^2 = \dfrac{352.91}{556.08} = 0.635$ and $R^2_{ad} = 1-\dfrac{11.95}{30.8} = 0.613$.

Contributors

Cathy Wang

General Structure of Test Statistic

Descriptive Measure of Association Between \(X\) and \(Y\)

Contributors