13.4: Hypothesis Testing in Regression

Last updated
Save as PDF

Page ID: 7171

Foster et al.
University of Missouri-St. Louis, Rice University, & University of Houston, Downtown Campus via University of Missouri’s Affordable and Open Access Educational Resources Initiative

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

Regression, like all other analyses, will test a null hypothesis in our data. In regression, we are interested in predicting \(Y\) scores and explaining variance using a line, the slope of which is what allows us to get closer to our observed scores than the mean of \(Y\) can. Thus, our hypotheses concern the slope of the line, which is estimated in the prediction equation by \(b\). Specifically, we want to test that the slope is not zero:

\[\begin{array}{c}{\mathrm{H}_{0}: \text { There is no explanatory relation between our variables }} \\ {\mathrm{H}_{0}: \beta=0}\end{array} \nonumber \]

\[\begin{array}{c}{\mathrm{H}_{\mathrm{A}}: \text {There is an explanatory relation between our variables}} \\ {\mathrm{H}_{\mathrm{A}}: \beta>0} \\ {\mathrm{H}_{\mathrm{A}}: \beta<0} \\ {\mathrm{H}_{\mathrm{A}}: \beta \neq 0}\end{array} \nonumber \]

A non-zero slope indicates that we can explain values in \(Y\) based on \(X\) and therefore predict future values of \(Y\) based on \(X\). Our alternative hypotheses are analogous to those in correlation: positive relations have values above zero, negative relations have values below zero, and two-tailed tests are possible. Just like ANOVA, we will test the significance of this relation using the \(F\) statistic calculated in our ANOVA table compared to a critical value from the \(F\) distribution table. Let’s take a look at an example and regression in action.