3.2: First Results

So, the previous section laid out the mathematical results based on a rather minor requirement. As mathematicians, the next usual step is to explore to find out what more we can learn. Sometimes, the exploration leads to dead ends or trivial results. However, what keeps the mathematician excited for the material is that they will frequently learn something quite interesting.

In this section, let's see what consequences arise from our assumption.

✦•················• ✦ •··················•✦

Now that we have formulas for our estimators, we have a few mathematical results. The first result is that the OLS line of best fit passes through the center of gravity.

Theorem \(\PageIndex{1}\): The Center of Gravity

The point \((\bar{x},\ \bar{y})\), the center of gravity, is on the OLS line of best fit.

Proof:
To see this just substitute \(\bar{x}\) for \(x\) in the prediction equation and show that \(\hat{y} = \bar{y}\).

From the formula for the OLS estimator of \(\beta_0\) given in Section 3.1, we have

\begin{align}
\hat{y} &= b_0 + b_1 x \\[1em]
&= b_0 + b_1 \bar{x} \\[1em]
&= \left( \bar{y} - b_1 \bar{x} \right) + b_1 \bar{x} \\[1em]
&= \bar{y}
\end{align}

Thus, we have shown that \(\hat{y} = \bar{y}\) when \(x = \bar{x}\). In other words, we showed that the OLS line of best fit passes through the center of gravity.

∎

Note

You should also be able to prove that any line passing through the center of gravity has the sum of the residuals being zero.

Example \(\PageIndex{1}\)

Illustrate this result using the previous example. In other words, show that the point \((\bar{x},\ \bar{y}) =(0.5,\ 2)\) is on the line.

Solution:
In the previous section, we already showed that the line of best fit is \( \hat{y} = 1.5 + x \). Substituting \( \bar{x}=0.5 \) gives \( \hat{y} = 1.5 + 0.5 = 2\). Note that \(2\) is also the value of \(\bar{y}\).

Thus, we have illustrated Theorem \(\PageIndex{1}\).

───── ⋆⋅☆⋅⋆ ─────

A second result is that the slope estimator \(b_1\) is the ratio of the covariance between \(x\) and \(y\) to the variance of \(x\).

Theorem \(\PageIndex{2}\)

An equivalent formula is
\begin{equation}b_1 = \frac{Cov[x,y]}{V[x]} = \frac{s_{xy}}{s_x^2}\end{equation}

Proof:
To see this we substitute the formulas for the covariance and variance into this equation and quickly simplify:

\begin{align}
b_1 &= \frac{\displaystyle s_{xy}}{\displaystyle s_x^2} \\[1em]
&= \frac{\displaystyle \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\displaystyle \frac{1}{n-1}\sum_{i=1}^n (x_i - \bar{x})^2} \\[1em]
&= \frac{\displaystyle \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{\displaystyle \sum_{i=1}^n (x_i - \bar{x})^2}
\end{align}

\(\blacksquare\)

Two additional results are

Theorem \(\PageIndex{3}\)

The slope estimator can also be represented as
\begin{equation}b_1 = r_{xy} \frac{s_y}{s_x}\end{equation}

In other words, the slope estimator is the correlation between the two variables times the ratio of their standard deviations.

Theorem \(\PageIndex{4}\)

The slope estimator is zero if the y-values do not vary.

I leave these last two theorems as exercises.