8.3: Adjustments for Heteroskedasticity

Linear regression assumes constant error variance, i.e., homoskedasticity. When this assumption is violated, as often happens with real-world data like electoral results, our standard errors become unreliable, leading to invalid hypothesis tests and confidence intervals. While transformations can sometimes remedy this, they may introduce other model problems.

This page re-introduces a powerful alternative: the Huber-White sandwich estimator. Instead of altering the model itself, this method directly adjusts the estimated standard errors to account for observed heteroskedasticity, providing "heteroskedasticity-consistent" inference. You will see the mathematical foundation of this adjustment and learn to implement it in R with a single command. Using the South Sudan referendum case, we'll compare original and adjusted results, highlighting how this correction can change — or affirm — your conclusions while leaving the original coefficient estimates unchanged. This technique is a crucial tool for robust statistical inference when model assumptions are violated.

✦•················• ✦ •··················•✦

The previous transformations also work well on fixing problems with heteroskedasticity and non-Normality. Unfortunately, if you perform an appropriate transformation to fix the problem with model fit, further transformations to fix heteroskedasticity may end up creating a new problem with model fit. Thus, it may happen that you cannot find a way of fixing the heteroskedasticity without breaking something else. In such cases, we can adjust the standard errors using a technique introduced by White in 1980.

Recall from ordinary least squares estimation that our estimator for \(\mathbf{B}\) is

\begin{equation}
\mathbf{b} = \left(\mathbf{X}^\prime \mathbf{X}\right)^{-1}\mathbf{X}^\prime \mathbf{Y}
\end{equation}

From this we showed that this estimator was unbiased; that is, \(E[\mathbf{b}] = \mathbf{B}\). We also calculated the variance of the estimator as

\begin{equation}
V[\mathbf{b}] = \sigma^2 \left( \mathbf{X}^\prime \mathbf{X}\right)^{-1}
\end{equation}

Homoskedasticity

That result, however, required that \(V[\mathbf{Y}] = \sigma^2 \mathbf{I}\). This is the assumption of homoskedasticity. Under heteroskedasticity, \(V[\mathbf{Y}]\) cannot be reduced. This leaves the variance of our OLS estimators as

\begin{equation}\label{eq:trans-sandwichRaw}
V[\mathbf{b}] = \left(\mathbf{X}^\prime \mathbf{X}\right)^{-1}\mathbf{X}^\prime\ \ V[\mathbf{Y}\ |\ \mathbf{X}]\ \ \mathbf{X}\left(\mathbf{X}^\prime \mathbf{X}\right)^{-1}
\end{equation}

So, to better estimate \(V[\mathbf{b}]\), we need to estimate \(V[\mathbf{Y} \ |\ \mathbf{X}]\) from the data. (Everything else in Equation \(\ref{eq:trans-sandwichRaw}\) is known.)

• How do we estimate \(V[\mathbf{Y} \ |\ \mathbf{X}]\) from the data?

We recall that \(V[\mathbf{Y} \ |\ \mathbf{X}] = V[\mathbf{E}]\). Thus, we can estimate \(V[\mathbf{Y}]\) from the residuals, specifically from how large each residual is. If the \(i^{\text{th}}\) residual is large, then the value of \(V[Y_i]\) will be large; if \(e_i\) is small, then \(V[Y_i]\) will be small.

Note

The Huber-White Sandwich Estimator makes adjustments for heteroskedasticity. These adjustments, however, are based on the data, which are random. Including this new source of randomness affects all confidence intervals and needs to be acknowledged.

It seems to me that it would be easier to create an entirely new estimation method than to just pile on adjustment after adjustment on OLS. This is, in fact, where we are headed in this book.

This was all covered in greater detail in Section 6.3: Homoskedasticity. I encourage you to review that section before continuing.

Having R Do This for Us

Instead of performing the above calculations by hand, we can have R do the adjustments for us. That helps with the accuracy and precision. The summaryHCE function in the KnoxStats package provides the adjustment and presents it in the form of our usual regression table.

Example \(\PageIndex{1}\): South Sudan Again

To illustrate the operation of the summaryHCE function, let us calculate the White-adjusted standard errors for the South Sudanese model of Section 8.2: The South Sudanese Referendum.

Solution.
We have already calculated the regression table for the logit model. From looking at the graphic, it seems as though there may be heteroskedasticity. There appears to be a lot more variation in the invalidation rate for smaller values of secession support than for larger values of secession support.

Running the following adjusts the standard errors to reflect the observed heteroskedasticity.

summaryHCE(model.xsd)

The heteroskedasticity-adjusted regression table is given below. Note that the estimates remain the same. That is because heteroskedasticity does not affect the estimates. The only changes are in the standard errors (and the test statistics and the p-values).

Original Regression Table

            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   1.8978     0.7690   2.468   0.0155 *  
p.ind        -9.3991     0.8287 -11.342   <2e-16 ***

Heteroskedasticity-Adjusted Regression Table

             Estimate HC Std. Error HC t value HC Pr(>|t|)
(Intercept)  1.897846     0.6704021   2.830907    0.004642
p.ind       -9.399121     0.7420048 -12.667196    0.000000

Notice that adjusting the standard errors is rather easy using R. It is just a single line. Also, notice that we did not model the heteroskedasticity, we merely adjusted for it.

Think About

At some level, it is unsettling to me to merely adjust for model weaknesses. It is a strong model that does not need fixes. Thus, if you can avoid using these Huber-White standard errors, I recommend it strongly. Heteroskedasticity is an important part of the data/model. It seems sinful to ignore it.