10.6: Conclusion

In the previous chapters, we established the ordinary least squares (OLS) framework as a foundational tool for linear modeling. Its power derives from a core set of assumptions, most notably that the model residuals are independent and identically distributed (iid). When this condition holds, OLS yields not only unbiased estimates but also the Best Linear Unbiased Estimators (BLUE), allowing for straightforward hypothesis testing and the construction of reliable confidence intervals from a single, elegant formula.

However, the reality of empirical data often violates this tidy assumption. Residuals are frequently neither identically distributed (exhibiting heteroskedasticity) nor independent (exhibiting serial or spatial correlation). While we previously introduced post-hoc adjustments, like the Huber-White sandwich estimator, to "fix" the standard errors after an OLS fit, this chapter has argued for a more principled approach: directly modeling the violation to gain efficiency and insight. Weighted Least Squares (WLS) embodies this philosophy for heteroskedasticity. Rather than treating unequal variance as a nuisance to be corrected, WLS formally incorporates it into the estimation procedure. By weighting each observation inversely to its variance, WLS gives less influence to noisier data points, yielding more precise estimates — provided we have a theoretical or empirical basis for specifying those variances. Without such knowledge, we are indeed relegated to the corrective measures discussed earlier.

Extending this logic to the problem of dependence, Generalized Least Squares (GLS) offers a comprehensive framework. When observations are correlated — as in time series, spatial data, or clustered samples — GLS explicitly models the structure of that dependence through a known covariance matrix. By transforming the data using the inverse square root of this matrix, GLS effectively "whitens" the errors, restoring the iid condition and allowing us to apply the inferential machinery of OLS to the transformed model. The critical caveat, and a key limitation, is that this powerful technique rests on the feasibility of that transformation; the covariance matrix must be invertible and its structure must be correctly specified. When these conditions are met, GLS moves us beyond merely patching violations to fundamentally improving our model by embracing the true, complex structure of our data.

Building upon this progression from patching violations to explicitly modeling them, we now turn to a method that challenges a more fundamental OLS assumption: that our primary interest lies in the conditional mean. The techniques of WLS and GLS refine our estimate of the mean relationship by accounting for variance and dependence, but they remain anchored to that central tendency. In the next chapter, we will explore quantile regression, a paradigm shift that liberates us from this focus. Rather than modeling how predictors influence the average outcome, quantile regression allows us to examine how they shape the entire conditional distribution — be it the median, the 90th percentile, or someplace in the tails. This is particularly powerful when heteroskedasticity is not just a violation to be corrected, but a core feature of the phenomenon, indicating that a variable's effect differs across the distribution of the outcome. Thus, our journey from OLS to WLS and GLS, which taught us to model the structure of errors, now leads us to a method that models the structure of the outcome itself, offering a richer, more complete portrait of the relationships within our data.