14.1: The CLM and the GLM

This section serves as a bridge between the Classical Linear Model (CLM) and the more flexible framework of Generalized Linear Models (GLM). It begins by reviewing the core assumptions of the CLM — that the response variable is continuous, unbounded, and has constant variance — and acknowledges that real-world data often violate these conditions. While the previous chapters have demonstrated various patches and corrections (such as transformations, weighted least squares, and robust standard errors) to make the CLM work for non-ideal scenarios, this approach becomes increasingly cumbersome and ultimately inadequate for certain types of discrete outcomes like binary or count data. The section therefore introduces the GLM as a new, unified paradigm that relaxes these restrictive assumptions and provides a coherent statistical foundation for modeling a much wider range of response variable types.

 

Learning Objectives

By the end of this section, you will be able to:

1. Identify the limitations of the Classical Linear Model (CLM), particularly its assumption that the response variable is continuous and unbounded, which makes it poorly suited for discrete outcomes like binary or count data.
2. Recognize that while the CLM can be patched with various corrections (e.g., transformations, weighted least squares, robust standard errors) to handle some violations, these approaches become increasingly complicated and inadequate for certain types of dependent variables.
3. Explain that the Generalized Linear Model (GLM) provides a new, unified framework that relaxes the key assumptions of the CLM, allowing for the modeling of different types of response variables (including binary, count, and nominal data) while maintaining a coherent statistical theory.

 

✦•················• ✦ •··················•✦

 

The Classical Linear Model (CLM) assumes that the relationship between the dependent and the independent variables is linear and that the response variable can take on all possible values; i.e., \(\mathbf{Y} \in \mathbb{R}\). Furthermore, to come to statistical conclusions, least squares methods assume that the errors are normally distributed.

Unfortunately, we learned that not all relationships fit this model. Statisticians who realized this, modified the CLM to handle many different types of relationships. Thus, if the dependent variable is continuous and bounded, we modify the dependent variable. If there is heteroskedasticity in the model, we pre- and post-multiply the variance-covariance matrix to better approximate the true standard errors. If you need to weight the data based on some information (such as reliability), you multiply by the weight matrix. And so forth.

However, there are certain types of dependent variables that cannot be fit using this model (or fit optimally). These are the models with discrete dependent variables. If we want to hold on to the CLM paradigm, we will have to pretend such variables are continuous. Often, this assumption is not a good one. When variables are binary, continuous approximations result in predictions that do not reflect reality. When variables are counts, the variances are functions of the expected value and are heteroskedastic. When the dependent variable is nominal, there is little we can do using the classical linear model.

The Classical Linear Model can usually be altered to create good predictions. However, the further your variable is from being continuous and unbounded, the more corrections you will have to make, and the more complicated the process of estimation and prediction becomes — if even possible.

This chapter serves to bridge the gap between the classical linear model (CLM) and the generalized linear model (GLM). In this chapter, we will regenerate the results from the previous chapters, but use a different paradigm. This new paradigm will help us understand the assumptions underlying ordinary least squares regression. It will also serve as a basis for understanding the assumptions of this new modeling paradigm.