15.2: Latent Variable Modeling

This section introduces the concept of a latent variable as a solution to the challenges of modeling binary outcomes. Rather than attempting to predict the observed 0/1 outcome directly, the focus shifts to modeling an unobserved (latent) continuous variable: the probability that the event occurs. This probability is naturally bounded between 0 and 1, making it suitable for transformation via the logit function into an unbounded linear predictor. The section establishes the foundational equation for logistic regression, where the logit of the probability is modeled as a linear function of the predictors, and shows how predictions are made by first calculating the linear predictor and then applying the inverse logit (logistic) transformation to obtain an estimated probability.

 

Learning Objectives

By the end of this section, you will be able to:

1. Explain the concept of a latent variable in the context of binary outcomes, recognizing that we model the underlying probability of success (a continuous variable bounded by 0 and 1) rather than the observed 0/1 outcome itself.
2. State the fundamental logistic regression model as: \( \mathrm{logit}(\pi) = \mathbf{XB}\), where \(\pi\) is the probability of success, and the logit link function transforms this bounded probability into an unbounded linear predictor.
3. Describe the two-step process for making predictions from a logistic regression model: first calculate the value of the linear predictor (\(\pi = \mathbf{XB}\)), then apply the inverse logit (logistic) transformation to obtain an estimated probability: \(\pi = 1 / (1 + \mathrm{exp}(-\eta) )\).

 

✦•················• ✦ •··················•✦

 

In the previous example, we discovered that Václav has a something of 1.5121 to buy insurance. What is the something? Our gut really wants us to say that it is the probability that he buys insurance. In fact, it would be very helpful if we could predict Václav's probability of buying life insurance. Unfortunately, what we estimated cannot be a probability, as the value is greater than 1.

Notice, however, that we have just made an unconscious step in our minds: We are no longer thinking in terms of modeling the actual outcome (1 or 0); we are once again thinking in terms of modeling the expected value of the outcome, \(E[Y \mid x]\); here, that is the probability of a success, \(\pi\).

In other words, we are now modeling a variable we cannot measure — a latent variable. Instead of modeling an actual outcome, we now think in terms of modeling the underlying probability that the person will purchase life insurance. This has the dual advantage of being a continuous variable and of being bounded by 0 and 1 — exclusive.

As such, we can model it using previous techniques. Remember that the predicted value will be a probability, not an actual outcome we can measure. To predict the outcome, there is an additional step: selecting a threshold value, \(\tau\), above which we predict the individual bought insurance; below which, not. The traditional threshold value is \(\tau=0.500\); however, there is no reason we cannot alter it to better fit the data (see the section on maximum accuracy).

 

Thus, our research model in the life insurance example becomes

\begin{equation*}
\mathrm{logit} \Big( P[\texttt{insurance}] \Big) = \beta_0 + \beta_1 \texttt{age} + \beta_2 \texttt{income}
\end{equation*}

 

We use the logit function for the same reason we used it before: to transform the bounded variable into an unbounded variable. The right hand side of the equation is \(\eta\) ("eta"), a linear function that can take on all real values — the linear predictor. Figure \(\PageIndex{1}\) below shows a schematic of what we are actually modeling. The diagonal line in the top figure is the line of best fit for the linear predictor. The horizontal line is the threshold value we chose to distinguish between "Success" predictions and "Failure" predictions, which corresponds to \(\mathrm{logit}(\tau)\) in this top graph, \(\tau\) in the bottom. The bottom figure is the linear predictor back-transformed into "probability" units. The horizontal line is the actual \(\tau\) chosen, here \(\tau=0.500\).

 

Figure \(\PageIndex{1}\): Plot of the linear predictor and a possible threshold for a typical latent binary dependent variable model. The logit of the Linear Predictor is in level units (proportion units).

 

If we need to actually calculate the probability that Václav will purchase life insurance, we can calculate it from the linear predictor:

\begin{align*}
& \mathrm{logit}\big(\Pr[\text{insurance}]\big) = \eta \\
\end{align*}

This is equivalent to: \(P[\text{insurance}] = \text{logistic}\big( \eta \big)\).

 

 

This section examined the relationship between the line of best fit for the linear predictor, \(\eta\), and the predicted probability of a success. However, we did not discuss how that line of best fit was determined. The next section does just that.