14.4: The Gaussian Distribution

To illustrate what we did in the previous sections, let us apply what we know to the Gaussian distribution, and show that it is exponential class. This will allow us to determine the canonical link, the expected value, and the variance. Hopefully, the results will not surprise us.

✦•················• ✦ •··················•✦

The Gaussian is Exponential Class

We start with the probability density function (pdf) of the Gaussian.

\begin{equation}
f(y) = \frac{1}{\sqrt{2 \pi \sigma^2}}\ \exp \left[ - \frac{(y - \mu)^2}{2 \sigma^2} \right] \\[1em]
\end{equation}

Now, to write this in standard form, we use algebra and some logarithm rules.

\begin{align}
f(y) & = \exp \left[ - \frac{(y - \mu)^2}{2 \sigma^2} + \log \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) \right] \\[2em]
& = \exp \left[ - \frac{y^2 -2y\mu + \mu^2}{2 \sigma^2} + \log \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) \right] \\[2em]
& = \exp \left[ - \frac{y^2}{2 \sigma^2} + \frac{y\mu}{\sigma^2} - \frac{\mu^2}{2 \sigma^2} + \log \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) \right] \\[2em]
& = \exp \left[ \frac{y\mu -\frac{1}{2} \mu^2}{\sigma^2} + \log \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) - \frac{y^2}{2 \sigma^2} \right]
\end{align}

Recall from the previous section that the "standard form" is

\begin{equation}
f(y) = \exp \left[ \frac{y \theta - b(\theta)}{a(\phi)} + c(y,\phi) \right]
\end{equation}

Comparing the two distributions gives us the following:

• \(y = y\)
• \(\theta = \mu\)
• \(a(\phi)=\sigma^2\)
• \(b(\theta) = \frac{1}{2} \mu^2 = \frac{1}{2} \theta^2\)
• \(c(y, \phi)= \log \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \right) - \frac{y^2}{2 \sigma^2}\)

Thus, according to this list, the canonical link is \(g(\mu) = \mu\), also known as the identity function.

The dispersion parameter is \(a(\phi) = \sigma^2\).

Also note that the expected value is

\begin{align}
E[Y] & = b^\prime\!(\theta) \\[1em]
&= \frac{\text{d}}{\text{d}\theta} \left( \frac{1}{2} \theta^2 \right) \\[1em]
&= \theta \\[1em]
&= \mu
\end{align}

Hopefully, this is as we expect. Finally, note that the variance is

\begin{align}
V[Y] & = b^{\prime\prime}\!(\theta) a(\phi) \\[1em]
&= \frac{\text{d}^2}{\text{d}\theta^2} \left( \frac{1}{2} \theta^2 \sigma^2 \right)\\[1em]
&= \frac{\text{d}}{\text{d}\theta} \left( \theta \sigma^2 \right)\\[1em]
&= \sigma^2
\end{align}

Also as we expect, hopefully.

Other Link Functions

While the canonical link is the identity function (\(\eta = \mu\)), it is not the only allowable link function. In Section 8.1: The Issue of Boundedness, we transformed the continuous dependent variable because it was bounded below by (but never equaled) zero. In such a case, the logarithm is an appropriate link function: The dependent variable has a restricted range. The link function converts that range to an unbounded range. The same is true under the GLM framework. Similarly, the logit function is frequently an appropriate link function, as it was in Section 8.1: The Issue of Boundedness.

With that, we start to see that for continuous dependent variables, what we did under the CLM paradigm we can do under the GLM paradigm. This is always true; the GLM paradigm extends the CLM paradigm to handle different classes of dependent variables.