22.3: Population Parameters

Let us spend a section examining population parameters. A population parameter is a measurement on the population, analogous to the sample statistic being a measurement on the sample. Note that it is the population parameters we seek to know. The sample statistics are only used to give us information about them. Some usual population parameters of interest are the mean, variance, covariance, and correlation.

✦•················• ✦ •··················•✦

Population Mean

The population mean is a measure of center (or "typicalness") for the entire distribution (or random variable). It is known by a few different names: population mean, expected value, and first moment. It is represented by some variation of either \(\mu\) or \(E[Y]\).

If the distribution is discrete, the formula for the expected value is defined as
\begin{equation}
E[Y] \stackrel{\text{def}}{=} \sum_{i \in \mathcal{S}}\ y_i\ P[Y=y_i] \label{eq:appS-expectedValueD}
\end{equation}

If the distribution is continuous, the formula for the expected value is
\begin{equation}
E[Y] \stackrel{\text{def}}{=} \int_{\mathcal{S}}\ y\ f(y)\ \text{d}y \label{eq:appS-expectedValueC}
\end{equation}

Note the similarities between the two formulas. The difference is due solely to the mathematics underlying Equations \(\ref{eq:appS-expectedValueD}\) and \(\ref{eq:appS-expectedValueC}\). Those mathematics tend to be covered in an undergraduate real analysis course.

Another interpretation of the expected value is the "long-run average" of the outcomes. I frequently find this useful in checking that my results match my expectations.

Example \(\PageIndex{1}\)

Let \(Y\) be the number of heads in one flip of a fair coin. What is the expected number of heads?

Solution.
The random variable \(Y\) is discrete with sample space \(\mathcal{S}=\set{0,1}\). Thus, the formula for the expected value gives us

\begin{align}
E[Y] &\stackrel{\text{def}}{=} \sum_{i \in \mathcal{S}}\ y_i\ P[Y=y_i] \\[1em]
&= 0\ P[Y=0] + 1\ P[Y=1] \\[1em]
&= 0 (0.500) + 1 (0.500)
\end{align}

Thus, the expected number of heads is \(E[Y] = 0.500\).

The expected value of a random variable does not need to be an element of the sample space (as here). Thus, the interpretation as a long-run average helps in understanding and interpreting the mean.

Example \(\PageIndex{2}\)

Let \(Y\) be the time I spend at a stoplight. If the light has a 120-second cycle, spending 55s on green, 5s on yellow, and 60s on red, calculate the expected time I wait, \(given\) that I have to stop.

Solution.
The random variable \(Y\) is continuous with sample space \(\mathcal{S}=[0, 60]\). Without any information beyond knowing that there is a lower and an upper bound, we should assume the random variable follows a Uniform distribution.

Thus, the formula for the expected value gives us

\begin{align}
E[Y] &\stackrel{\text{def}}{=} \int_{\mathcal{S}} y\ f(y) \text{d}y \\[1ex]
&= \int_0^{60} y\ \frac{1}{60} \text{d}y \\[1ex]
&= \frac{1}{60} \int_0^{60} y \text{d}y \\[1ex]
&= \frac{1}{60} \left. \frac{y^2}{2} \right|_{y=0}^{60} \\[1ex]
&= \frac{1}{60} \left( \frac{60^2}{2} - \frac{0^2}{2} \right) \\[1em]
&= 30
\end{align}

Thus, the expected time I spend at the stop light is \(E[Y] = 30\) seconds. Why does that not surprise me?

\(\blacksquare\)

For those interested in understanding why the Uniform is the distribution, please read up on "maximum entropy distributions."

Example \(\PageIndex{3}\)

Let \(Y\) be the time I spend at a stoplight. If the light has a 120-second cycle, spending 55s on green, 5s on yellow, and 60s on red, calculate the expected time I wait.

Solution.
This is quite different from the previous example. There, we knew the light was red. Now, we do not know if it is green (no time stopped), yellow (no time stopped), or red.

One can follow the above procedure to solve the problem. However, I would like to use a more helpful procedure. To motivate the procedure, note that the mean has been calculated as the sum of each value times its probability. Thus, we could calculate the expected time to wait as

\begin{align}
E[\text{wait}] &= P[\text{green}]\ E[\text{green}] + P[\text{yellow}]\ E[\text{yellow}] + P[\text{red}]\ E[\text{red}] \\[1em]
&= \frac{55}{120}\ 0 + \frac{5}{120}\ 0 + \frac{60}{120}\ 30 \\[1em]
&= \frac{1800}{120}
\end{align}

Thus, without the additional information that I actually stopped at the stoplight, the expected wait time is only 15s.

\(\blacksquare\)

───── ⋆⋅☆⋅⋆ ─────

With formulas for the expected value (equations \(\ref{eq:appS-expectedValueD}\) and \(\ref{eq:appS-expectedValueC}\)), one can find the expected value of any function of a random variable, too. For instance, \(E[Y^2]\) is the expected value of the square of the random variable. It is calculated as

\begin{equation}
E[Y^2] = \sum_{i \in \mathcal{S}}\ y_i^2\ P[Y=y_i]
\end{equation}

if \(Y\) is discrete, and

\begin{equation}
E[Y^2] = \int_{\mathcal{S}}\ y_i^2\ f(y)\ \text{d}y
\end{equation}

if \(y\) is continuous.

This will come in handy in terms of notation.

Example \(\PageIndex{4}\)

Let \(Y \sim Bern(\pi=0.500)\). Calculate \(E[Y]\), \(E[Y^2]\), and \(E[Y^3]\).

Solution.
Recall that the probability mass function for \(Y\) is given by

\begin{equation}
f(y) = \left\{ \begin{array}{ll} 0.500 & y=0 \\ 0.500 & y=1 \end{array}\right.
\end{equation}

Thus, we have

\begin{align}
E[Y] &\stackrel{\text{def}}{=} \sum_{y \in S}\ p(y)\ y \\[1em]
&= (0.500)\ 0 + (0.500)\ 1 \\[1ex]
&= 0.500
\end{align}

and

\begin{align}
E[Y^2] &= \sum_{y \in S}\ p(y)\ y^2 \\[1em]
&= (0.500)\ 0^2 + (0.500)\ 1^2 \\[1ex]
&= 0.500
\end{align}

and

\begin{align}
E[Y^3] &= \sum_{y \in S}\ p(y)\ y^3 \\[1em]
&= (0.500)\ 0^3 + (0.500)\ 1^3 \\[1ex]
&= 0.500
\end{align}

Clearly, that these three moments are equal is a feature of the Bernoulli distribution. Such will not necessarily be true for any other distribution.

Lemma \(\PageIndex{1}\): Bernoulli Moments

If \(Y \sim Bern(\pi)\), then \(E[Y^n] = \pi\) for any positive, finite-valued \(n\).

Proof.
Recall that the probability mass function for \(Y\) is given by

\begin{equation}
f(y) = \left\{ \begin{array}{ll} 1-\pi & y=0 \\ \pi & y=1 \end{array}\right.
\end{equation}

Thus, we have

\begin{align}
E[Y^n] &= \sum_{y \in S}\ p(y)\ y^n \\[1em]
&= (1-\pi)\ 0^n + (\pi)\ 1^n \\[1ex]
&= (1-\pi)\ 0 + (\pi)\ 1 \\[1ex]
&= \pi
\end{align}

Thus, it is proven.

\(\blacksquare\)

When discussing the random variable, one will tend to refer to \(E[Y]\). When discussing the distribution of the random variable, one will tend to refer to \(\mu\). Since random variables "follow" (or "have") a distribution, the two terms tend to be treated as being interchangeable, especially since \(E[Y] = \mu\).

Formula \(\ref{eq:appS-expectedValueD}\) for the expected value is the one we all see in our introductory statistics course. There is another formula that may be useful to know. Given \(F(y)\) is the cumulative distribution function (CDF) for \(Y\), then this is true:

\begin{equation}\label{eq:appS-meanCDF}
E[Y] = \int_0^{\infty}\ \Big( 1-F(y) \Big)\ \text{d}y
\end{equation}

Note that this formula only works if the support of the random variable is non-negative.

Example \(\PageIndex{5}\)

Let \(Y\) be the time I spend at a stoplight. If the light has a 120-second cycle, spending 55s on green, 5s on yellow, and 60s on red, calculate the expected time I wait, given that the light is red. Use Formula \(\ref{eq:appS-meanCDF}\).

Solution.
This is the same problem as above, so we should get the same answer. This time, however, we need to use equation \(\ref{eq:appS-meanCDF}\). We know that the CDF for the above Uniform distribution is

\begin{equation}
F(y) = \left\{ \begin{array}{cl} 0 & y \leq 0 \\ y/60 & 0 < y \leq 60 \\ 1 & y > 60 \\ \end{array} \right.
\end{equation}

Thus, \(E[Y]\) is

\begin{align}
E[Y] &= \int_0^{\infty}\ \Big( 1-F(y) \Big)\ \text{d}y \\[1em]
&= \int_0^{\infty}\ \left( \begin{array}{cl} 1-y/60 & 0 < y \leq 60 \\ 0 & y > 60 \\ \end{array}\right) \ \text{d}y \\[1ex]
&= \int_0^{60}\ 1-y/60\ \text{d}y \\[1ex]
\ldots \nonumber
\end{align}

I leave the rest as an exercise for you.

\(\blacksquare\)

Population Variance

The population variance is a measure of uncertainty (or of spread or uncertainty) for the distribution. It is symbolized as \(\sigma^2\) and as \(V[Y]\). When discussing the random variable, one will tend to refer to \(V[Y]\); however, when discussing the distribution of the random variable, one will tend to refer to \(\sigma^2\), as above.

The typical population variance formulas are

\begin{align}
V[Y] &= \sum_{i \in S}\ (y_i-\mu)^2\ P[Y=y_i] \label{eq:appS-varFormulaD} \\[1em]
&= \int_S\ (y_i-\mu)^2\ f(y) \text{d}y \label{eq:appS-varFormulaC}
\end{align}

Note that these are merely

\begin{equation}
V[Y] = E[(Y-\mu)^2]
\end{equation}

In other words, the variance is defined as the second central moment.

This last definition may be more helpful in our understanding of the variance, especially as it leads to a different formula for the variance:

\begin{equation}
E[Y^2] = \sigma^2 + \mu^2
\end{equation}

Theorem \(\PageIndex{2}\): Another Variance Formula

Let \(Y\) be a random variable with finite mean and variance.

\begin{equation}E[Y^2] = V[Y] + E[Y]^2\end{equation}

Proof.
I leave the proof as an exercise.

Another Variance Formula

The variance formulas given above in Equations \(\ref{eq:appS-varFormulaD}\) and \(\ref{eq:appS-varFormulaC}\) are the typical ones provided in undergraduate statistics. They depend on the probability mass (or density) function (pmf or pdf). There is a formula for the population variance that relies on the cumulative distribution function (CDF) instead (much like formula \(\ref{eq:appS-meanCDF}\)). Here it is.

\begin{align}
\sigma^2\ &=\ 2 \int_{0}^{\infty}\ y \Big( 1-F(y) \Big)\ \text{d}y\ -\ \left( \int_0^{\infty}\ \Big( 1-F(y) \Big)\ \text{d}y \right)^2
\end{align}

Note that this formula only works for a non-negative random variable... that is, for random variables whose possible values must be positive. Also note that this formula should only be used if the CDF is easier to work with than the pdf. This is a rare event, which illustrates why few see this formula.

Example \(\PageIndex{6}\)

Show that the variance of a standard Uniform distribution is

\begin{equation}
V[Y] = \frac{1}{12}
\end{equation}

Use both the pdf and the CDF method.

Solution.
I leave this as an exercise for you.

Population Covariance and Correlation

The formula for the population covariance is

\begin{equation}
\sigma_{xy} = E[(X-\mu_x)(Y-\mu_y)]
\end{equation}

Note this reduces to \(V[Y] = Cov[Y,Y]\) and \(\sigma^2_{y} = \sigma_{yy}\).

The formula for the population correlation is

\begin{equation}
\rho_{xy} = \frac{\sigma_{xy}}{\sigma_x\sigma_y}
\end{equation}

The Greek letter \(\rho\) is "rho." It is not a "p."

Example \(\PageIndex{7}\)

What is the covariance between a random variable and a constant?

Solution.
I leave this as an exercise for you.

Example \(\PageIndex{8}\)

What is the covariance between two constants?

Solution.
I leave this as an exercise for you.