4.5: Exponential and Gamma Distributions

In this section, we introduce two families of continuous probability distributions that are commonly used.

Exponential Distributions

Definition \(\PageIndex{1}\)

A random variable \(X\) has an exponential distribution with parameter \(\lambda>0\), write \(X\sim\text{exponential}(\lambda)\), if \(X\) has pdf given by
$$f(x) = \left\{\begin{array}{l l}
\lambda e^{-\lambda x}, & \text{for}\ x\geq 0, \\
0, & \text{otherwise.}
\end{array}\right.\notag$$

Figure 1: Graph of pdf for exponential(\(\lambda=5\)) distribution.

Example \(\PageIndex{1}\)

A typical application of exponential distributions is to model waiting times or lifetimes. For example, each of the following gives an application of an exponential distribution.

• \(X=\) lifetime of a radioactive particle
• \(X=\) how long you have to wait for an accident to occur at a given intersection
• \(X=\) length of interval between consecutive occurrences of Poisson distributed events

---

The parameter \(\lambda\) is referred to as the rate parameter, it represents how quickly events occur.  For example, in the first case above where \(X\) denotes the lifetime of a radioactive particle, \(\lambda\) would give the rate at which such particles decay.

Properties of Exponential Distributions

If \(X\sim\text{exponential}(\lambda)\), then the following hold.

1. The cdf of \(X\) is given by
   $$F(x) = \left\{\begin{array}{l l}
   0, & \text{for}\ x< 0, \\
   1- e^{-\lambda x}, & \text{for}\ x\geq 0. \\
   \end{array}\right.\notag$$
2. For any \(0 < p < 1\), the \((100p)^{\text{th}}\) percentile is \(\displaystyle{\pi_p = \frac{-\ln(1-p)}{\lambda}}\).
3. The mean of \(X\) is \(\displaystyle{\text{E}[X] = \frac{1}{\lambda}}\).
4. The variance of \(X\) is \(\displaystyle{\text{Var}(X) = \frac{1}{\lambda^2}}\).
5. The mgf of \(X\) is
   $$M_X(t) = \frac{1}{1-(t/\lambda)}, \quad\text{for}\ t<\lambda.\notag$$
6. \(X\) satisfies the Memoryless Property, i.e., \(P(X>t+s\ |\ X>s) = P(X>t)\), for any \(t,s \geq0\).

Partial Proof

We prove Properties #1 & #3, the others are left as an exercise.

For the first property, we consider two cases based on the value of \(x\). First, if \(x<0\), then the pdf is constant and equal to 0, which gives the following for the cdf:
$$F(x) = \int^{x}_{-\infty} f(t) dt = \int^x_{-\infty} 0 dt = 0 \notag$$
Second, if \(x\geq0\), then the pdf is \(\lambda e^{-\lambda x}\), and the cdf is given by $$F(x) = \int^x_{-\infty} f(t) dt = \int^x_0 \lambda e^{-\lambda t} dt = -e^{-\lambda t}\Big|^x_0 = -e^{-\lambda x} - (-e^0) = 1-e^{-\lambda x}. \notag$$

For the third property, we Definition 4.2.1 to calculate the expected value of a continuous random variable:
$$\text{E}[X] = \int^{\infty}_{-\infty} x\cdot f(x) dx = \int^{\infty}_0 x\cdot \lambda e^{-\lambda x} dx = -x\cdot e^{-\lambda x}\big|^{\infty}_0 + \int^{\infty}_0 e^{-\lambda x} dx = 0 + \frac{-e^{-\lambda x}}{\lambda}\big|^{\infty}_0 = \frac{1}{\lambda}. \notag$$

In words, the Memoryless Property of exponential distributions states that, given that you have already waited more than \(s\) units of time (\(X>s)\), the conditional probability that you will have to wait \(t\) more (\(X>t+s\)) is equal to the unconditional probability you just have to wait more than \(t\) units of time. For example, suppose you are waiting for the bus and the amount of time you have to wait is exponentially distributed. If you have already been waiting 5 minutes at the bus stop, the probability that you have to wait 4 more minutes (so more than 9 minutes total) is equal to the probability that you only had to wait more than 4 minutes once arriving at the bus stop. In calculating the conditional probability, the exponential distribution "forgets" about the condition or the time already spent waiting and you can just calculate the unconditional probability that you have to wait longer. Note that we saw earlier that geometric distributions also have the Memoryless Property.

Gamma Distributions

Definition \(\PageIndex{2}\)

A random variable \(X\) has a gamma distribution with parameters \(\alpha, \lambda>0\), write \(X\sim\text{gamma}(\alpha, \lambda)\), if \(X\) has pdf given by
$$f(x) = \left\{\begin{array}{l l}
\displaystyle{\frac{\lambda^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\lambda x}}, & \text{for}\ x\geq 0, \\
0 & \text{otherwise,}
\end{array}\right. \notag$$
where \(\Gamma(\alpha)\) is a function (referred to as the gamma function) given by the following integral:
$$\Gamma(\alpha) = \int^{\infty}_0 t^{\alpha-1}e^{-t}dt. \notag$$

Figure 2: Graph of pdf's for various gamma distributions. On the left, for the purple pdf \(\alpha=0.5\) and for the green pdf \(\alpha=1.5\). On the right, for the blue pdf \(\alpha=4\) and for the orange pdf \(\alpha=8\). For all pdf's, \(\lambda=5\).

Note that the gamma function, \(\Gamma(\alpha)\), ensures that the gamma pdf is valid, i.e., that it integrates to \(1\), which you are asked to show in the following exercise. The value of \(\Gamma(\alpha)\) depends on the value of the parameter \(\alpha\), but for a given value of \(\alpha\) it is just a number, i.e., it is a constant value in the gamma pdf, given specific parameter values. In this case, \(\Gamma(\alpha)\) is referred to as a scaling constant, since it "scales" the rest of the pdf, \(\lambda^{\alpha}x^{\alpha - 1}e^{-\lambda x}\), which is referred to as the kernel of the distribution, so that the result integrates to \(1\).

Exercise \(\PageIndex{1}\)

Show: \(\displaystyle{\int^{\infty}_0 \frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x} dx = 1}\)

Answer

In the integral, we can make the substitution: \(u = \lambda x \rightarrow du = \lambda dx\). Therefore, we have
$$\int^{\infty}_0 \frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x} dx = \int^{\infty}_0 \frac{\lambda \lambda^{\alpha-1}}{\Gamma(\alpha)} x^{\alpha-1}e^{-\lambda x} dx = \frac{1}{\Gamma(\alpha)}\int^{\infty}_0 u^{\alpha-1}e^{-u} du = \frac{1}{\Gamma(\alpha)}\Gamma(\alpha) = 1. \notag$$

Properties of Gamma Distributions

If \(X\sim\text{gamma}(\alpha,\lambda)\), then the following hold.

1. The mgf of \(X\) is
   $$M_X(t) = \frac{1}{\left(1-(t/\lambda)\right)^{\alpha}}, \quad\text{for}\ t<\lambda.\notag$$
2. The mean of \(X\) is \(\displaystyle{\text{E}[X] = \frac{\alpha}{\lambda}}\).
3. The variance of \(X\) is \(\displaystyle{\text{Var}(X) = \frac{\alpha}{\lambda^2}}\).

Notes about Gamma Distributions:

• If \(\alpha = 1\), then the corresponding gamma distribution is given by the exponential distribution, i.e., \(\text{gamma}(1,\lambda) = \text{exponential}(\lambda)\). This is left as an exercise for the reader.
• The parameter \(\alpha\) is referred to as the shape parameter, and \(\lambda\) is the rate parameter. Varying the value of \(\alpha\) changes the shape of the pdf, as is seen in Figure 2 above, whereas varying the value of \(\lambda\) corresponds to changing the units (e.g., from inches to centimeters) and does not alter the shape of the pdf.
• A closed form does not exist for the cdf of a gamma distribution, computer software must be used to calculate gamma probabilities. Here is a link to a gamma calculator online. (Note that different notation is used on this online calculator, namely, \(\lambda\) is referred to as \(\beta\) instead.)

Example \(\PageIndex{2}\)

A typical application of gamma distributions is to model the time it takes for a given number of events to occur. For example, each of the following gives an application of a gamma distribution.

• \(X=\) lifetime of 5 radioactive particles
• \(X=\) how long you have to wait for 3 accidents to occur at a given intersection

In these examples, the parameter \(\lambda\) represents the rate at which the event occurs, and the parameter \(\alpha\) is the number of events desired. So, in the first example, \(\alpha=5\) and \(\lambda\) represents the rate at which particles decay.

---