17.2: Appendix B to Applied Probability- some mathematical aids
- Page ID
- 11785
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Series
1. Geometric series From the expression \((1 - r) (1 + r + r^2 + \cdot\cdot\cdot + r^n) = 1 - r^{n + 1}\), we obtain
\(\sum_{k = 0}^{n} r^k = \dfrac{1 - r^{n + 1}}{1 - r}\) for \(r \ne 1\)
For \(|r| < 1\), these sums converge to the geometric series \(\sum_{k = 0}^{\infty} r^k = \dfrac{1}{1 - r}\)
Differentiation yields the following two useful series:
\(\sum_{k = 1}^{\infty} kr^{k - 1} = \dfrac{1}{(1 - r)^2}\) for \(|r| < 1\) and \(\sum_{k = 2}^{\infty} k(k - 1)r^{k - 2} = \dfrac{2}{(1 - r)^3}\) for \(|r| < 1\)
For the finite sum, differentiation and algebraic manipulation yields
\(\sum_{k = 0}^{n} k r^{k - 1} = \dfrac{1 - r^n [1 + n(1 - r)]}{(1 - r)^2}\) which converges to \(\dfrac{1}{(1 - r)^2}\) for \(|r| < 1\)
2. Exponential series. \(e^k = \sum_{k = 0}^{\infty} \dfrac{x^k}{k!}\) and \(e^{-s} = \sum_{k = 0}^{\infty} (-1)^k \dfrac{x^k}{k!}\) for any \(x\)
Simple algebraic manipulation yields the following equalities useful for the Poisson distribution:
\(\sum_{k = n}^{\infty} k \dfrac{x^k}{k!} = x \sum_{k = n - 1}^{\infty} \dfrac{x^k}{k!}\) and \(\sum_{k = n}^{\infty} k (k - 1) \dfrac{x^k}{k!} = x^2 \sum_{k = n - 2}^{\infty} {x^k}{k!}\)
3. Sums of powers of integers \(\sum_{i = 1}^{n} i = \dfrac{n(n + 1)}{2}\) \(\sum_{i = 1}^{n} i^2 = \dfrac{n(n + 1)(2n + 1)}{6}\)
Some useful integrals
1. The gamma function \(\Gamma(r) = \int_{0}^{\infty} t^{r - 1} e^{-t}\ dt\) for \(r > 0\)
Integration by parts shows \(Gamma (r) = (r - 1) \Gamma (r - 1)\) for \(r > 1\)
By induction \(\Gamma (r) = (r - 1)(r - 2) \cdot\cdot\cdot (r - k) \Gamma (r - k)\) for \(r > k\)
For a positive integer \(n\), \(\Gamma (n) = (n - 1)!\) with \(\Gamma (1) = 0! = 1\)
2. By a change of variable in the gamma integral, we obtain
\(\int_{0}^{\infty} t^r e^{-\lambda t}\ dt = \dfrac{\Gamma (r+1)}{\lambda^{r + 1}}\) \(r > -1\), \(\lambda > 0\)
3. A well known indefinite integral gives
\(\int_{a}^{\infty} te^{-\lambda t}\ dt = \dfrac{m!}{\lambda^{m + 1}} e^{-\lambda a} [1 + \lambda a + \dfrac{(\lambda a)^2}{2!} + \cdot\cdot\cdot + \dfrac{(\lambda a)^m}{m!}]\)
4. The following integrals are important for the Beta distribution.
\(\int_{0}^{1} u^r (1 - u)^s\ du = \dfrac{\Gamma (r + 1) \Gamma (s + 1)}{\Gamma (r + s + 2)}\) \(r > -1\), \(s > -1\)
For nonnegative integers \(m, n\) \(\int_{0}^{1} u^m (1 - u)^n\ du = \dfrac{m! n!}{(m + n + 1)!}\)
Some basic counting problems
We consider three basic counting problems, which are used repeatedly as components of more complex problems. The first two, arrangements and occupancy are equivalent. The third is a basic matching problem.
Arrangements of \(r\) objects selected from among \(n\) distinguishable objects.
a. The order is significant.
b. The order is irrelevant.
For each of these, we consider two additional alternative conditions.
1. No element may be selected more than once.
2. Repitition is allowed.
Occupancy of \(n\) distinct cells by \(r\) objects. These objects are
a. Distinguishable.
b. Indistinguishable.
The occupancy may be
1. Exclusive.
2. Nonexclusive (i.e., more than one object per cell)
The results in the four cases may be summarized as follows:
a. 1. Ordered arrangements, without repetition (permutations). Distinguishable objects, exclusive occupancy.
\(P(n, r) = \dfrac{n!}{(n - r)!}\)
2. Ordered arrangements, with repitition allowed. Distinguishable objects, nonexclusive occupancy.
\(U(n,r) = n^r\)
b. 1. Arrangements without repetition, order irrelevant (combinations). Indistinguishable objects, exclusive occupancy.
\(C(n, r) = \dfrac{n!}{r!(n - r)!} = \dfrac{P(n, r)}{r!}\)
2. Unordered arrangements, with repetition. Indistinguishable objects, nonexclusive occupancy.
\(S(n, r) = C(n + r - 1, r)\)
Matching \(n\) distinguishable elements to a fixed order. Let \(M(n, k)\) be the number of permutations which give \(k\) matches.
\(n = 5\)
Natural order 1 2 3 4 5
Permutation 3 2 5 4 1 (Two matches– positions 2, 4)
We reduce the problem to determining \(m(n, 0)\), as follows:
Select \(k\) places for matches in \(C(n, k)\) ways.
Order the \(n - k\) remaining elements so that no matches in the other \(n - k\) places.
\(M(n, k) = C(n, k) M(n - k, 0)\)
Some algebraic trickery shows that \(M(n, 0)\) is the integer nearest \(n!/e\). These are easily calculated by the MATLAB command M = round(gamma(n+1)/exp(1))
For example >> M = round(gamma([3:10]+1)/exp(1)); >> disp([3:6;M(1:4);7:10;M(5:8)]') 3 2 7 1854 4 9 8 14833 5 44 9 133496 6 265 10 1334961
Extended binomial coefficients and the binomial series
The ordinary binomial coefficient is \(C(n, k) = \dfrac{n!}{k!(n - k)!}\) for integers \(n > 0\), \(0 \le k \le n\)
For any real \(x\), any integer \(k\), we extend the definition by
\(C(x, 0) = 1\), \(C(x, k) = 0\) for \(k < 0\), and \(C(n, k) = 0\) for a positive integer \(k > n\)
and
\(C(x, k) = \dfrac{x(x - 1) (x - 2) \cdot\cdot\cdot (x - k + 1)}{k!}\) otherwise
The Pascal's relation holds: \(C(x, k) = C(x - 1, k - 1) + C(x - 1, k)\)
The power series expansion about \(t = 0\) shows
\((1 + t)^x = 1 + C(x, 1)t + C(x, 2)t^2 + \cdot\cdot\cdot\) \(\forall x\), \(-1 < t < 1\)
For \(x = n\), a positive integer, the series becomes a polynomial of degree \(n\)
Cauchy's equation
Let \(f\) be a real-valued function defined on \((0, \infty)\), such that
a. \(f(t + u) = f(t) + f(u)\) for \(t, u > 0\), and
b. There is an open interval \(I\) on which \(f\) is bounded above (or is bounded below).
Then \(f(t) = f(1) t\) \(\forall t > 0\)
Let \(f\) be a real-valued function defined on (\(0, \infty\)) such that
a. \(f(t + u) = f(t)f(u)\) \(\forall t, u > 0\), and
b. There is an interval on which \(f\) is bounded above.
Then, either \(f(t) = 0\) for \(t > 0\), or there is a constant \(a\) such that \f(t) = e^{at}\) for \(t >0\)
[For a proof, see Billingsley, Probability and Measure, second edition, appendix A20]
Countable and uncountable sets
A set (or class) is countable iff either it is finite or its members can be put into a one-to-one correspondence with the natural numbers.
Examples
- The set of odd integers is countable.
- Thee finite set \(\{n: 1 \le n \le 1000\}\) is countable.
- The set of all rational numbers is countable. (This is established by an argument known as diagonalization).
- The set of pairs of elements from two countable sets is countable.
- The union of a countable class of countable sets is countable.
A set is uncountable iff it is neither finite nor can be put into a one-to-one correspondence with the natural numbers.
Examples
- The class of positive real numbers is uncountable. A well known operation shows that the assumption of countability leads to a contradiction.
- The set of real numbers in any finite interval is uncountable, since these can be put into a one-to-one correspondence of the class of all positive reals.