4.1: Independence of Events

Historically, the notion of independence has played a prominent role in probability. If events form an independent class, much less information is required to determine probabilities of Boolean combinations and calculations are correspondingly easier. In this unit, we give a precise formulation of the concept of independence in the probability sense. As in the case of all concepts which attempt to incorporate intuitive notions, the consequences must be evaluated for evidence that these ideas have been captured successfully.

Independence as lack of conditioning

There are many situations in which we have an “operational independence.”

• Supose a deck of playing cards is shuffled and a card is selected at random then replaced with reshuffling. A second card picked on a repeated try should not be affected by the first choice.
• If customers come into a well stocked shop at different times, each unaware of the choice made by the others, the the item purchased by one should not be affected by the choice made by the other.
• If two students are taking exams in different courses, the grade one makes should not affect the grade made by the other.

The list of examples could be extended indefinitely. In each case, we should expect to model the events as independent in some way. How should we incorporate the concept in our developing model of probability?

We take our clue from the examples above. Pairs of events are considered. The “operational independence” described indicates that knowledge that one of the events has occured does not affect the likelihood that the other will occur. For a pair of events {\(A\), \(B\)}, this is the condition

\(P(A|B) = P(A)\)

Occurrence of the event \(A\) is not “conditioned by” occurrence of the event \(B\). Our basic interpretation is that \(P(A)\) indicates of the likelihood of the occurrence of event \(A\). The development of conditional probability in the module Conditional Probability, leads to the interpretation of \(P(A|B)\) as the likelihood that \(A\) will occur on a trial, given knowledge that \(B\) as occurred. If such knowledge of the occurrence of \(B\) does not affect the likelihood of the occurrence of \(A\), we should be inclined to think of the events \(A\) and \(B\) as being independent in a probability sense.

Independent pairs

We take our clue from the condition \(P(A|B) = P(A)\). Property (CP4) for conditional probability (in the case of equality) yields sixteen equivalent conditions as follows.

\(P(A|B) = P(A)\)	\(P(B|A) = P(B)\)	\(P(AB) = P(A) P(B)\)
\(P(A|B^c) = P(A)\)	\(P(B^c|A) = P(B^c)\)	\(P(AB^c) = P(A) P(B^c)\)
\(P(A^c|B) = P(A^c)\)	\(P(B|A^c) = P(B)\)	\(P(A^c B) = P(A^c)P(B)\)
\(P(A^c|B^c) = P(A^c)\)	\(P(B^c|A^c) = P(B^c)\)	\(P(A^cB^c) = P(A^c) P(B^c)\)

\(P(A|B) = P(A|B^c)\)

\(P(A^c|B) = P(A^c|B^c)\)

\(P(B|A) = P(B|A^c)\)

\(P(B^c|A) = P(B^c|A^c)\)

These conditions are equivalent in the sense that if any one holds, then all hold. We may chose any one of these as the defining condition and consider the others as equivalents for the defining condition. Because of its simplicity and symmetry with respect to the two events, we adopt the product rule in the upper right hand corner of the table.

Definition. The pair {\(A\), \(B\)} of events is said to be (stochastically) independent iff the following product rule holds:

\(P(AB) = P(A) P(B)\)

Remark. Although the product rule is adopted as the basis for definition, in many applications the assumptions leading to independence may be formulated more naturally in terms of one or another of the equivalent expressions. We are free to do this, for the effect of assuming any one condition is to assume them all.

The equivalences in the right-hand column of the upper portion of the table may be expressed as a replacement rule, which we augment and extend below:

If the pair {\(A\), \(B\)} independent, so is any pair obtained by taking the complement of either or both of the events.

We note two relevant facts

• Suppose event \(N\) has probability zero (is a null event). Then for any event \(A\), we have \(0 \le P(AN) \le P(N) = 0 = P(A)P(N)\), so that the product rule holds. Thus {\(N\), \(A\)} is an independent pair for any event \(A\).
• If event \(S\) has probability one (is an almost sure event), then its complement \(S^c\) is a null event. By the replacement rule and the fact just established, \({S^c\), \(A\)} is independent, so {\(S\), \(A\)} is independent.

The replacement rule may thus be extended to:

Replacement Rule

If the pair {\(A\), \(B\)} independent, so is any pair obtained by replacing either or both of the events by their complements or by a null event or by an almost sure event.

CAUTION

1. Unless at least one of the events has probability one or zero, a pair cannot be both independent and mutually exclusive. Intuitively, if the pair is mutually exclusive, then the occurrence of one requires that the other does not occur. Formally: Suppose \(0 < P(A) < 1\) and \(0 < P(B) < 1\). {\(A\), \(B\)} mutually exclusive implies \(P(AB) = P(\emptyset) = 0 \ne P(A) P(B)\). {\(A\), \(B\)} independent implies \(P(AB) = P(A) P(B) > 0 = P(\emptyset)\)
2. Independence is not a property of events. Two non mutually exclusive events may be independent under one probability measure, but may not be independent for another. This can be seen by considering various probability distributions on a Venn diagram or minterm map.

Independent classes

Extension of the concept of independence to an arbitrary class of events utilizes the product rule.

Definition. A class of events is said to be (stochastically) independent iff the product rule holds for every finite subclass of two or more events in the class.

A class {\(A\), \(B\), \(C\)} is independent iff all four of the following product rules hold

\(P(AB) = P(A) P(B)\) \(P(AC) = P(A) P(C)\) \(P(BC) = P(B) P(C)\) \(P(ABC) = P(A) P(B) P(C)\)

If any one or more of these product expressions fail, the class is not independent. A similar situation holds for a class of four events: the product rule must hold for every pair, for every triple, and for the whole class. Note that we say “not independent” or “nonindependent” rather than dependent. The reason for this becomes clearer in dealing with independent random variables.

We consider some classical exmples of nonindependent classes

SOME NONINDEPENDENT CLASSES

1. Suppose {\(A_1\), \(A_2\), \(A_3\), \(A_4\)} is a partition, with each \(P(A_i) = 1/4\). Let
   
   \(A = A_1 \bigvee A_2 B = A_1 \bigvee A_3 C = A_1 \bigvee A_4\)
   
   Then the class {\(A\), \(B\), \(C\)} has \(P(A) = P(B) = P(C) = 1/2\) and is pairwise independent, but not independent, since \(P(AB) = P(A_1) = 1/4 = P(A) P(B)\) and similarly for the other pairs, but \(P(ABC) = P(A_1) = 1/4 \ne P(A)P(B)P(C)\)
2. Consider the class {\(A\), \(B\), \(C\), \(D\)} with \(AD = BD = \emptyset\), \(C = AB \bigvee D\), \(P(A) = P(B) = 1/4\), \(P(AB) = 1/64\), and \(P(D) = 15/64\). Use of a minterm maps shows these assignments are consistent. Elementary calculations show the product rule applies to the class {\(A\), \(B\), \(C\)} but no two of these three events forms an independent pair.

As noted above, the replacement rule holds for any pair of events. It is easy to show, although somewhat cumbersome to write out, that if the rule holds for any finite number \(k\) of events in an independent class, it holds for any \(k + 1\) of them. By the principle of mathematical induction, the rule must hold for any finite subclass. We may extend the replacement rule as follows.

General Replacement Rule

If a class is independent, we may replace any of the sets by its complement, by a null event, or by an almost sure event, and the resulting class is also independent. Such replacements may be made for any number of the sets in the class. One immediate and important consequence is the following.

Minterm Probabilities

If {\(A_i: 1 \le i \le n\)} is an independent class and the the class {\(P(A_i):1 \le i \le n\)} of individual probabilities is known, then the probability of every minterm may be calculated.

Minterm probabilities for an independent class

Suppose the class {\(A\), \(B\), \(C\)} is independent with respective probabilities \(P(A) = 0.3\), \(P(B) = 0.6\), and \(P(C) = 0.5\). Then

{\(A^c\), \(B^c\), \(C^c\)} is independent and \(P(M_0） = P(A^c)P(B^c)P(C^c) = 0.14\)

{\(A^c\), \(B^c\), \(C\)} is independent and \(P(M_1) = P(A^c)P(B^c)P(C) = 0.14\)

Similarly, the probabilities of the other six minterms, in order, are 0.21, 0.21, 0.06, 0.06, 0.09, and 0.09. With these minterm probabilities, the probability of any Boolean combination of \(A\), \(B\), and \(C\) may be calculated

In general, eight appropriate probabilities must be specified to determine the minterm probabilities for a class of three events. In the independent case, three appropriate probabilities are sufficient.

Three probabilities yield the minterm probabilities

Suppose {\(A\), \(B\), \(C\)} is independent with \(P(A \cup BC) = 0.51\), \(P(AC^c) = 0.15\), and \(P(A) = 0.30\). Then \(P(C^c) = 0.15/0.3 = 0.5 = P(C)\) and

\(P(A) + P(A^c) P(B) P(C) = 0.51\) so that \(P(B) = \dfrac{0.51 - 0.30}{0.7 \times 0.5} = 0.6\)

With each of the basic probabilities determined, we may calculate the minterm probabilities, hence the probability of any Boolean combination of the events.

MATLAB and the product rule

Frequently we have a large enough independent class {\(E_1\), \(E_2\), \(\cdot \cdot\ cdot\), \(E_n\)} that it is desirable to use MATLAB (or some other computational aid) to calculate the probabilities of various “and” combinations (intersections) of the events or their complements. Suppose the independent class {\(E_1\), \(E_2\), \(\cdot \cdot\ cdot\), \(E_{10}\)} has respective probabilities

0.13 0.37 0.12 0.56 0.33 0.71 0.22 0.43 0.57 0.31

It is desired to calculate (a) \(P(E_1 E_2 E_3^c E_4 E_5^c E_6^c E_7)\), and (b) \(P(E_1^c E_2 E_3^c E_4 E_5^c E_6^c E_7 E_8 E_9^c E_{10})\).

We may use the MATLAB function prod and the scheme for indexing a matrix.

>> p = 0.01*[13 37 12 56 33 71 22 43 57 31];
>> q = 1-p;
>> % First case
>> e = [1 2 4 7];                  % Uncomplemented positions
>> f = [3 5 6];                    % Complemented positions
>> P = prod(p(e))*prod(q(f))       % p(e) probs of uncomplemented factors
P = 0.0010                         % q(f) probs of complemented factors
>> % Case of uncomplemented in even positions; complemented in odd positions
>> g = find(rem(1:10,2) == 0);     % The even positions
>> h = find(rem(1:10,2) ~= 0);     % The odd positions
>> P = prod(p(g))*prod(q(h))
P = 0.0034

In the unit on MATLAB and Independent Classes, we extend the use of MATLAB in the calculations for such classes.