4.3: Foundational Probability Rules
- Page ID
- 58899
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Now that we’ve built an idea of probability as long-run behavior, we’re ready to learn the tools and terminology that help us compute it. That starts with identifying the building blocks behind any chance-based scenario.
Definition: Experiment
An experiment is a process or action that produces a result, called an outcome, under conditions that involve randomness or unpredictability.
In probability, an experiment is something we can do repeatedly, where the exact result cannot be predicted in advance, but the set of all possible outcomes is known. Many of our examples will be simple for ease of learning, but realistic scenarios can also be thought of as experiments.
Examples:
- Flipping a coin
- Rolling a die
- Drawing a card from a shuffled deck of playing cards
- Collecting daily rainfall data
- Surveying a randomly selected person about their favorite food
- Administering a new drug to a patient and recording if they encounter side-effects
- Investing in a stock and seeing the resulting profit
- Giving an exam to a student and grading their results
In probability, everything begins with a clearly defined experiment. Each time we run the experiment, we observe one specific outcome. We may not know in advance which outcome will occur, but we can describe all the possible outcomes that could happen. Defining the experiment and the possible outcomes first helps us stay clear about what we’re measuring and what each result means.
Definitions: Outcome & Sample Space
An outcome is a single possible result of a defined experiment. Every time we run a probability experiment, one specific result occurs. That result is the outcome.
For example, if the experiment is rolling a six-sided die, one possible outcome might be 4. Order generally matters with outcomes. Rolling two dice and getting a 4 and a 2 is a different outcome than a 2 and a 4.
The sample space, denoted by \( S \), is the set of all possible outcomes of a probability experiment.
Using the die example, the sample space would be:
\( S = \{1, 2, 3, 4, 5, 6\} \)
In summary: An experiment produces one outcome, and the sample space lists every outcome that could possibly happen.
A few examples
Experiment: Flip two fair coins.
Sample space: \(\{(H, H), (H, T), (T, H), (T, T)\}\)
Experiment: Give a student an exam.
Sample space: \(\{A, B, C, D, F\}\)
Experiment: Recording daily rainfall
Sample space: All positive real numbers
Note that our sample space can be discrete or continuous just like the data from chapter 2. Additionally, there may be different ways to construct the sample space. If we wanted the percentage of the exam, our sample space would be all percentages between 0% and 100% instead of letter grades.
Definition: Event
An event is a set of outcomes from a random experiment (a subset of the sample space).
This is best illustrated with a few examples
Experiment: Flip two coins
Event description: Get at least one Heads
Event: \(\{(H,H), (H,T), (T, H)\}\)
Experiment: Give a student an exam
Event description: Student passes
Event: \(\{A, B, C\}\)
Experiment: Recording daily rainfall
Event description: More than 2 inches of rain
Event: All real numbers greater than 2. We can write this as \(R > 2\) where \(R\) is the number of inches of rain.
Related Video - Sample Space and Events
When Events Overlap or Not
Definition: Mutually Exclusive Events
Two events are mutually exclusive if they cannot happen at the same time. Their outcomes do not overlap. We also call these events disjoint.
Example:
Event A = “Both coins show heads” → {(H, H)}
Event B = “Both coins show tails” → {(T, T)}
A and B are mutually exclusive — they cannot both be true on the same trial.
Counterexample:
Event C = “At least one head” → {(H, H), (H, T), (T, H)}
Event D = “First coin is heads” → {(H, H), (H, T)}
C and D are not mutually exclusive — they both include (H, H).
Check Your Understanding: Are These Events Mutually Exclusive?
Question 1:
Event A: “Roll a 4 on a die”
Event B: “Roll an even number on a die”
Question 2:
Event A: “Draw a red card from a deck”
Event B: “Draw a heart from a deck”
Question 3:
Event A: “Student is in 10th grade”
Event B: “Student is in 11th grade”
Related Video - Mutually Exclusive
Probability as a Value
Definition revisit: Probability
We defined probability in 4.1 as the following:
Probability is the long-run relative frequency of an event. This is the proportion of times an outcome would occur if we repeated a random process over and over, under the same conditions.
Now that we have a definition of an event we can meaningfully use this idea. To make it more concrete we add the following conditions to the probability of an event:
Probabilities must be recorded as a real number between 0 and 1. A probability of 0 means that an event cannot occur and a probability of 1 indicates that an event is guaranteed to occur.
If we have denoted an event as \(A\), we write probabilities as: \( P(A) \), which means "the probability that event \(A\) occurs".
Basic Examples
Experiment: We flip two coins
Sample space: \(\{(H,H), (H,T), (T, H), (T,T)\}\)
Event: We get two heads; denoted as \(H2\)
Probability: Looking at the sample space we see there are 4 possible outcomes. However, only one of these outcomes yields two heads, when we have \((H,H)\). Since all outcomes are equally likely, we have a 1 in 4 change of this event occurring. We would say \(P(H2) = 1/4 = 0.25\). It is okay to represent probabilities as either fractions or decimals, although one notation may be preferable for communication.
Experiment: We roll a die
Sample space: \(\{1, 2, 3, 4, 5, 6\}\)
Event: We get a roll of 5 or higher. We can denote this by \(R\geq 5\) where \(R\) is our roll.
Probability: Since there are two possible rolls in this event, 5 or 6, out of a total of 6 outcomes in the sample space, we have \(P(R \geq 5) = 2/6 = 0.3333\)
Building Events from Events
Sometimes, we define new events by combining existing ones. These compound events use the logic of “or”, "and", and “not”.
- "A or B" (written as \( A \cup B \)) means event \(A\) happens, or event \(B\) happens, or both. It can be considered "at least one of \(A\) or \(B\) happens.
- "A and B" (written as \(A \cup B\)) means event \(A\) and \(B\) both occur. Notice that if \(A\) and \(B\) are disjoint, then \(P(A\cup B) = 0\) since they cannot simultaneously happen.
- "Not A" (written as \( A^c \)) means \(A\) does not occur. This is also called the complement of \(A\).
Definition: Complement
The complement of event A (written \( A^c \)) is the set of outcomes in the sample space that are not in A.
To show these concepts visually, we often use Venn diagrams. They’re helpful models for compound events.
The Addition Rule
The addition rule helps calculate the probability that event \(A\) or event \(B\) occurs:
\[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \]
If A and B are mutually exclusive → \( P(A \cap B) = 0 \), so:
\[ P(A \cup B) = P(A) + P(B) \]
Think About It:
Why do we subtract \( P(A \cap B) \) in the general addition rule? What would happen if we didn’t?
Examples Using the Addition Rule
Example 1: Events with Overlap (Not Mutually Exclusive)
In a group of 100 students:
- 40 have taken Algebra (Event A)
- 30 have taken Geometry (Event B)
- 15 have taken both Algebra and Geometry
Using the addition rule: \[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \] \[ = \frac{40}{100} + \frac{30}{100} - \frac{15}{100} = \frac{55}{100} = 0.55 \]
Answer 55% of students have taken Algebra or Geometry.
We can analyze this piece by piece too. There are 40 students who have taken Algebra, but 15 out of that forty who have taken Geometry too. That leaves 25 who have only taken Algebra. Looking at the 30 students with Geometry, there are 15 who have also taken Algebra and 15 who haven't. This gives us now three disjoint groups. 25 students who have taken Algebra only, 15 who have taken Geometry only, and 15 who have taken both. You can check that adding up these values gives us 55 students out of the 100. This is an effective technique, but more time consuming and conceptual than the addition formula.
Example 2: Mutually Exclusive Events
When rolling a fair die:
- Let A = “roll a 1” → \( P(A) = \frac{1}{6} \)
- Let B = “roll a 5” → \( P(B) = \frac{1}{6} \)
Answer Probability of rolling a 1 or a 5 is \( \frac{2}{6} = 0.333 \)
Try These: Addition Rule
Question 1: In a class of 50 students:
- 30 like movies (Event A)
- 20 like sports (Event B)
- 10 like both
Question 2: You flip a fair coin and roll a die:
- A = “get heads” → \( P = 0.5 \)
- B = “roll a 6” → \( P = \frac{1}{6} \)
Question 3: Rolling a die, let:
- A = “even number” → \{2, 4, 6\}
- B = “number less than 4” → \{1, 2, 3\}
What is the probability of A or B?
Related Video - The Addition Rule
Probability Rules So Far
- \( P(A) \in [0, 1] \)
- \( P(A^c) = 1 - P(A) \)
- \( P(A \cup B) = P(A) + P(B) - P(A \cap B) \)
- If A and B are mutually exclusive, then \( P(A \cap B) = 0 \)
Now that you’ve built a solid foundation with basic probability rules, you're ready to go further. In the next sections, we’ll explore one of the most important ideas in statistics and probability: independence, or what it means for one event to have no effect on the chance of another. These concepts allow us to calculate more complex probabilities and model real-world scenarios more accurately.