Introduction to Probability
- Page ID
- 31301
CO-6: Apply basic concepts of probability, random variation, and commonly used statistical probability distributions.
Video: Probability Introduction (7:41)
Now that we understand how probability fits into the Big Picture as a key element behind statistical inference, we are ready to learn more about it. Our first goal is to introduce some fundamental terminology (the language) and notation that is used when discussing probability.
Probability is Not Always Intuitive
Although most of the probability calculations we will conduct will be rather intuitive due to their simplicity, we start with two fun examples that will illustrate the interesting and sometimes complex nature of probability.
Often, relying only on our intuition is not enough to determine probability, so we’ll need some tools to work with, which is exactly what we’ll study in this section.
For the next two examples, do not be concerned with the solution of the problem. Only how the answers to probability questions are not always easy to believe or determine.
Here is the first of two motivating examples:
“Let’s Make a Deal” was the name of a popular television game show, which first aired in the 1960s. The “Let’s Make a Deal” Paradox is named after that show. In the show, the contestant had to choose between three doors. One of the doors had a big prize behind it such as a car or a lot of cash, and the other two were empty. (Actually, for entertainment’s sake, each of the other two doors had some stupid gift behind it, like a goat or a chicken, but we’ll refer to them here as empty.)
The contestant had to choose one of the three doors, but instead of revealing the chosen door, the host revealed one of the two unchosen doors to be empty. At this point of the game, there were two unopened doors (one of which had the prize behind it) — the door that the contestant had originally chosen and the remaining unchosen door.
The contestant was given the option either to stay with the door that he or she had initially chosen, or switch to the other door.
What do you think the contestant should do, stay or switch? What do you think is the probability that you will win the big prize if you stay? What about if you switch?
In order for you to gain a feel for this game, you can play it a few times using an applet.
Interactive Applet: Let’s Make a Deal
Now, what do you think a contestant should do?
Learn By Doing: Let’s Make a Deal
The intuition of most people is that the chance of winning is equal whether we stay or switch — that there is a 50-50 chance of winning with either selection. This, however, is not the case.
Actually, there is a 67% chance — or a probability of 2/3 (2 out of three) — of winning by switching, and only a 33% chance — or a probability of 1/3 (1 out of 3) — of winning by staying with the door that was originally chosen.
This means that a contestant is twice as likely to win if he/she switches to the unchosen door. Isn’t this a bit counterintuitive and confusing? Most people think so, when they are first faced with this problem.
We will now try to explain this paradox to you in two different ways:
Video: Let’s Make a Deal (Explanation #1) (1:10)
If you are still not convinced (or even if you are), here is a different way of explaining the paradox:
Video: Let’s Make a Deal (Explanation #2) (1:37)
If this example still did not persuade you that probability is not always intuitive, the next example should definitely do the trick.
Suppose that you are at a party with 59 other people (for a total of 60). What are the chances (or, what is the probability) that at least 2 of the 60 guests share the same birthday?
To clarify, by “share the same birthday,” we mean that 2 people were born on the same date, not necessarily in the same year. Also, for the sake of simplicity, ignore leap years, and assume that there are 365 days in each year.
Learn By Doing: Birthday Problem
Indeed, there is a 99.4% chance that at least 2 of the 60 guests share the same birthday. In other words, it is almost certain that at least 2 of the guests share the same birthday. This is very counterintuitive.
Unlike the “Let’s Make a Deal” example, for this scenario, we don’t really have a good step-by-step explanation that will give you insight into this surprising answer.
From these two examples, (maybe) you have seen that your original hunches cannot always be counted upon to give you correct predictions of probabilities.
We won’t think any more about these examples as they are from the “harder” end of the complexity spectrum but hopefully they have motivated you to learn more about probability and you do not need to be convinced of their solution to continue!
In general, probability is not always intuitive.
Need a Laugh?
Watch this (funny) video which has an excellent point about “how probability DOES NOT work”: clip from the Daily Show with Jon Stewart about the Large Hadron Collider (5:58).
It is possible viewers in other countries may not be able to view the clip from this source. You may or may not be able to find it online through searching. Here is the transcript summary I sometimes use in class to get the point across (it isn’t quite as funny but I think you can still figure out what is wrong here):
- John Oliver: So, roughly speaking, what are the chances that the world is going to be destroyed? (by the large hadron collider) One-in-a-million? One-in-a-billion?
- Walter: Well, the best we can say right now is about a one-in-two chance.
- John Oliver: 50-50?
- Walter: Yeah, 50-50… It’s a chance; it’s a 50-50 chance.
- John Oliver: You keep coming back to this 50-50 thing, it’s weird Walter.
- Walter: Well, if you have something that can happen and something that won’t necessarily happen, it’s going to either happen or it’s going to not happen. And, so, it’s … the best guess is 1 in 2.
- John Oliver: I’m not sure that’s how probability works, Walter.
And … John Oliver is correct! :-)
What is Probability?
LO 6.4: Relate the probability of an event to the likelihood of this event occurring.
Eventually we will need to develop a more formal approach to probability, but we will begin with an informal discussion of what probability is.
Probability is a mathematical description of randomness and uncertainty. It is a way to measure or quantify uncertainty. Another way to think about probability is that it is the official name for “chance.”
Probability is the Likelihood of Something Happening
One way to think of probability is that it is the likelihood that something will occur.
Probability is used to answer the following types of questions:
- What is the chance that it will rain tomorrow?
- What is the chance that a stock will go up in price?
- What is the chance that I will have a heart attack?
- What is the chance that I will live longer than 70 years?
- What is the likelihood that when rolling a pair of dice, I will roll doubles?
- What is the probability that I will win the lottery?
- What is the probability that I will become diabetic?
Each of these examples has some uncertainty. For some, the chances are quite good, so the probability would be quite high. For others, the chances are not very good, so the probability is quite low (especially winning the lottery).
Certainly, the chance of rain is different each day, and is higher during some seasons. Your chance of having a heart attack, or of living longer than 70 years, depends on things like your current age, your family history, and your lifestyle. However, you could use your intuition to predict some of those probabilities fairly accurately, while others you might have no instinct about at all.
Notation
We think you will agree that the word probability is a bit long to include in equations, graphs and charts, so it is customary to use some simplified notation instead of the entire word.
If we wish to indicate “the probability it will rain tomorrow,” we use the notation “P(rain tomorrow).” We can abbreviate the probability of anything. If we let A represent what we wish to find the probability of, then P(A) would represent that probability.
We can think of “A” as an “event.”
| NOTATION | MEANING |
| P(win lottery) | the probability that a person who has a lottery ticket will win that lottery |
| P(A) | the probability that event A will occur |
| P(B) | the probability that event B will occur |
PRINCIPLE: The “probability” of an event tells us how likely it is that the event will occur.
What values can the probability of an event take, and what does the value tell us about the likelihood of the event occurring?
Video: Basic Properties of Probability (0:53)
Did I Get This?: Basic Properties of Probability
PRINCIPLE: The probability that an event will occur is between 0 and 1 or 0 ≤ P(A) ≤ 1.
Many people prefer to express probability in percentages. Since all probabilities are decimals, each can be changed to an equivalent percentage. Thus, the latest principle is equivalent to saying, “The chance that an event will occur is between 0% and 100%.”
Probabilities can be determined in two fundamental ways. Keep reading to find out what they are.
Determining Probability
There are 2 fundamental ways in which we can determine probability:
- Theoretical (also known as Classical)
- Empirical (also known as Observational)
Classical methods are used for games of chance, such as flipping coins, rolling dice, spinning spinners, roulette wheels, or lotteries.
The probabilities in this case are determined by the game (or scenario) itself and are often found relatively easily using logic and/or probability rules.
Although we will not focus on this type of probability in this course, we will mention a few examples to get you thinking about probability and how it works.
A coin has two sides; we usually call them “heads” and “tails.”
For a “fair” coin (one that is not unevenly weighted, and does not have identical images on both sides) the chances that a “flip” will result in either side facing up are equally likely.
Thus, P(heads) = P(tails) = 1/2 or 0.5.
Letting H represent “heads,” we can abbreviate the probability: P(H) = 0.5.
Classical probabilities can also be used for more realistic and useful situations.
A practical use of a coin flip would be for you and your roommate to decide randomly who will go pick up the pizza you ordered for dinner. A common expression is “Let’s flip for it.” This is because a coin can be used to make a random choice with two options. Many sporting events begin with a coin flip to determine which side of the field or court each team will play on, or which team will have control of the ball first.
Each traditional (cube-shaped) die has six sides, marked in dots with the numbers 1 through 6.
On a “fair” die, these numbers are equally likely to end up face-up when the die is rolled.
Thus, P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6 or about 0.167.
Here, again, is a practical use of classical probability.
Suppose six people go out to dinner. You want to randomly decide who will pick up the check and pay for everyone. Again, the P(each person) = 1/6.
This particular spinner has three colors, but each color is not equally likely to be the result of a spin, since the portions are not the same size.
Since the blue is half of the spinner, P(blue) = 1/2. The red and yellow make up the other half of the spinner and are the same size. Thus, P(red) = P(yellow) = 1/4.
Suppose there are 2 freshmen, 1 sophomore, and one junior in a study group. You want to select one person. The P(F) = 2/4 = 1/2; P(S) = 1/4; and P(J) = 1/4, just like the spinner.
Suppose we had three students and wished to select one of them randomly. To do this you might have each person write his/her name on a (same-sized) piece of paper, then put the three papers in a hat, and select one paper from the hat without looking.
Since we are selecting randomly, each is equally likely to be chosen. Thus, each has a probability of 1/3 of being chosen.
A slightly more complicated, but more interesting, probability question would be to propose selecting 2 of the students pictured above, and ask, “What is the probability that the two students selected will be different genders?”
We will now shift our discussion to empirical ways to determine probabilities.
A Question
A single flip of a coin has an uncertain outcome. So, every time a coin is flipped, the outcome of that flip is unknown until the flip occurs.
However, if you flip a fair coin over and over again, would you expect P(H) to be exactly 0.5? In other words, would you expect there to be the same number of results of “heads” as there are “tails”?
The following activity will allow you to discover the answer.
Learn By Doing: Empirical Probability #1
The above Learn by Doing activity was our first example of the second way of determining probability: Empirical (Observational) methods. In the activity, we determined that the probability of getting the result “heads” is 0.5 by flipping a fair coin many, many times.
A Second Question
After doing this experiment, an important question naturally comes to mind. How would we know if the coin was not fair? Certainly, classical probability methods would never be able to answer this question. In addition, classical methods could never tell us the actual P(H). The only way to answer this question is to perform another experiment.
The next activity will allow you to do just that.
Learn By Doing: Empirical Probability #2
So, these types of experiments can verify classical probabilities and they can also determine when games of chance are not following fair practices. However, their real importance is to answer probability questions that arise when we are faced with a situation that does not follow any pattern and cannot be predetermined. In reality, most of the probabilities of interest to us fit the latter description.
To Summarize So Far
- Probability is a way of quantifying uncertainty.
- We are interested in the probability of an event — the likelihood of the event occurring.
- The probability of an event ranges from 0 to 1. The closer the probability is to 0, the less likely the event is to occur. The closer the probability is to 1, the more likely the event is to occur.
- There are two ways to determine probability: Theoretical (Classical) and Empirical (Observational).
- Theoretical methods use the nature of the situation to determine probabilities.
- Empirical methods use a series of trials that produce outcomes that cannot be predicted in advance (hence the uncertainty).
Relative Frequency
LO 6.5: Apply the relative frequency approach to estimate the probability of an event.
If we toss a coin, roll a die, or spin a spinner many times, we hardly ever achieve the exact theoretical probabilities that we know we should get, but we can get pretty close. When we run a simulation or when we use a random sample and record the results, we are using empirical probability. This is often called the Relative Frequency definition of probability.
Here is a realistic example where the relative frequency method was used to find the probabilities:
Researchers discovered at the beginning of the 20th century that human blood comes in various types (A, B, AB, and O), and that some types are more common than others. How could researchers determine the probability of a particular blood type, say O?
Just looking at one or two or a handful of people would not be very helpful in determining the overall chance that a randomly chosen person would have blood type O. But sampling many people at random, and finding the relative frequency of blood type O occurring, provides an adequate estimate.
For example, it is now well known that the probability of blood type O among white people in the United States is 0.45. This was found by sampling many (say, 100,000) white people in the country, finding that roughly 45,000 of them had blood type O, and then using the relative frequency: 45,000 / 100,000 = 0.45 as the estimate for the probability for the event “having blood type O.”
(Comment: Note that there are racial and ethnic differences in the probabilities of blood types. For example, the probability of blood type O among black people in the United States is 0.49, and the probability that a randomly chosen Japanese person has blood type O is only 0.3).
Let’s review the relative frequency method for finding probabilities:
To estimate the probability of event A, written P(A), we may repeat the random experiment many times and count the number of times event A occurs. Then P(A) is estimated by the ratio of the number of times A occurs to the number of repetitions, which is called the relative frequency of event A.
Did I Get This?: Relative Frequency
Learn By Doing: Relative Frequency
So, we’ve seen how the relative frequency idea works, and hopefully the activities have convinced you that the relative frequency of an event does indeed approach the theoretical probability of that event as the number of repetitions increases. This is called the Law of Large Numbers.
The Law of Large Numbers states that as the number of trials increases, the relative frequency becomes the actual probability. So, using this law, as the number of trials increases, the empirical probability gets closer and closer to the theoretical probability.
PRINCIPLE: Law of Large Numbers – The actual (or true) probability of an event (A) is estimated by the relative frequency with which the event occurs in a long series of trials.
Interactive Applet: Law of Large Numbers
Comments:
- Note that the relative frequency approach provides only an estimate of the probability of an event. However, we can control how good this estimate is by the number of times we repeat the random experiment. The more repetitions that are performed, the closer the relative frequency gets to the true probability of the event.
- One interesting question would be: “How many times do I need to repeat the random experiment in order for the relative frequency to be, say, within 0.001 of the actual probability of the event?” We will come back to that question in the inference section.
- A pedagogical comment: We’ve introduced relative frequency here in a more practical approach, as a method for estimating the probability of an event. More traditionally, relative frequency is not presented as a method, but as a definition:
Relative Frequency: (Definition) The probability of an event (A) is the relative frequency with which the event occurs in a long series of trials.
4. There are many situations of interest in which physical circumstances do not make the probability obvious. In fact, most of the time it is impossible to find the theoretical probability, and we must use empirical probabilities instead.
Let’s Summarize
Probability is a way of quantifying uncertainty. In this section, we defined probability as the likelihood or chance that something will occur and introduced the basic notation of probability such as P(win lottery).
You have seen that all probabilities are values between 0 and 1, where an event with no chance of occurring has a probability of 0 and an event which will always occur has a probability of 1.
We have discussed the two primary methods of calculating probabilities
- Theoretical or Classical Probability: uses the nature of the situation to determine probabilities
- Empirical or Observational Probability: uses a series of trials that produce outcomes that cannot be predicted in advance (hence the uncertainty)
In our course we will focus on Empirical probability and will often calculate probabilities from a sample using relative frequencies.
This is useful in practice since the Law of Large Numbers allows us to estimate the actual (or true) probability of an event by the relative frequency with which the event occurs in a long series of trials. We can collect this information as data and we can analyze this data using statistics.