9.2: Outcomes, Type I and Type II Errors

Last updated
Save as PDF

Page ID: 16267

$ \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } $ $ \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} $$\newcommand{\id}{\mathrm{id}}$ $ \newcommand{\Span}{\mathrm{span}}$ $ \newcommand{\kernel}{\mathrm{null}\,}$ $ \newcommand{\range}{\mathrm{range}\,}$ $ \newcommand{\RealPart}{\mathrm{Re}}$ $ \newcommand{\ImaginaryPart}{\mathrm{Im}}$ $ \newcommand{\Argument}{\mathrm{Arg}}$ $ \newcommand{\norm}[1]{\| #1 \|}$ $ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $ \newcommand{\Span}{\mathrm{span}}$ $\newcommand{\id}{\mathrm{id}}$ $ \newcommand{\Span}{\mathrm{span}}$ $ \newcommand{\kernel}{\mathrm{null}\,}$ $ \newcommand{\range}{\mathrm{range}\,}$ $ \newcommand{\RealPart}{\mathrm{Re}}$ $ \newcommand{\ImaginaryPart}{\mathrm{Im}}$ $ \newcommand{\Argument}{\mathrm{Arg}}$ $ \newcommand{\norm}[1]{\| #1 \|}$ $ \newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $ \newcommand{\Span}{\mathrm{span}}$$\newcommand{\AA}{\unicode[.8,0]{x212B}}$

When you perform a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis H₀ and the decision to reject or not.

The outcomes are summarized in the following table:

	H₀ is actually
Action	True	False
Do not reject H₀	Correct Outcome	Type II Error ( $\beta$ )
Reject H₀	Type I Error ( $\alpha$ )	Correct Outcome

The four possible outcomes in the table are:

The decision is not to reject H₀ when H₀ is true (correct decision).
The decision is to reject H₀ when H₀ is true (incorrect decision known as a Type I error).
The decision is not to reject H₀when H₀ is false (incorrect decision known as a Type II error).
The decision is to reject H₀ when H₀ is false (correct decision whose probability is called the Power of the Test).

Each of the errors occurs with a particular probability. The Greek letters α and β represent the probabilities.

α = probability of a Type I error = P(Type I error) = probability of rejecting the null hypothesis when the null hypothesis is true.

β = probability of a Type II error = P(Type II error) = probability of not rejecting the null hypothesis when the null hypothesis is false.

α and β should be as small as possible because they are probabilities of errors. They are rarely zero.

The Power of the Test is 1 – β.
Since β is probability of making type II error, we want this probability to be small.
In other words, we want the value 1 – β to be as closed to one as possible.
Increasing the sample size can increase the Power of the Test.

Example 1

Suppose the null hypothesis, H₀, is: Frank’s rock climbing equipment is safe.

Type I error: Frank thinks that his rock climbing equipment may not be safe when, in fact, it really is safe.
Type II error: Frank thinks that his rock climbing equipment may be safe when, in fact, it is not safe.

α = Probability that Frank thinks his rock climbing equipment may not be safe when it really is safe.
β = Probability that Frank thinks his rock climbing equipment may be safe when it is not safe.

	Null Hypothesis: The rock climbing equipment is safe.
Frank’s decision	True (The equipment is safe)	False (The equipment is not safe.)
Not reject H₀	Correct decision	Type II Error
Reject H₀	Type I Error	Correct decision

Notice that, in this case, the error with the greater consequence is the Type II error.
(If Frank thinks his rock climbing equipment is safe, he will go ahead and use it.)

Try It

Suppose the null hypothesis, H₀, is: the blood cultures contain no traces of pathogen X.
State the Type I and Type II errors.

[practice-area rows=”2″][/practice-area]
[reveal-answer q=”313018″]Solution[/reveal-answer]
[hidden-answer a=”313018″]

Type I error: The researcher thinks the blood cultures do contain traces of pathogen X, when in fact, they do not.
Type II error: The researcher thinks the blood cultures do not contain traces of pathogen X, when in fact, they do.

	Null hypothesis: The blood cultures contain no traces of pathogen X.
Decision (What the researcher thinks…)	True (The blood cultures contain no traces of pathogen X. )	False (The blood cultures contain traces of pathogen X.)
Not reject H₀	Correct decision	Type II Error
Reject H₀	Type I Error	Correct decision

[/hidden-answer]

Example 2

Suppose the null hypothesis, H₀: The victim of an automobile accident is alive when he arrives at the emergency room of a hospital.
State the 4 possible outcomes of performing a hypothesis test.

Solution:

α = probability that the emergency crew thinks the victim is dead when, in fact, he is really alive = P(Type I error).
β = probability that the emergency crew does not know if the victim is alive when, in fact, the victim is dead =P(Type II error).

	Null hypothesis: The victim’s situation is alive.
Decision	True (The victim is alive.)	False (The victim is dead.)
Not reject H₀	Correct decision	Type II Error
Reject H₀	Type I Error	Correct decision

The error with the greater consequence is the Type I error. (If the emergency crew thinks the victim is dead, they will not treat him.)

Thumbnail for the embedded element "Type 1 errors | Inferential statistics | Probability and Statistics | Khan Academy"

A YouTube element has been excluded from this version of the text. You can view it online here: http://pb.libretexts.org/esm/?p=120

Try It

Suppose the null hypothesis, H₀, is: A patient is not sick. Which type of error has the greater consequence, Type I or Type II?

[practice-area rows=”1=2″][/practice-area]
[reveal-answer q=”3933″]Solution[/reveal-answer]
[hidden-answer a=”3933″]

Type I Error: The patient will not be thought well when, in fact, he is not sick.

Type II Error: The patient will be thought well when, in fact, he is sick.

	Null hypothesis: A patient is not sick.
Decision	True (The patient is not sick.)	False (The patient is sick.)
Not reject H₀	Correct decision	Type II Error
Reject H₀	Type I Error	Correct decision

The error with the greater consequence is the Type II error: the patient will be thought well when, in fact, he is sick.
He will not be able to get treatment.[/hidden-answer]

Example 3

Boy Genetic Labs claim to be able to increase the likelihood that a pregnancy will result in a boy being born.
Statisticians want to test the claim.
Suppose that the null hypothesis, H₀, is: Boy Genetic Labs has no effect on gender outcome.
Which type of error has the greater consequence, Type I or Type II?

Solution:

H₀: Boy Genetic Labs has no effect on gender outcome.

H_a: Boy Genetic Labs has effect on gender outcome.

	Null Hypothesis : Boy Genetic Labs has no effect on gender outcome.
Decision	True (No Effect)	False (Effect)
Not reject H₀	Correct decision	Type II Error
reject H₀	Type I Error	Correct decision

Type I error: This results when a true null hypothesis is rejected. In the context of this scenario, we would state that we believe that Boy Genetic Labs influences the gender outcome, when in fact it has no effect.
The probability of this error occurring is denoted by the Greek letter alpha, α.
Type II error: This results when we fail to reject a false null hypothesis. In context, we would state that Boy Genetic Labs does not influence the gender outcome of a pregnancy when, in fact, it does.
The probability of this error occurring is denoted by the Greek letter beta, β.

The error of greater consequence would be the Type I error since couples would use the Boy Genetic Labs product in hopes of increasing the chances of having a boy.

Try It

“Red tide” is a bloom of poison-producing algae–a few different species of a class of plankton called dinoflagellates. When the weather and water conditions cause these blooms, shellfish such as clams living in the area develop dangerous levels of a paralysis-inducing toxin. In Massachusetts, the Division of Marine Fisheries (DMF) monitors levels of the toxin in shellfish by regular sampling of shellfish along the coastline. If the mean level of toxin in clams exceeds 800 μg (micrograms) of toxin per kg of clam meat in any area, clam harvesting is banned there until the bloom is over and levels of toxin in clams subside.
Describe both a Type I and a Type II error in this context, and state which error has the greater consequence.

[reveal-answer q=”432609″]Solution[/reveal-answer]
[hidden-answer a=”432609″]

In this scenario, an appropriate null hypothesis would be
H₀: the mean level of toxins is at most 800 μg. ( H₀ : μ₀ ≤ 800 μg )
H_a: the mean level of toxins exceeds 800 μg. (H_a: μ₀ > 800 μg )

Type I error: The DMF believes that toxin levels are still too high when, in fact, toxin levels are at most 800 μg.
The DMF continues the harvesting ban.
Type II error: The DMF believes that toxin levels are within acceptable levels (are at least 800 μg) when, in fact, toxin levels are still too high (more than 800 μg).
The DMF lifts the harvesting ban.

	Null Hypothesis: The mean level of toxins is at most 800 μg.
Decision	True	False
Not reject H₀	Correct decision	Type II Error
Reject H₀	Type I Error	Correct decision

This error could be the most serious. If the ban is lifted and clams are still toxic, consumers could possibly eat tainted food. In summary, the more dangerous error would be to commit a Type II error, because this error involves the availability of tainted clams for consumption.[/hidden-answer]

Example 4

A certain experimental drug claims a cure rate of higher than 75% for males with prostate cancer.

Describe both the Type I and Type II errors in context. Which error is the more serious?

Solution:

H₀: The cure rate is less than 75%.

H_a: The cure rate is higher than 75%.

	Null hypothesis (The cure rate is less than 75%.)
Decision	True (The cure rate is less than 75%.)	False (The cure rate is higher than 75%.)
Not reject H₀	Correct	Type II Error
Reject H₀	Type I Error	Correct

Type I: A cancer patient believes the cure rate for the drug is more than 75% when the cure rate actually is less than 75%.
Type II: A cancer patient believes the the cure rate is less than 75% cure rate when the cure rate is actually higher than 75%.

In this scenario, the Type II error contains the more severe consequence. If a patient believes the drug works at least 75% of the time, this most likely will influence the patient’s (and doctor’s) choice about whether to use the drug as a treatment option.

Try It

Determine both Type I and Type II errors for the following scenario:

Assume a null hypothesis, H₀, that states the percentage of adults with jobs is at least 88%.

Identify the Type I and Type II errors from these four statements.

a)Not to reject the null hypothesis that the percentage of adults who have jobs is at least 88% when that percentage is actually less than 88%

b)Not to reject the null hypothesis that the percentage of adults who have jobs is at least 88% when the percentage is actually at least 88%.

c)Reject the null hypothesis that the percentage of adults who have jobs is at least 88% when the percentage is actually at least 88%.

d)Reject the null hypothesis that the percentage of adults who have jobs is at least 88% when that percentage is actually less than 88%.

[reveal-answer q=”76062″]Type I error:[/reveal-answer]
[hidden-answer a=”76062″]c [/hidden-answer]
[reveal-answer q=”975662″]Type II error:[/reveal-answer]
[hidden-answer a=”975662″]b[/hidden-answer]

Example 5

Determine both Type I and Type II errors for the following scenario:

Assume a null hypothesis, H₀, that states the percentage of adults with jobs is at least 88%.

Identify the Type I and Type II errors from these four statements.

a)Not to reject the null hypothesis that the percentage of adults who have jobs is at least 88% when that percentage is actually less than 88%

b)Not to reject the null hypothesis that the percentage of adults who have jobs is at least 88% when the percentage is actually at least 88%.

c)Reject the null hypothesis that the percentage of adults who have jobs is at least 88% when the percentage is actually at least 88%.

d)Reject the null hypothesis that the percentage of adults who have jobs is at least 88% when that percentage is actually less than 88%.

Solution:

If H₀: The percentage of adults with jobs is at least 88%, then H_a: The percentage of adults with jobs is less than 88%.

[reveal-answer q=”864484″]Type I error: [/reveal-answer]
[hidden-answer a=”864484″]c [/hidden-answer]
[reveal-answer q=”126260″]Type II error:[/reveal-answer]
[hidden-answer a=”126260″]b[/hidden-answer]

Concept Review

In every hypothesis test, the outcomes are dependent on a correct interpretation of the data. Incorrect calculations or misunderstood summary statistics can yield errors that affect the results. A Type I error occurs when a true null hypothesis is rejected. A Type II error occurs when a false null hypothesis is not rejected.

The probabilities of these errors are denoted by the Greek letters α and β, for a Type I and a Type II error respectively. The power of the test, 1 – β, quantifies the likelihood that a test will yield the correct result of a true alternative hypothesis being accepted. A high power is desirable.

Formula Review

α = probability of a Type I error = P(Type I error) = probability of rejecting the null hypothesis when the null hypothesis is true.

β = probability of a Type II error = P(Type II error) = probability of not rejecting the null hypothesis when the null hypothesis is false.

OpenStax, Statistics, Outcomes and the Type I and Type II Errors. Provided by: OpenStax. Located at: http://cnx.org/contents/30189442-6998-4686-ac05-ed152b91b9de@17.44. License: CC BY: Attribution
Introductory Statistics . Authored by: Barbara Illowski, Susan Dean. Provided by: Open Stax. Located at: http://cnx.org/contents/30189442-6998-4686-ac05-ed152b91b9de@17.44. License: CC BY: Attribution. License Terms: Download for free at http://cnx.org/contents/30189442-699...2b91b9de@17.44

Type 1 errors | Inferential statistics | Probability and Statistics | Khan Academy. Authored by: Khan Academy. Located at: https://youtu.be/EowIec7Y8HM. License: All Rights Reserved. License Terms: Standard YouTube License