8.2: Type I and II Errors

Learning Objectives

• Understand the concepts of Type I and Type II errors in hypothesis testing.
• Identify a Type I error as rejecting a true null hypothesis (false positive).
• Identify a Type II error as failing to reject a false null hypothesis (false negative).
• Recognize how significance levels (α) influence the risk of Type I errors.
• Understand how sample size and study design affect the likelihood of Type II errors.
• Learn strategies to manage and minimize the risks of both error types in hypothesis testing.

How do you quantify small? Is 5%, 10%, or 15% really small? How do you decide? That depends on your field of study and the importance of the situation. Is this a pilot study? Is someone’s life at risk? Would you lose your job? Most industry standards use 5% as the cutoff point for how small is small enough, but 1%, 5%, and 10% are frequently used depending on what the situation calls for.

Now, how small is small enough? To answer that you want to know the types of errors you can make in hypothesis testing.

The first error is if you say that H₀ is false when in fact it is true. This means you reject H₀ when H₀ is true. The second error is if you say that H₀ is true when in fact it is false. This means you fail to reject H₀ when H₀ is false.

Figure 8-4 shows that if we “Reject H₀ ” when H₀ is true, we are committing a Type I error. The probability of committing a Type I error is the Greek letter \(\alpha\), pronounced alpha. This can be controlled by the researcher by choosing a specific level of significance \(\alpha\).

Figure \(\PageIndex{1}\): Matrix for Types of Errors

Figure 8-4 shows that if we “Do Not Reject H₀ ” when H₀ is false, we are committing a Type II error. The probability of committing a Type II error is denoted by the Greek letter β, pronounced beta. When we increase the sample size, this will reduce β. The power of a test is 1 – β.

A jury trial is about to take place to decide if a person is guilty of committing murder. The hypotheses for this situation would be:

• \(H_0\): The defendant is innocent
• \(H_1\): The defendant is not innocent

The jury has two possible decisions to make: either acquit or convict the person on trial, based on the evidence that is presented. There are two possible ways that the jury could make a mistake. They could convict an innocent person or they could let a guilty person go free. Both are bad news, but if the death penalty is imposed on the convicted person, the justice system could be killing an innocent person. If a murderer is let go without enough evidence to convict them, then they could murder again. In statistics, we call these two types of mistakes Type I and Type II errors.

Figure 8-5 is a diagram showing the four possible jury decisions and two errors.

Figure \(\PageIndex{2}\): Metaphorical Example of Court Case Results using Type I and II Errors

Definition: Type I and II Errors

Type I Error is rejecting H₀ when H₀ is true, and Type II Error is failing to reject H₀ when H₀ is false.

Since these are the only two possible errors, one can define the probabilities attached to each error.

\(\alpha\) = P(Type I Error) = P(Rejecting H₀ | H₀ is true)

β = P(Type II Error) = P(Failing to reject H₀ | H₀ is false)

Example \(\PageIndex{1}\)

An investment company wants to build a new food cart. They know from experience that food carts are successful if they have on average more than 100 people a day walk by the location. They have a potential site to build on, but before they begin, they want to see if they have enough foot traffic. They observe how many people walk by the site every day over a month. They will build if there is more than an average of 100 people who walk by the site each day. In simple terms, explain what the type I & II errors would be using context from the problem.

Solution

The hypotheses are: H₀: μ = 100 and H₁: μ > 100.

Sometimes it is helpful to use words next to your hypotheses instead of formal symbols

• H₀: μ ≤ 100 (Do not build)
• H₁: μ > 100 (Build).

A Type I error would be to reject the null hypothesis when in fact it is true. Take your finger and cover up the null hypothesis (we decide to reject the null), then what is showing? The alternative hypothesis is what action we take.

If we reject H_0, then we will build the new food cart. However, H₀ was true, which means that the mean was less than or equal to 100 people walking by.

In simpler terms, this would mean that our evidence showed that we have enough foot traffic to support the food cart. Once we built, though, there were not, on average, more than 100 people who walked by, and the food cart may fail.

A type II error would be to fail to reject the null when in fact the null is false. Evidence shows that we should not build on the site, but this actually would have been a prime location to build on.

The missed opportunity of a type II error is not as bad as possibly losing thousands of dollars on a bad investment.

What is more severe of an error is dependent on what side of the desk you are sitting on. For instance, if a hypothesis is about miles per gallon for a new car the hypotheses may be set up differently depending on if you are buying the car or selling the car. For this course, the claim will be stated in the problem and always set up the hypotheses to match the stated claim. In general, the research question should be set up as some type of change in the alternative

Controlling for Type I Error

The significance level used by the researcher should be picked before the collection and analysis of data. This is called “a priori,” versus picking α after you have done your analysis, which is called “post hoc.” When deciding on what significance level to pick, one needs to look at the severity of the consequences of the type I and type II errors. For example, if the type I error may cause the loss of life or large amounts of money, the researcher would want to set \(\alpha\) low.

Controlling for Type II Error

The power of a test is the complement of a Type II error or correctly rejecting a false null hypothesis. You can increase the power of the test and hence decrease the type II error by increasing the sample size. Similar to confidence intervals, we can reduce our margin of error when we increase the sample size. In general, we would like to have a high confidence level and a high power for our hypothesis tests. When you increase your confidence level, then in turn the power of the test will decrease. Calculating the probability of a Type II error is a little more difficult, and it is a conditional probability based on the researcher’s hypotheses and is not discussed in this course.

8.2.1 Finding Critical Values

A researcher decides they want to have a 5% chance of making a Type I error, so they set α = 0.05. What z-score would represent that 5% area? It would depend on if the hypotheses were a left-tailed, two-tailed, or right-tailed test. This z-score is called a critical value. Figure 8-8 shows examples of critical values for the three possible sets of hypotheses.

Figure \(\PageIndex{3}\): Critical Regions for Different Types of Tests.

Two-tailed Test

If we are doing a two-tailed test then the \(\alpha\) = 5% area gets divided into both tails. We denote these critical values \(z_{\alpha / 2}\) and \(z_{1-\alpha / 2}\). When the sample data finds a z-score (test statistic) that is either less than or equal to \(z_{\alpha / 2}\) or greater than or equal to \(z_{1-\alpha / 2}\) then we would reject H₀. The area to the left of the critical value \(z_{\alpha / 2}\) and to the right of the critical value \(z_{1-\alpha / 2}\) is called the critical or rejection region. See Figure 8-9.

Figure \(\PageIndex{4}\): Standard Normal Distribution with Critical Regions and Critical Values.

When \(\alpha\) = 0.05 then the critical values \(z_{\alpha / 2}\) and \(z_{1-\alpha / 2}\) are found using the following technology.

Excel: \(z_{\alpha / 2}\) =NORM.S.INV(0.025) = –1.96 and \(z_{1-\alpha / 2}\) =NORM.S.INV(0.975) = 1.96

TI-Calculator: \(z_{\alpha / 2}\) = invNorm(0.025,0,1) = –1.96 and \(z_{1-\alpha / 2}\) = invNorm(0.975,0,1) = 1.96

Since the normal distribution is symmetric, you only need to find one side’s z-score and we usually represent the critical values as ± \(z_{\alpha / 2}\).

Most of the time, we will be finding a probability (p-value) instead of the critical values. The p-value and critical values are related and tell the same information, so it is important to know what a critical value represents.

Right-tailed Test

If we are doing a right-tailed test then the \(\alpha\) = 5% area goes into the right tail. We denote this critical value \(z_{1-\alpha}\). When the sample data finds a z-score more than \(z_{1-\alpha}\), then we would reject H₀; reject H₀ if the test statistic is ≥ \(z_{1-\alpha}\). The area to the right of the critical value \(z_{1-\alpha}\) is called the critical region. See Figure 8-10.

Figure \(\PageIndex{5}\): Standard Normal Distribution with a Right-Tailed Critical Region

When \(\alpha\) = 0.05 then the critical value \(z_{1-\alpha}\) is found using the following technology.

Excel: \(z_{1-\alpha}\) =NORM.S.INV(0.95) = 1.645 Figure 8-10

TI-Calculator: \(z_{1-\alpha}\) = invNorm(0.95,0,1) = 1.645

Left-tailed Test

If we are doing a left-tailed test then the \(\alpha\) = 5% area goes into the left tail. If the sampling distribution is normal, then we can use the inverse normal function in Excel or a calculator to find the corresponding z-score. We denote this critical value \(z_{\alpha}\).

When the sample data finds a z-score less than \(z_{\alpha}\), then we would reject H0, and reject Ho if the test statistic is ≤ \(z_{\alpha}\). The area to the left of the critical value \(z_{\alpha}\) is called the critical region. See Figure 8-11.

Figure \(\PageIndex{6}\): Standard Normal Distribution with a Left-Tailed Critical Region

When \(\alpha\) = 0.05 then the critical value \(z_{\alpha}\) is found using the following technology.

Excel: \(z_{\alpha}\) =NORM.S.INV(0.05) = –1.645

TI-Calculator: \(z_{\alpha}\) = invNorm(0.05,0,1) = –1.645

The Claim and Summary

The wording of the summary statement changes depending on which hypothesis the researcher claims to be true. We really should always be setting up the claim in the alternative hypothesis, since most of the time we are collecting evidence to show that a change has occurred, but occasionally a textbook will have the claim in the null hypothesis. Do not use the phrase “accept H₀” since this implies that H₀ is true. The lack of evidence is not evidence of anything.

There were only two possible correct answers for the decision step.

i. Reject H₀

ii. Fail to reject H₀

Caution! If we fail to reject the null, this does not mean that there was no change; we just do not have any evidence that change has occurred. The absence of evidence is not evidence of absence. On the other hand, we need to be careful when we reject the null hypothesis, we have not proved that there is a change.

When we reject the null hypothesis, there is only evidence that a change has occurred. Our evidence could have been false and led to an incorrect decision. If we use the phrase, “accept H_0,” this implies that H₀ was true, but we just do not have evidence that it is false. Hence, you will be marked incorrect for your decision if you use accept H₀, use instead “fail to reject H_0,” or “do not reject H₀.”

Authors

"8.2: Type I and II Errors" by Toros Berberyan, Tracy Nguyen, and Alfie Swan is licensed under CC BY-SA 4.0

Attributions

"8.2: Type I and II Errors" by Rachel Webb is licensed under CC BY-SA 4.0

"9.E: Hypothesis Testing with One Sample (Exercises)" by OpenStax is licensed under CC BY 4.0

Exercises

1. Twenty-nine percent of high school seniors get drunk each month. For this example, state the Type I and Type II errors in complete sentences.

Scan the QR code or click on it to open the MyOpenMath version of the above question with step-by-step guidance.
MyOpenMath is a free online learning platform designed to support math instruction through automated homework, quizzes, and assessments. You must register for MyOpenMath and sign in to view the question.

2. Fewer than 5% of adults ride the bus to work in Los Angeles. For this example, state the Type I and Type II errors in complete sentences.

Scan the QR code or click on it to open the MyOpenMath version of the above question with step-by-step guidance.
MyOpenMath is a free online learning platform designed to support math instruction through automated homework, quizzes, and assessments. You must register for MyOpenMath and sign in to view the question.

3. The mean number of cars a person owns in his or her lifetime is more than ten. For this example, state the Type I and Type II errors in complete sentences.

Scan the QR code or click on it to open the MyOpenMath version of the above question with step-by-step guidance.
MyOpenMath is a free online learning platform designed to support math instruction through automated homework, quizzes, and assessments. You must register for MyOpenMath and sign in to view the question.

Answers

If you are an instructor and want the solutions to all the exercise questions for each section, please email Toros Berberyan.