1.4: Sources of Bias and Randomness

Bias and the Importance of Randomness

Now that we understand how we often use a sample to learn about a larger population, let’s talk about a key idea that can interfere with our conclusions: bias.

What Is Bias?

Bias occurs when the method used to collect data causes certain individuals or outcomes to be favored or excluded, making the sample systematically different from the population.

Bias can lead us to incorrect conclusions about a population because the information in our sample doesn’t fairly or fully represent the entire group. While some variability is natural in any sample, biased samples introduce systematic errors that can’t be explained by random chance.

Examples of Bias in Sampling

• Convenience Bias: A teacher wants to know how much students are studying on campus. She surveys the students in the library at 9am. But those students might be more prepared or studious than the general student population.
• Nonresponse Bias: A health department emails a survey on exercise habits, but only 20% of people respond. If more active people are more likely to answer, the final summary may overestimate how much exercise the population gets.
• Undercoverage Bias: A city survey collects responses entirely via smartphones. This may exclude older adults or those with limited internet access—leaving out important parts of the population.
• Voluntary Response Bias: An online poll asks if customers are “satisfied with service.” People who feel extremely positive or negative are more likely to respond, skewing the results.

In each case, the problem is that the method causes misrepresentation. If we base policies or decisions on biased data, we may end up hurting the very communities we aim to serve.

How Randomness Helps

Random sampling is one of the most important defenses against bias.

By using a random process to select individuals from the population, giving each one a known and fair chance of being included, we reduce the influence of personal judgment, patterns, convenience, or external factors.

Randomness also lets us make statistical claims later, because it gives us a defensible reason to treat our sample as representative of the full population within the limits of chance.

Example: Comparing Two Approaches

A researcher wants to know the average commuting time for people in a city. They consider two approaches:

• Approach A: They survey 300 people in the downtown train station at 8:30am on a weekday.
• Approach B: They randomly select 300 residents from city voter rolls and call them at home.

Approach A is likely to lead to bias those passing through the station during rush hour might have different commuting patterns than city residents as a whole. Some might not even live in the city. Approach B has a better chance of reaching people across neighborhoods, schedules, and lifestyles — especially if non-response is addressed properly.

Beyond Sampling: Measurement and Design Bias

Bias isn't limited to sampling. It can also creep in when:

• Survey questions are worded in leading or confusing ways.
• Instruments (like scales or tests) aren’t well-calibrated or fair.
• Researchers influence responses consciously or unconsciously.

We’ll talk more about these issues when we learn how to design good studies and interpret study results fairly.

Can We Avoid Bias Completely?

Bias is often invisible unless we actively look for it. Even with good intentions, human judgments and systems can introduce unfair patterns. While we may not eliminate all sources of bias, awareness and thoughtful design give us the best possible chance to reduce it.

Reflection: Can You Spot the Bias?

• A school district sends out a digital survey asking about lunch program satisfaction. Who might be unknowingly excluded?
• A company lets employees “opt in” to a survey on job satisfaction. What kind of responses might they get?
• A sports article collects Twitter replies about a controversial call in a playoff game. Is that representative of all fans?

Quick Quiz: Identify the Type of Bias

For each example below, choose the type of sampling bias that is most likely present.

1. A news website runs an online poll asking readers if they're satisfied with government policy. Only extremely positive and negative responses pour in.
Select one:--Choose--Convenience BiasNonresponse BiasVoluntary Response BiasUndercoverage Bias

2. A public health survey is distributed only through a smartphone app. Many older adults never see it and aren’t included in the data.
--Choose--Convenience BiasNonresponse BiasVoluntary Response BiasUndercoverage Bias

3. A school researcher surveys students who are already standing in front of the cafeteria to get quick answers about lunch preferences.
--Choose--Convenience BiasNonresponse BiasVoluntary Response BiasUndercoverage Bias

4. A market research firm emails a survey to 1,000 customers, but only 210 respond — most of whom are satisfied repeat buyers.
--Choose--Convenience BiasNonresponse BiasVoluntary Response BiasUndercoverage Bias

What's Next?

Now that we understand the importance of fair sampling and the dangers of bias, we’re ready to think about how to ask good statistical questions and design reliable studies. In the next section, we’ll learn how to plan an investigation that collects data in a way that’s focused, useful, and ethical from the start.