8.4: Comparing Two Means (Independent Samples)

Sometimes a single sample isn’t enough. Instead, we want to compare two groups — to test whether one mean is higher (or lower) than the other.

In these situations, we’ll use a two-sample t-test. If we do not assume equal variances between the groups, we’ll specifically use Welch’s t-test.

Questions That Involve Comparing Two Means

Here are five real-world questions where comparing group averages is a natural fit:

• Do students who attend tutoring score higher on exams than those who don’t?
• Is it colder on average in Leadville, Colorado than in Anchorage, Alaska?
• Do iPhone users spend more on monthly mobile plans than Android users?
• Does a new fertilizer increase the average crop yield compared to the standard?
• Do influencers post more times per day on TikTok than Twitter users do on Twitter?

All of these questions can be framed as tests comparing the means of two independent samples.

Welch’s t-test Statistic

Used when the sample sizes or variances may be unequal:

\[ T = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}} \]

We'll use this statistic to test differences between two groups. This test statistic follows the Student t-distribution just like the t-tests we saw in the last section.

Let’s try it out in a real example.

---

Example: “Is it colder in Leadville than Anchorage?”

You’re arguing with a friend: They claim it's obviously colder in Alaska. But you're not so sure — Leadville, Colorado is at 10,000 feet after all.

You decide to test whether the average winter temperature in Leadville is colder than in Anchorage.

Step 1: State the Hypotheses

You'd like to test:

• \( H_0: \mu_{Leadville} = \mu_{Anchorage} \)
• \( H_A: \mu_{Leadville} < \mu_{Anchorage} \)

This is a left-tailed test. You're looking for evidence that winter temperatures are lower in Leadville.

Step 2: Calculate the Appropriate Test Statistic

You collect 10 average January temperatures from each city (in °F):

• Leadville: \( \bar{x}_1 = 12.2, s_1 = 4.8, n_1 = 10 \)
• Anchorage: \( \bar{x}_2 = 17.6, s_2 = 6.2, n_2 = 10 \)

This is a two-sample independent t-test. Sample standard deviations are not equal → We use Welch's t-test.

Plug into the equation:

\[ T = \frac{12.2 - 17.6}{\sqrt{\frac{(4.8)^2}{10} + \frac{(6.2)^2}{10}}} = \frac{-5.4}{\sqrt{2.304 + 3.844}} = \frac{-5.4}{\sqrt{6.148}} \approx \frac{-5.4}{2.478} \approx -2.18 \]

Step 3: Choose Significance Level

You select the standard \( \alpha = 0.05 \) as your cutoff for a Type I error (false positive).

Step 4: Compute P-value Using Technology

The degrees of freedom here are computed using the Welch-Satterthwaite formula(opens in new window). Generally this calculation is included in whatever technology you're using. In some cases we may use a simplified, conservative option where the degrees of freedom is the smaller of \( n_1 -1\) and \( n_2-1\) but this does typically result in slightly larger P-values.

Use technology to find P( t < -2.18). We get \( p \approx 0.023 \)

Step 5: Make Your Conclusion

Since 0.023 < 0.05, you reject \( H_0 \).

Conclusion: There is statistically significant evidence that winters in Leadville are colder than in Anchorage.

Interpretation: While Leadville may not be as famously cold as Anchorage, your data suggest that altitude beats latitude!

---

Looking Ahead: Comparing Proportions

We’ve now compared two means — but many research questions involve questions about proportions, like:

• Will more than 50% of voters support a ballot measure?
• Is one treatment more likely to succeed than another?
• Are more students passing with the new class schedule?

We’ll use a new test statistic to for proportions, but the already familiar sampling distribution following the Normal Distribution. Again, the process stays the same — only the test statistic changes.