9.4: Example Related Samples t-Tests

Example A: Increasing Satisfaction at Work

Workers at a local company have been complaining that working conditions have gotten very poor, hours are too long, and they don’t feel supported by the management. The company hires a consultant to come in and help fix the situation before it gets so bad that the employees start to quit. The consultant first assesses 40 of the employees’ level of job satisfaction as part of focus groups used to identify specific changes that might help. The company institutes some of these changes, and six months later the consultant returns to measure job satisfaction again. Knowing that some interventions miss the mark and can actually make things worse, the consultant tests for a difference in either direction (i.e., an increase or a decrease in average job satisfaction) at the a = .05 level of significance.

Step 1: State the Hypotheses

First, we state our null and alternative hypotheses:

\(
\begin{aligned}
\qquad H_0&: \text{There is no change in average job satisfaction} \\
\qquad H_0&: \mu_D = 0 \\[2.5ex]
\qquad H_A&: \text{There is an increase in average job satisfaction} \\
\qquad H_A&: \mu_D > 0
\end{aligned}
\)

In this case, we are hoping that the changes we made will improve employee satisfaction, and, because we based the changes on employee recommendations, we have good reason to believe that they will. Thus, we will use a one-directional alternative hypothesis.

Step 2: Find the Critical Values

Our critical values will once again be based on our level of significance, which we know is \(\alpha=.05\), the directionality of our test, which is one-tailed to the right, and our degrees of freedom. For our related-samples t-test, the degrees of freedom are still given as \(df=n-1\). For this problem, we have 40 people, so our degrees of freedom are 39. Going to our t-table, we find that the critical value is t* = 1.685 as shown in Figure \(\PageIndex{1}\).

Figure \(\PageIndex{1}\): Critical region for one-tailed t-test at \(\alpha=.05\). (“Critical Region t = 1.685” by Judy Schmitt is licensed under CC BY-NC-SA 4.0.)

Step 3: Calculate the Test Statistic and Effect Size

Now that the criteria are set, it is time to calculate the test statistic. The data obtained by the consultant found that the difference scores from Time 1 to Time 2 had a mean of M_D = 2.96 and a standard deviation of s_D= 2.85. Using this information, plus the size of the sample (n = 40), we first calculate the standard error:

\[
\Large
s_{M_D}=\frac{s_D}{\sqrt{n}}=\frac{2.85}{\sqrt{40}}=\frac{2.85}{6.32}=0.46
\nonumber
\]

Now, we can put that value, along with our sample mean and null hypothesis value, into the formula for t and calculate the test statistic:

\[
\Large
t=\frac{M_D-\mu_D}{s_{M_D}}=\frac{2.96-0}{0.46}=6.43
\nonumber
\]

Notice that, because the null hypothesis value of a related samples t-test is always 0, we can simply divide our obtained sample mean by the standard error.

Next, we will calculate Cohen’s d as an effect size using the same format as we did for the last t-test:

\[
\Large
d=\frac{M_D-\mu_D}{s_D}=\frac{2.96}{2.85}=1.04
\nonumber
\]

This is a large effect size. Notice again that we can omit the null hypothesis value here because it is always equal to 0.

Step 4: Make the Decision

We have obtained a test statistic of t = 6.43 that we can compare to our previously established critical value of t* = 1.685. The number 6.43 is larger than 1.685, so t > t* and we reject the null hypothesis:

Reject \(H_0\). Based on the sample data from 40 workers, we can say that the intervention statistically significantly improved job satisfaction (M_D = 2.96, SD_D = 2.85) among the workers, t(39) = 6.43, p < .05, d = 1.04.

Hopefully, the above example made it clear that running a related samples t test to look for differences before and after some treatment works exactly the same way as a regular one-sample t test does, which was just a small change in how z-tests were performed in Chapter 7. At this point, this process should feel familiar, and we will continue to make small adjustments to this familiar process as we encounter new types of data to test new types of research questions.

Example B: Bad Press

Let’s say that a bank wants to make sure that their new commercial will make them look good to the public, so they recruit 7 people to view the commercial as a focus group. The focus group members fill out a short questionnaire about how they view the company, then watch the commercial and fill out the same questionnaire a second time. The bank really wants to find significant results, so they test for a change at a = .10. However, they use a two-tailed test since they know that past commercials have not gone over well with the public, and they want to make sure the new one does not backfire. They decide to test their hypothesis using a confidence interval to see just how spread-out the opinions are. As we will see, confidence intervals work the same way as they did before, just like with the test statistic.

Step 1: State the Hypotheses

As always, we start with hypotheses:

\(
\begin{aligned}
\qquad H_0&: \text{There is no change in how people view the bank} \\
\qquad H_0&: \mu_D = 0 \\[2.5ex]
\qquad H_A&: \text{There is a change in how people view the bank} \\
\qquad H_A&: \mu_D \neq 0
\end{aligned}
\)

Step 2: Find the Critical Values

Just like with our regular hypothesis testing procedure, we will need critical values from the appropriate level of significance and degrees of freedom in order to form our confidence interval. Because we have 7 participants, our degrees of freedom are df = 6. From our t table, we find that the critical value corresponding to this df at this level of significance is t* = 1.943.

Step 3: Calculate the Confidence Interval

The data collected before (Time 1) and after (Time 2) the participants viewed the commercial are presented in Table \(\PageIndex{2}\). In order to build our confidence interval, we will first have to calculate the mean and standard deviation of the difference scores, which are also in Table \(\PageIndex{2}\). As a reminder, the difference scores are calculated as Time 2 − Time 1.

Table \(\PageIndex{2}\): Opinions about the bank.

Time 1	Time 2	XD
3	2	-1
3	6	3
5	3	-2
8	4	-4
3	9	6
1	2	1
4	5	1

The mean of the difference scores is:

\[
\Large
M_D=\frac{\sum{X_D}}{n}=\frac{4}{7}=0.57
\nonumber
\]

The standard deviation will be solved by first using the sum of squares table (Table \(\PageIndex{3}\)):

Table \(\PageIndex{3}\): Sum of squares.

XD	XD - MD	(XD - MD)^2
-1	-1.57	2.46
3	2.43	5.90
-2	-2.57	6.60
-4	-4.57	20.88
6	5.43	29.48
1	0.43	0.18
1	0.43	0.18
ΣXD = 4	Σ(XD - MD) = 0	Σ(XD - MD)^2 = 65.68

\[
\Large
s_D=\frac{SS}{df}=\sqrt{\frac{65.68}{6}}=\sqrt{10.95}=3.31
\nonumber
\]

Finally, we find the standard error:

\[
\Large
s_{M_D}=\frac{s_D}{\sqrt{n}}=\frac{3.31}{\sqrt{7}}=1.25
\nonumber
\]

We now have all the pieces needed to compute our confidence interval:

\[
\begin{aligned}
\text{90% CI} =& M_D \pm t^*(s_{M_D})\\
=& 0.57 \pm 1.943(1.25)\\[2.5ex]
\text{Lower bound} =& 0.57 - 1.943(1.25)\\
=& 0.57 - 2.43\\
=& -1.86\\[2.5ex]
\text{Upper bound} =& 0.57 + 1.943(1.25)\\
=& 0.57 + 2.43\\
=& 3.00\\[2.5ex]
\text{90% CI} =& (-1.86, 3.00)\\
\end{aligned}
\nonumber
\]

Step 4: Make the Decision

Remember that the confidence interval represents a range of values that seem plausible or reasonable based on our observed data. The interval spans −1.86 to 3.00, which includes 0, our null hypothesis value. Because the null hypothesis value is in the interval, it is considered a reasonable value, and because it is a reasonable value, we have no evidence against it. We fail to reject the null hypothesis.

Fail to reject \(H_0\). Based on our focus group of 7 people, we cannot say that the average change in opinion (M_D = 0.57, SD_D = 3.31) was any better or worse after viewing the commercial, 90% CI: (−1.86, 3.00).

As with before, we only report the confidence interval to indicate how we performed the test.

Test Your Knowledge

Question \(\PageIndex{1}\)

Question \(\PageIndex{2}\)

Question \(\PageIndex{3}\)