11.6: Practice on Mindset Data
- Page ID
- 22116
Okay, here’s your time to practice the whole process of a Between Groups ANOVA, including each Sum of Squares formula and the pairwise comparisons!
Scenario
Dr. MO helped student researchers try to improve the mindset of their fellow community college students (If you need a refresher, check out the Growth Mindset section, or search online for “growth mindset”). Three different professors (all women) tried three different interventions (shown below) to see if they could improve mindset by the end of the semester. The three intervention levels were:
- No Intervention: This was the comparison control group. It was a behavioral statistics course (like you are taking right now!). The instructor had students complete the Mindset Quiz as part of class related to surveys and data collection, but the professor never explained what Growth Mindset was or how it helps students succeed. No class activities or assignments were related to Growth Mindset.
- Minimal Intervention: This was an Early Childhood Education class that explained what Growth Mindset was, and how it could help both the students themselves, and the children that they would be taking care of or teaching. The course had a few assignments and class activities related to Growth Mindset.
- Super Intervention: This was a History course. This professor explained Growth Mindset and how it could help them succeed, as well as weekly readings, discussions, and activities related to Growth Mindset.
Students who enrolled in these courses were participants in one of the three different interventions, and filled out the Mindset Quiz at the end of the semester. Mindset Quiz scores can range from 20 to 60, with higher scores showing that the student has more Growth Mindset (and less Fixed Mindset). At the end of the semester, 10 students in each course completed the Mindset Quiz and had attended most classes throughout the semester to experience any activities related to mindset (N = 30).
After reading the scenario, can you describe the sample, population, IV, and DV?
Answer the following questions to understand the variables and groups that we are working with.
- Who is the sample?
- Who do might be the population?
- What is the IV (groups being compared)?
- What is the DV (quantitative variable being measured)?
- Answer
-
- 30 community college students.
- Community college students? Community college students with women professors?
- Intervention levels: None, Minimal, Super
- Mindset Quiz
Step 1: State the Hypotheses
Using the following the research question and the descriptive statistics in Table \(\PageIndex{1}\), what could be the research hypothesis?
Research Question: The student researchers were interested to know if Growth Mindset could be taught, and how much time the instructor needed to devote to these intervention activities to improve growth mindset.
| N: | Mean: | SD: | Median: | |
|---|---|---|---|---|
| No Intervention | 10 | 42.70 | 8.18 | 43.00 |
| Minimal Intervention | 10 | 44.40 | 5.87 | 43.00 |
| Super Intervention | 10 | 51.10 | 4.93 | 50.50 |
| Total | 30 | 46.07 | 7.25 | 48.00 |
You are encourages to use the raw data found in Table \(\PageIndex{2}\) in Example \(\PageIndex{2}\) to practice calculating the means, standard deviations, and medians yourself!
What's your research hypothesis? Remember, the research hypothesis should predict how all levels of the IV related to all other levels of the IV.
What is a research hypothesis in words and symbols?
- Answer
-
- Research Hypothesis: Students in the No Intervention group average score will be lower on the Mindset Quiz than both students in the Minimal Intervention Group and the Super Intervention group. Similarly, students in the Minimal Intervention group average score will be lower than students in the Super Intervention group.
- Symbols:
- \(\overline{X}_N < \overline{X}_M \)
- \(\overline{X}_N < \overline{X}_S \)
- \(\overline{X}_M < \overline{X}_S \)
Your research hypothesis might be slightly different than this one. Just make sure that at least one mean is predicted to be different than one other mean.
What is the null hypothesis with this scenario?
State the null hypothesis in words and symbols.
- Answer
-
- Null Hypothesis: Students in the No Intervention group will have a similar average score on the Mindset Quiz as both students in the Minimal Intervention Group and the Super Intervention group. Similarly, students in the Minimal Intervention group will have a similar average score as students in the Super Intervention group.
- Symbols:
- \(\overline{X}_N = \overline{X}_M \)
- \(\overline{X}_N = \overline{X}_S \)
- \(\overline{X}_M = \overline{X}_S \)
Step 2: Find the Critical Values
This step might be easier after you’ve completed the ANOVA Summary Table because you will have the Degrees of Freedom for both groups, but let’s try it now.
Using the Critical Values of F Table by going to the page or finding the link in the Common Critical Values page at the end of the textbook, what is the critical value at \(α\) = 0.05 with three groups and a total of 30 people?
- Answer
-
Critical F(2,27)=3.35
The first Degree of Freedom (2), for the numerator (dfB) is found through: k–1, with k being the number of groups.
The second Degree of Freedom (27), for the denominator (dfW) is found through \(N – k (30 – 3 = 27)\). A common mistake is to use the Degrees of Freedom of the numerator instead of the number of groups..
Step 3: Compute the Test Statistic
If you will never have to calculate the Sums of Squares by hand, skip this part and just fill in the ANOVA Summary Table (Table \(\PageIndex{4}\)) at the end of this section. If you are practicing the Sums of Squares, each Sum of Square will have it’s own Example. Heads up, to do all of these can take about an hour!
Let’s go!
Calculate the Between Groups Sums of Squares.
Solution
\[S S_{B}=\sum_{EachGroup} \left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} * (n_{group}) \right] \nonumber \]
The \(\sum_{EachGroup}\) means that you do everything following that for each intervention level, then add them all together. Let’s start with what’s inside the brackets for the No Intervention group.
\(\sum_{EachGroup}\)
\[S S_{B}=\sum_{EachGroup} \left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} * (n_{group}) \right] \nonumber \]
\[\text{No Intervention} =\left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} * (n_{group}) \right] \nonumber \]
\(\overline{X}_{group}\) is asking for the mean of the group that we're looking at, so the No Intervention group right now. \(\overline{X}_{T}\) is asking for the total mean, the mean for all 30 scores. Both of these means were provided in Table \(\PageIndex{1}\). The number of scores in the group that we're looking at right now is what \(n_{group}\) is asking.
So let's plug those all in!
\[\text{No Intervention} =\left[ \left(42.70 - 46.07\right)^{2} * (10) \right] \nonumber \]
\[\text{No Intervention} =\left[ \left(-3.37 \right)^{2} * (10) \right] \nonumber \]
The mean of the group minus the mean of the sample should be negative, but that negative sign goes away when we square it:
\[\text{No Intervention} =\left[ \left(11.36 \right) * (10) \right] \nonumber \]
\[\text{No Intervention} =\left[ 113.60 \right] \nonumber \]
And let’s do that process two more times, once for the Minimal Intervention Group and once for the Super Intervention group, and then we’ll add those three numbers together to get the SSB.
\[\text{Minimal Intervention} =\left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} * (n_{group}) \right] \nonumber \]
\[\text{Minimal Intervention} =\left[ \left(44.40 - 46.07\right)^{2} * (10) \right] \nonumber \]
\[\text{Minimal Intervention} =\left[ \left(-1.67 \right)^{2} * (10) \right] \nonumber \]
\[\text{Minimal Intervention} =\left[ \left(2.79 \right) * (10) \right] \nonumber \]
\[\text{Minimal Intervention} =\left[ 27.90 \right] \nonumber \]
Why don’t you try to do it on your own for the Super Intervention group?
\[\text{Super Intervention} =\left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} * (n_{group}) \right] \nonumber \]
\[\text{Super Intervention} =\left[ 253.10 \right] \nonumber \]
Next step, add them all together! It’s easy to forget this step, but the Sum of Squares ends up to be one number, so when you get lost or forget the next step, look back at the full formula:
\[S S_{B}=\sum_{EachGroup} \left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} * (n_{group}) \right] \nonumber \]
\[S S_{B}=\sum_{EachGroup} = \left[ 113.60 + 27.90 + 253.10 \right] \nonumber \]
\[S S_{B}=394.60 \nonumber \]
You did it! Only two more Sums of Squares to go!
Calculate the Within (Error) Sum of Squares.
Solution
\[S S_{W}=\sum \left[ \left(X-\overline{X}_{group}\right)^{2}\right] \nonumber \]
As before, complete the calculations in the brackets first for each group, then add them all together. To complete the calculations in the brackets, it's easiest to use a table with all of the values since you subtract the mean of the group that you're working with from each score. Do you remember doing this with standard deviations?
Table \(\PageIndex{2}\) shows the group mean subtracted from each score in the column to the right of the raw scores, then that is squared in the column to the next right. The squared values are then summed for each group.
| No Intervention | minus group mean | squared | Minimal Intervention | minus group mean | squared | Super Intervention | minus group mean | squared |
|---|---|---|---|---|---|---|---|---|
| 30 | -12.70 | 161.29 | 37 | -7.40 | 54.76 | 43 | -8.10 | 65.61 |
| 35 | -7.70 | 59.29 | 38 | -6.40 | 40.96 | 45 | -6.10 | 37.21 |
| 36 | -6.70 | 44.89 | 39 | -5.40 | 29.16 | 48 | -3.10 | 9.61 |
| 36 | -6.70 | 44.89 | 42 | -2.40 | 5.76 | 50 | -1.10 | 1.21 |
| 43 | 0.30 | 0.09 | 42 | -2.40 | 5.76 | 50 | -1.10 | 1.21 |
| 43 | 0.30 | 0.09 | 44 | -0.40 | 0.16 | 51 | -0.10 | 0.01 |
| 49 | 6.30 | 39.69 | 48 | 3.60 | 12.96 | 54 | 2.90 | 8.41 |
| 49 | 6.30 | 39.69 | 48 | 3.60 | 12.96 | 56 | 4.90 | 24.01 |
| 53 | 10.30 | 106.09 | 53 | 8.60 | 73.96 | 56 | 4.90 | 24.01 |
| 53 | 10.30 | 106.09 | 53 | 8.60 | 73.96 | 58 | 6.90 | 47.61 |
| Sum: | 602.10 | Sum: | 310.40 | Sum: | 218.90 | |||
| \(\overline{X}_N\) = 42.7 | N/A | N/A | \(\overline{X}_M\) = 44.4 | N/A | N/A | \(\overline{X}_S\) = 51.1 | N/A | N/A |
Now that we have the sum of each squared score of the subtraction, we can finish the formula:
\[S S_{W}=\sum_{EachGroup} \left[ \sum \left(\left(X-\overline{X}_{group}\right)^{2}\right) \right] \nonumber \]
\[S S_{W}=\sum_{EachGroup} = \left[ 602.10 + 310.40 + 218.90\right] \nonumber \]
\[S S_{W}=1131.40 \nonumber \]
And, on to the final Sum of Squares!
Calculate the Total Sum of Squares.
Solution
\[S S_{T}=\sum \left[ \left(X - \overline{X}_{T}\right)^{2} \right] \nonumber \]
This formula is also saying to subtract a mean from each score, but this time we should be subtracting the Total mean (\(\overline{X}_T\) = 46.07, found in Table \(\PageIndex{1}\)). This is again easiest to compute in a table.
Table \(\PageIndex{3}\) shows the Total mean subtracted from each score in the column to the right of the raw scores, then that is squared in the column to the next right. The squared values are then summed for all of the scores.
| IV Levels | Mindset Quiz Scores | minus Total mean | squared |
|---|---|---|---|
| No Intervention | 30 | -16.07 | 258.24 |
| No Intervention | 35 | -11.07 | 122.54 |
| No Intervention | 36 | -10.07 | 101.40 |
| No Intervention | 36 | -10.07 | 101.40 |
| No Intervention | 43 | -3.07 | 9.42 |
| No Intervention | 43 | -3.07 | 9.42 |
| No Intervention | 49 | 2.93 | 8.58 |
| No Intervention | 49 | 2.93 | 8.58 |
| No Intervention | 53 | 6.93 | 48.02 |
| No Intervention | 53 | 6.93 | 48.02 |
| Minimal Intervention | 37 | -9.07 | 82.26 |
| Minimal Intervention | 38 | -8.07 | 65.12 |
| Minimal Intervention | 39 | -7.07 | 49.98 |
| Minimal Intervention | 42 | -4.07 | 16.56 |
| Minimal Intervention | 42 | -4.07 | 16.56 |
| Minimal Intervention | 44 | -2.07 | 4.28 |
| Minimal Intervention | 48 | 1.93 | 3.72 |
| Minimal Intervention | 48 | 1.93 | 3.72 |
| Minimal Intervention | 53 | 6.93 | 48.02 |
| Minimal Intervention | 53 | 6.93 | 48.02 |
| Super Intervention | 43 | -3.07 | 9.42 |
| Super Intervention | 45 | -1.07 | 1.14 |
| Super Intervention | 48 | 1.93 | 3.72 |
| Super Intervention | 50 | 3.93 | 15.44 |
| Super Intervention | 50 | 3.93 | 15.44 |
| Super Intervention | 51 | 4.93 | 24.30 |
| Super Intervention | 54 | 7.93 | 62.88 |
| Super Intervention | 56 | 9.93 | 98.60 |
| Super Intervention | 56 | 9.93 | 98.60 |
| Super Intervention | 58 | 11.93 | 142.32 |
| \(\sigma\) | N/A | \(\sigma = 1525.87\) |
\[S S_{T}=\sum \left[ \left(X - \overline{X}_{T}\right)^{2} \right] = 1525.87 \nonumber \]
Yay! You finished the Sum of Squares! But before we move on, let’s do the computation check to see if we did the Sums of Squares correctly: \(S S_{T}=S S_{B}+S S_{W} \)
Let’s check:
\[S S_{T}=S S_{B}+S S_{W} = 394.40 + 1131.40 = 1526.00 \nonumber \]
This is close to our SST of 1525.87, so Dr. MO checked to make sure all of the calculations were done correctly with both a calculator and with Excel; it appears that this slight discrepancy is due to rounding differences.
But we’re not done yet! Now, plug those three Sums of Squares into the ANOVA Summary Table (Table \(\PageIndex{4}\)) so that you can fill out the rest of the table to calculate the final ANOVA F-value.
| Source | \(SS\) | \(df\) | \(MS\) | \(F\) |
|---|---|---|---|---|
| Between | 394.60 | |||
| Within | 1131.40 | |||
| Total | 1525.87 |
Fill out the rest of the ANOVA Summary Table.
Solution
| Source | \(SS\) | \(df\) | \(MS\) | \(F\) |
|---|---|---|---|---|
| Between | 394.60 | \(k-1 = 3 – 1 = 2\) | \(\frac{S S_{B}}{d f_{B}} = \frac{394.60}{2} = 197.30 \) | \(\frac{MS_{B}}{MS_{W}} = \frac{197.30}{41.90} = 4.71\) |
| Within | 1131.40 | \(N-k = 30 – 3 = 27\) | \(\frac{S S_{W}}{d f_{W}} = \frac{1131.40}{27} = 41.90 \) | leave blank |
| Total | 1525.87 | \(N-1 = 30 – 1 = 29\) | leave blank | leave blank |
If you do a computation check (which you should) of the degrees of freedom, you find that:
\[df_{T}=df_{B} + df_{W} = 2 + 27 = 29 \nonumber \]
Since we did that part correctly, we can move on to the next step of the process.
Step 4: Make the Decision
Based on the completed ANOVA Summary Table (Table \(\PageIndex{5}\)), our calculated F-score is 4.71. If you remember all the way back to Step 2, we found our critical F-score to be 3.35.
(Critical \(<\) Calculated) \(=\) Reject null \(=\) At least one mean is different from at least one other mean. \(= p<.05\)
(Critical \(>\) Calculated) \(=\) Retain null \(=\) All of the means are similar. \(= p>.05\)
Since our critical value is smaller than our calculated value, we reject the null hypothesis that all of the means are similar, which means that at least one mean is different from at least one other mean. That’s not specific enough to evaluate whether our research hypothesis is correct or not, so on to pairwise comparisons!
Pairwise Comparisons
Let’s start with finding the differences between each pair of means.
What are the mean differences for each pair of means?
Solution
\[ \overline{X}_N - \overline{X}_M = 42.70 – 44.40 = -1.7 \nonumber\]
\[ \overline{X}_N - \overline{X}_S = 42.7 – 51.10 = -8.4 \nonumber\]
\[ \overline{X}_M - \overline{X}_S = 44.40 – 51.10 = -6.7 \nonumber\]
Great! Now, let’s compute Tukey’s HSD to see if any of these mean differences are big enough to say that they are statistically significantly different.
Using the q-value for our degrees of freedom from the Alpha = 0.05 table from Real-Statistics.com’s q table, compute Tukey’s HSD. If you will never have to calculate a post-hoc analysis by hand, skip this part and use the calculated Tukey’s HSD to make your decisions and write-up the conclusion.
Solution
From the critical q table, we find that for our study with 3 groups and the degrees of freedom for the denominator (dfW = 27), the correct q-score to plug into the formula is 3.506. The other information is in the completed ANOVA Summary Table (Table \(\PageIndex{5}\)).
\[ HSD = q * \sqrt{\dfrac{MSw}{n_{group}}} \nonumber \]
\[ HSD = 3.506 * \sqrt{\dfrac{41.90}{10}} \nonumber \]
\[ HSD = 3.506 * \sqrt{4.19} \nonumber \]
\[ HSD = 3.506 * 2.05 \nonumber \]
\[ HSD = 7.18 \nonumber \]
Now what?
Well, you have a critical value (Tukey’s HSD) and some mean differences, let’s compare them!
(Critical \(<\) Calculated) \(=\) Reject null \(=\) At least one mean is different from at least one other mean. \(= p<.05\)
(Critical \(>\) Calculated) \(=\) Retain null \(=\) All of the means are similar. \(= p>.05\)
Using the Tukey’s HSD of 7.18, which means are statistically significantly different from each other?
Solution
Our pairwise comparison critical value is 7.18. Of the absolute value of the mean differences, that means that the Super Intervention was statistically significantly higher than the No Intervention group’s average score on the Mindset Quiz, but none of the other pairs of means were significantly different.
\[ \overline{X}_N - \overline{X}_M = 42.70 – 44.40 = -1.7 \nonumber\]
\[ \overline{X}_N - \overline{X}_S = 42.7 – 51.10 = -8.4 \nonumber\]
\[ \overline{X}_M - \overline{X}_S = 44.40 – 51.10 = -6.7 \nonumber\]
How does this relate to the research hypothesis?
- \(\overline{X}_N < \overline{X}_M \): This was NOT supported. Although the mean of the Minimal Intervention group was larger than the group with No Intervention, it was not significantly larger (because the critical value of Tukey’s HSD of 7.18 was bigger than the absolute value of the mean difference between these two groups of -1.7)
- \(\overline{X}_N < \overline{X}_S \): This was supported.
- \(\overline{X}_M < \overline{X}_S \): This was NOT supported. The means were not different enough to reject the null hypothesis that the means are from different populations, so we conclude that the Minimal Intervention group had similar Mindset Quiz scores as the Super Intervention group.
This means that our research hypothesis was partially supported.
Write-Up
Okay, here’s the big finish!
Write a conclusion to describe the results of the analysis. Don’t forget to include the four components necessary in any report of results.
- Answer
-
The researchers hypothesized that students in the No Intervention group will score lower on average on the Mindset Quiz than both students in the Minimal Intervention Group and the Super Intervention group. Similarly, students in the Minimal Intervention group will score lower on average than students in the Super Intervention group.
This research hypothesis was partially supported (F(2,27) = 4.71, p < 0.05). The Super Intervention group’s average Mindset Quiz score (M = 51.10) was higher than the No Intervention group (M = 42.70), but the the Minimal Intervention group’s (M = 44.40) average Mindset Quiz scores were not statistically significantly different than either the Super Intervention group or the No Intervention group.
Did Dr. MO’s write-up include all of the components?
You did it! Take a break and reward yourself!
Next up, we’ll look at what to do if you don’t think that your distribution is normally distributed when you have three or more groups…