12.3.1: Practice with Mindset
 Page ID
 22126
Okay, here’s your time to practice the whole process of a Repeated Measures ANOVA, including each Sum of Squares formula!
Scenario
In another study by student researchers at a community college, mindset was measured with a Mindset Quiz at the beginning of the semester in a remedial English class, at the end the semester in the remedial English class, then at the beginning of the next semester in the next higher English class, and at the end of that semester in the next higher English class. Thus, the same students were measured four times to see if their mindset improved through journaling activities related to growth mindset. Mindset Quiz scores can range from 20 to 60, with higher scores showing that the student has more Growth Mindset (and less Fixed Mindset). At the end of the second semester, 12 students had completed the Mindset Quiz pretest and posttest during each class (at the beginning and end of each semester), and had attended most classes throughout the semester to experience the journaling activities related to mindset (N = 48).
After reading the scenario, can you describe scenario?
Answer the following questions to understand the variables and groups that we are working with.
 Who is the sample?
 Who do might be the population?
 What is the IV (groups being compared)?
 What is the DV (quantitative variable being measured)?
Solution
 The sample is 48 community college students in English classes, who started in remedial English.
 The population could be students taking remedial English at community colleges?
 IV: Time Beginning of Remedial English, End of Remedial English, Beginning of Next English, End of Next English
 DV: Mindset Quiz
Step 1: State the Hypotheses
Using the descriptive statistics in Table \(\PageIndex{1}\), what could be the research hypothesis?
N:  Mean:  SD:  

Beginning of Remedial English (R1)  12  39.17  6.46 
End of Remedial English (R2)  12  43.33  8.48 
Beginning of Next English Class (N1)  12  42.25  5.24 
End of Next English Class (N2)  12  45.42  6.35 
Total  48  42.54  6.90 
The notation in the parentheses is what we’ll use as the subscript labels in the formulas.
You are encourages to use the raw data found in Table \(\PageIndex{2}\) in Step 3 to practice calculating the means, standard deviations, and medians yourself!
Remember, the research hypothesis should predict how all levels of the IV related to all other levels of the IV. Because this scenario has four different IV levels, there’s actually six combinations of pairs of means!
What is a research hypothesis for this scenario in words and symbols?
Solution
Research Hypothesis: Students average Mindset Quiz scores will be higher in the beginning of the semesters than at the end, and higher in the next class than in the first (remedial) class.
Your research hypothesis might be slightly different than this one. Just make sure that at least one mean is predicted to be different than one other mean.
Symbols:
 \(\overline{X}_{R1} < \overline{X}_{R2} \)
 \(\overline{X}_{R1} < \overline{X}_{N1} \)
 \(\overline{X}_{R1} < \overline{X}_{N2} \)
 \(\overline{X}_{R2} < \overline{X}_{N1} \)
 \(\overline{X}_{R2} < \overline{X}_{N2} \)
 \(\overline{X}_{N1} < \overline{X}_{N2} \)
Despite having so many combinations, the null hypothesis for this scenario is still relatively simple, just like all null hypotheses.
State the null hypothesis in words and symbols.
 Answer

Null Hypothesis: Students will have a similar average scores on the Mindset Quiz in all four conditions: Beginning of Remedial English, End of Remedial English, Beginning of Next English Class, and End of Next English Class
Symbols: \(\overline{X}_{R1} = \overline{X}_{R2} = \overline{X}_{N1} = \overline{X}_{N2} \)
Step 2: Find the Critical Values
This step might be easier after you’ve completed the ANOVA Summary Table because you will have the Degrees of Freedom for both groups, but we’ll keep following the steps as we’ve learned them.
Using the Critical Values of F Table or finding the link in the Common Critical Values page at the end of the textbook, what is the critical value at \(α\) = 0.05?
 Answer

Critical F(3,33) = 2.92
The first Degree of Freedom (3), for the numerator (df_{B}) is found through: k – 1, with k being the number of groups.
The second Degree of Freedom (33), for the denominator (df_{W}) is found through \((k1)\times(Ps1)\), with Ps being the number of participants.
\[ (k1) \times (Ps1) = (41)\times(121) = 3 * 11 = 33 \nonumber \]
Take a breath, because now we will start with the messy calculations!
Step 3: Compute the Test Statistic
If you will never have to calculate the Sums of Squares by hand, skip this part and just fill in the ANOVA Summary Table (Table \(\PageIndex{5}\)) at the end of this section. If you are practicing the Sums of Squares, each Sum of Square will have its own Example. Heads up, to do all of these can take about an hour!
Participant  Pretest in Remedial English  Posttest in Remedial English  Pretest in Next English Level  Posttest in Next English Class  Total 

A  30  48  32  42  152 
B  31  39  35  32  137 
C  33  35  39  38  145 
D  35  32  40  50  157 
E  36  37  42  48  163 
F  38  33  42  51  164 
G  39  45  43  38  165 
H  42  52  45  48  187 
I  43  39  45  48  175 
J  47  51  46  51  195 
K  47  54  47  51  199 
L  49  55  51  48  203 
Sum:  470  520  507  545  2042 
N:  12  12  12  12  48 
Let’s go!
Between Groups SS
This is the same formula that we learned in the Between Groups ANOVA.
Calculate the Between Groups Sums of Squares.
Solution
\[S S_{B}=\sum_{EachGroup} \left[ \left(\overline{X}_{group}\overline{X}_{T}\right)^{2} \times (n_{group}) \right] \nonumber \]
The \(\sum_{EachGroup}\) means that you do everything following that for each intervention level, then add them all together. Let’s start with what’s inside the brackets for the Mindset Quiz at the beginning of the remedial English class (R1).
\[\text{R1} =\left[ \left(\overline{X}_{group}\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]
\(\overline{X}_{group}\) is asking for the mean of the group that we're looking at, and \(\overline{X}_{T}\) is asking for the total mean, the mean for all 48 scores. Both of these means were provided in Table \(\PageIndex{1}\). The number of scores in the group that we're looking at right now is what \(n_{group}\) is asking.
So let's plug those all in!
\[\text{R1} =\left[ \left(39.17 – 42.54\right)^{2} \times(12) \right] \nonumber \]
\[\text{R1} =\left[ \left(3.37 \right)^{2} \times(12) \right] \nonumber \]
The mean of the group minus the mean of the sample should be negative in this scenario, but that negative sign goes away when we square it:
\[\text{R1} =\left[ \left(11.36 \right) \times(12) \right] \nonumber \]
\[\text{R1} =\left[ 136.28 \right] \nonumber \]
And let’s do that process three more times, once for the End of the Remedial English class’s semester, once for the Beginning of the Next English Class, and once for the End of the Next English class. Then we’ll add those four numbers together to get the SS_{B}.
\[\text{R2} =\left[ \left(\overline{X}_{group}\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]
\[\text{R2} =\left[ \left(43.33 – 42.54\right)^{2} \times(12) \right] \nonumber \]
\[\text{R2} =\left[ \left(0.79 \right)^{2} \times(12) \right] \nonumber \]
\[\text{R2} =\left[ \left(0.62 \right) \times (12) \right] \nonumber \]
\[\text{R2} =\left[ 7.44 \right] \nonumber \]
If you are doing all of your calculations in Excel or somehow saving all of the decimal points for each answer, this calculation will be a little higher (7.49). Don’t worry, the rounding differences will mostly wash out by the end of the formula. We will be using the answers provided if you type the two numbers after the decimal point into a calculator.
Why don’t you try to do it on your own for the Next English class conditions?
\[\left[ \left(\overline{X}_{group}\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]
\[\text{N1} =\left[0.96 \right] \nonumber \]
\[\text{N2} =\left[99.48 \right] \nonumber \]
Next step, add them all together! It’s easy to forget this step, but the Sum of Squares ends up to be one number, so when you get lost or forget the next step, look back at the full formula:
\[\sum_{EachGroup} \left[ \left(\overline{X}_{group}\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]
\[\sum_{EachGroup} = \left[ 136.32 + 7.44 + 0.96 + 99.48 \right] \nonumber \]
\[\sum_{EachGroup}= 244.20 \nonumber \]
Again, if you had saved more than two decimals, you would end up with \(S S_{B}=244.31 \). And if you used that number in the ANOVA Summary Table, the calculated Fvalue would be nearly identical to the same calculations but with only the two numbers after the decimal.
You did it! Only three more Sums of Squares to go!
Participant SS
This one is new!
Calculate the Participant Sum of Squares.
Solution
\[SS_{Ps} = \left[\sum{\left(\dfrac{(\sum{X_{Ps}})^2}{k}\right)}\right] \dfrac{\left((\sum{X})^2\right)}{N} \nonumber \]
This one is easiest to do in a table, as shown in Table \(\PageIndex{3}\).
Participant  Pretest in Remedial English  Posttest in Remedial English  Pretest in Next English Level  Posttest in Next English Class  Sum of Ps  Squared  Divide by k 

A  30  48  32  42  152  23,104.00  5776.00 
B  31  39  35  32  137  18,769.00  4692.25 
C  33  35  39  38  145  21,025.00  5256.25 
D  35  32  40  50  157  24,649.00  6162.25 
E  36  37  42  48  163  26,569.00  6642.25 
F  38  33  42  51  164  26,896.00  6724.00 
G  39  45  43  38  165  27,225.00  6806.25 
H  42  52  45  48  187  34,969.00  8742.25 
I  43  39  45  48  175  30,625.00  7656.25 
J  47  51  46  51  195  38,025.00  9506.25 
K  47  54  47  51  199  39,601.00  9900.25 
L  49  55  51  48  203  41,209.00  10302.25 
Sum:  \(\sigma = 470\)  \(\sigma = 520\)  \(\sigma = 507\)  \(\sigma = 545\)  N/A  N/A  \(\sigma = 88,166.50\) 
N:  n=12  n=12  n=12  n=12  N/A  N/A  48 scores 
What’s happening in Table \(\PageIndex{3}\)? The first column is labeling each participant. When you see a column like this, then you know that the scores are related, meaning that you should run a dependent ttest or a Repeated Measures ANOVA. The two rows on the bottom (“Sum:” and “N:”) show the sum of each column, and the number of scores in that column. This makes is really easy to find the mean! The next four columns are the scores for the four conditions of the IV.
Then, we get some columns to help us calculate the Participant Sum of Squares. The Sum of Ps column just adds the four scores for each individual participant. For example,
\[\text{Sum of Participant A} = 30 + 48 + 32 = 42 = 152 \nonumber \]
The next column, Squared, is the individual participant’s summed scores squared \(\text{Participant A} = 152^2 = 23104.00 \). That number is then divided by k, the number of groups; we have four groups in this scenario. So, the full process for each individual participant is:
\[\text{Participant A} = \sum{X_{Participant A}} = 152 = 152^2 = 23104.00 = \dfrac{23,104}{4} = 5776.00 \nonumber \]
Once that process is completed for each participant (so, 12 times in this scenario), we add all of the numbers in the last column (the column that divides by k). That provides the bracketed information in the formula:
\[SS_{Ps} = \left[\sum{\left(\dfrac{(\sum{X_{Ps}})^2}{k}\right)}\right] \dfrac{\left((\sum{X})^2\right)}{N} \nonumber \]
The next set of calculations (\(\dfrac{(\sum{X})^2}{N} \)) works with all of the scores. The sum of all of the scores was provided in Table \(\PageIndex{2}\) (\(\sum{X} = 2042\) ). That score is squared, then divided by the total number of scores (not the total number of people); this is 48 for this scenario because we have 12 people in four conditions (\(N = Ps \times k = 12 \times 4 = 48 \)).
\[\dfrac{(2042)^2}{48} \nonumber \]
\[\dfrac{\left(4,169,764.00\right)}{48} = 86,870.08 \nonumber \]
The final step for the Participant Sum of Square is to subtract this squared average of the total (the stuff that we just calculated) from the individual participants’ sum of squared averages (the stuff we calculated on the table) .
\[SS_{Ps} = \left[\sum{\left(\dfrac{(\sum{X_{Ps}})^2}{k}\right)}\right] \dfrac{\left((\sum{X})^2\right)}{N} = 88,166.50  86,870.08 = 1,296.42 \nonumber \]
Note that Sums of Squares should always be positive, so if your answer is not positive, then you did something wrong...
We still can’t do the Within Groups (Error) Sum of Squares because we need the Total Sum of Squares to get it.
Total SS
This is also the same formula that we used in the Between Groups ANOVA.
Calculate the Total Sum of Squares.
Solution
\[S S_{T}=\sum \left[ \left(X  \overline{X}_{T}\right)^{2} \right] \nonumber \]
This formula is also saying to subtract a mean from each score, but this time we should be subtracting the Total mean (\(\overline{X}_T = 42.54\), found in Table \(\PageIndex{1}\)). This is again easiest to compute in a table.
Table \(\PageIndex{4}\) shows the Total mean subtracted from each score in the column to the right of the raw scores, then that is squared in the column to the next right. The squared values are then summed for all of the scores.
IV Levels  Mindset Quiz Scores  minus Total mean  squared 

R1  30  12.54  157.25 
R1  31  11.54  133.17 
R1  33  9.54  91.01 
R1  35  7.54  56.85 
R1  36  6.54  42.77 
R1  38  4.54  20.61 
R1  39  3.54  12.53 
R1  42  0.54  0.29 
R1  43  0.46  0.21 
R1  47  4.46  19.89 
R1  47  4.46  19.89 
R1  49  6.46  41.73 
R2  48  5.46  29.81 
R2  39  3.54  12.53 
R2  35  7.54  56.85 
R2  32  10.54  111.09 
R2  37  5.54  30.69 
R2  33  9.54  91.01 
R2  45  2.46  6.05 
R2  52  9.46  89.49 
R2  39  3.54  12.53 
R2  51  8.46  71.57 
R2  54  11.46  131.33 
R2  55  12.46  155.25 
N1  32  10.54  111.09 
N1  35  7.54  56.85 
N1  39  3.54  12.53 
N1  40  2.54  6.45 
N1  42  0.54  0.29 
N1  42  0.54  0.29 
N1  43  0.46  0.21 
N1  45  2.46  6.05 
N1  45  2.46  6.05 
N1  46  3.46  11.97 
N1  47  4.46  19.89 
N1  51  8.46  71.57 
N2  42  0.54  0.29 
N2  32  10.54  111.09 
N2  38  4.54  20.61 
N2  50  7.46  55.65 
N2  48  5.46  29.81 
N2  51  8.46  71.57 
N2  38  4.54  20.61 
N2  48  5.46  29.81 
N2  48  5.46  29.81 
N2  51  8.46  71.57 
N2  51  8.46  71.57 
N2  48  5.46  29.81 
Sum:  \(\sigma = 2042\)  N/A  \(\sigma=2240\) 
\[S S_{T}=\sum \left[ \left(X  \overline{X}_{T}\right)^{2} \right] = 2240 \nonumber \]
Okay, now we can do the Within Groups (Error) Sum of Squares! The easiest of all!
Within Groups (Error) SS
We get this Sum of Squares indirectly, through subtraction.
Calculate the Within (Error) Sum of Squares.
Solution
\[SS_{WG} = SS_{T}  SS_{BG}  S_{P} \nonumber \]
\[SS_{WG} = 2240  244.20  1296.42 = 699.40 \nonumber \]
If you saved more decimal points for all of the equations, you might end up with something like 699.19; that’s fine!
Yay! You finished the Sum of Squares! Unfortunately, because the Within Group (Error) Sum of Squares is calculated indirectly, we cannot do a computation check for the Sum of Squares. :(
We’re not done yet!
Now, plug those four Sums of Squares into the ANOVA Summary Table (Table \(\PageIndex{5}\)) so that you can fill out the rest of the table to calculate the final ANOVA Fvalue.
Source  \(SS\)  \(df\)  \(MS\)  \(F\) 

Between  144.72  
Participants  1296.40  
Error  699.40  
Total  2240.00 
Fill out the rest of the ANOVA Summary Table.
 Answer

Table \(\PageIndex{6}\) ANOVA Summary Table with Formulas Source \(SS\) \(df\) \(MS\) \(F\) Between 144.72 \(k1 = 4 – 1 = 3\) \(\frac{S S_{B}}{d f_{B}} = \frac{144.72}{3} = 48.24 \) \(\frac{MS_{B}}{MS_{E}} = \frac{48.24}{21.19} = 2.28\) Participants 1296.40 \(P1 = 12 1 = 11\) N/A N/A Error 699.40 \((k1)*(p1) 3 * 11 = 33\) \(\frac{S S_{E}}{d f_{E}} = \frac{699.40}{33} = 21.19 \) N/A Total 2240.00 \(N1 = 48 – 1 = 47\) N/A N/A *Your answer might also be 3.84 if you use a spreadsheet that keeps all of the decimals.
You can still do a computation check with the degrees of freedom:
\[df_{T}=df_{B} + + df_{Ps} = df_{E} = 3 + 11 + 33 = 47 \nonumber \]
Since we did that part correctly, we can move on to the next step of the process with this final ANOVA Summary Table (Table \(\PageIndex{7}\)):
Source  \(SS\)  \(df\)  \(MS\)  \(F\) 

Between  144.72  3  48.24  2.28* 
Participants  1296.40  11  
Error  699.40  33  21.19  
Total  2240.00  47 
*Your answer might also be 3.84 if you use a spreadsheet that keeps all of the decimals.
Step 4: Make the Decision
Based on the completed ANOVA Summary Table (Table \(\PageIndex{7}\)), our calculated Fscore is 3.84. If you remember all the way back to Step 2, we found our critical Fscore to be 2.92.
(Critical \(<\) Calculated) \(=\) Reject null \(=\) At least one mean is different from at least one other mean. \(= p<.05\)
(Critical \(>\) Calculated) \(=\) Retain null \(=\) All of the means are similar. \(= p>.05\)
Should we retain or reject the null hypothesis? Are we saying that the means are similar, or that at least one mean is different from at least one other mean?
 Answer

We reject the null hypothesis because our critical value is smaller than our calculated value. This means that at least one time period's average Mindset Quiz score was different from at least one other time period's average Mindset Quiz score groups.
Should we conduct pairwise comparisons? Why or why not?
Solution
Since we rejected the null hypothesis that all of the means are similar, we know that at least one mean is different from at least one other mean. But since we don't know yet which means are different from which other means, we need to conduct pairwise comparisons to find if the differences in the means match our research hypothesis.
To determine which time period’s means are different from each other, we can either calculate the critical value (in this case, we’ll use Tukey’s HSD) or we can start by calculating the differences between each mean from each other. Since we have been finding the critical value (Step 2 in our process) before calculating the test statistic (Step 3 in our process), let’s keep that pattern.
Using Tukey’s HSD, what is the critical value for differences between each pair of means?
Solution
The formula for Tukey’s HSD
\[ HSD = q * \sqrt{\dfrac{MSw}{n_{group}}} \nonumber \]
requires information from the ANOVA Summary Table, as well as the qvalue from a table. We’ve been using a table of critical qvalues from RealStatistics.com: https://www.realstatistics.com/statisticstables/studentizedrangeqtable/ Just make sure to use the Alpha = 0.05 set of tables, and use k (the number of groups) not the Between Groups Degrees of Freedom.
Using that table of critical qvalues, we find with k = 4 and the Degrees of Freedom from the Within Groups (Error) row (df_{WG} = 33) that the critical qvalue is 3.825. Let’s plug that into our formula:
\[ HSD = \left(q \times \sqrt{\dfrac{MSw}{n_{group}}}\right) \nonumber \]
\[ HSD = \left(3.825 \times \sqrt{\dfrac{21.19}{12}}\right) \nonumber \]
\[ HSD = \left(3.825 \times \sqrt{1.77}\right) \nonumber \]
\[ HSD = \left(3.825 \times 1.33 \right) \nonumber \]
\[ HSD = 5.08 \nonumber \]
Now that we have our critical value, let’s calculate each pair of mean differences.
What is the difference between each pair of means?
Solution
Using the means from Table \(\PageIndex{1}\), I subtract each mean from each other mean:
\[ \bar{X}_{R1}  \bar{X}_{R2} = 39.17  43.33 = 4.16 \nonumber \]
\[ \bar{X}_{R1}  \bar{X}_{N1} = 39.17  42.25 = 3.08 \nonumber \]
\[ \bar{X}_{R1}  \bar{X}_{N2} = 39.17  45.42 = 6.25 \nonumber \]
\[ \bar{X}_{R2}  \bar{X}_{N1} = 43.33  42.25 = 1.08\nonumber \]
\[ \bar{X}_{R2}  \bar{X}_{N2} = 43.33  45.42 =2.09 \nonumber \]
\[ \bar{X}_{N1}  \bar{X}_{N2} = 42.25  45.42 = 3.17 \nonumber \]
Comparing the absolute value of each mean difference to the critical mean difference of Tukey’s HSF=5.08, we find that only one pair of means is significantly different from each other; the mean of the Beginning of Remedial English (\( \bar{X}_{R1} = 39.17 \)) is statistically lower than the mean of the End of the Next English Class (\( \bar{X}_{N2} = 45.42 \)).
WriteUp
Okay, here’s the big finish!
Write a conclusion to describe the results of the analysis. Don’t forget to include the four components necessary in any report of results.
 Answer

The researchers hypothesized that students’ average Mindset Quiz scores will be higher in the beginning of the semesters than at the end, and higher in the next class than in the first (remedial) class. This research hypothesis was partially supported (F(3,33)=3.84, p<0.05) because the mean of the Beginning of Remedial English (\(M_{R1} = 39.17\)) is statistically lower than the mean of the End of the Next English Class (\(M_{N2} = 45.42\)). All other means were similar (\(M_{R2} = 43.33, M_{N1} = 42.25\)). This suggests that it takes a long time to change mindset, or that the journaling activity was not a strong intervention.
One addition to this writeup that isn’t part of the four required components is that last sentence that’s more of a conclusion about the whole study. The class that you are taking right now isn’t just to teach you how to calculate statistics, or even to interpret them, but to think about what the results mean. If I was a college administrator, and the English professors asked for funding for journals or a paid journaling account to improve mindset, I might not fund them. I passionately want students to understand that success is possible if they keep trying and learning from their mistakes, but this data is not strong support that the journaling activity as a way to change students’ minds.
You did it! Take a break and reward yourself!
Next up, we’ll look at what to do if you don’t think that your distribution is normally distributed when you have three or more related groups…