12.3.1: Practice with Mindset

Okay, here’s your time to practice the whole process of a Repeated Measures ANOVA, including each Sum of Squares formula!

Scenario

In another study by student researchers at a community college, mindset was measured with a Mindset Quiz at the beginning of the semester in a remedial English class, at the end the semester in the remedial English class, then at the beginning of the next semester in the next higher English class, and at the end of that semester in the next higher English class. Thus, the same students were measured four times to see if their mindset improved through journaling activities related to growth mindset. Mindset Quiz scores can range from 20 to 60, with higher scores showing that the student has more Growth Mindset (and less Fixed Mindset). At the end of the second semester, 12 students had completed the Mindset Quiz pre-test and post-test during each class (at the beginning and end of each semester), and had attended most classes throughout the semester to experience the journaling activities related to mindset (N = 48).

After reading the scenario, can you describe scenario?

Example \(\PageIndex{1}\)

Answer the following questions to understand the variables and groups that we are working with.

1. Who is the sample?
2. Who do might be the population?
3. What is the IV (groups being compared)?
4. What is the DV (quantitative variable being measured)?

Solution

1. The sample is 48 community college students in English classes, who started in remedial English.
2. The population could be students taking remedial English at community colleges?
3. IV: Time- Beginning of Remedial English, End of Remedial English, Beginning of Next English, End of Next English
4. DV: Mindset Quiz

Step 1: State the Hypotheses

Using the descriptive statistics in Table \(\PageIndex{1}\), what could be the research hypothesis?

Table \(\PageIndex{1}\)- Descriptive Statistics for each IV level and the Total

	N:	Mean:	SD:
Beginning of Remedial English (R1)	12	39.17	6.46
End of Remedial English (R2)	12	43.33	8.48
Beginning of Next English Class (N1)	12	42.25	5.24
End of Next English Class (N2)	12	45.42	6.35
Total	48	42.54	6.90

The notation in the parentheses is what we’ll use as the subscript labels in the formulas.

Note

You are encourages to use the raw data found in Table \(\PageIndex{2}\) in Step 3 to practice calculating the means, standard deviations, and medians yourself!

Remember, the research hypothesis should predict how all levels of the IV related to all other levels of the IV. Because this scenario has four different IV levels, there’s actually six combinations of pairs of means!

Example \(\PageIndex{2}\)

What is a research hypothesis for this scenario in words and symbols?

Solution

Research Hypothesis: Students average Mindset Quiz scores will be higher in the beginning of the semesters than at the end, and higher in the next class than in the first (remedial) class.

Your research hypothesis might be slightly different than this one. Just make sure that at least one mean is predicted to be different than one other mean.

Symbols:

• \(\overline{X}_{R1} < \overline{X}_{R2} \)
• \(\overline{X}_{R1} < \overline{X}_{N1} \)
• \(\overline{X}_{R1} < \overline{X}_{N2} \)
• \(\overline{X}_{R2} < \overline{X}_{N1} \)
• \(\overline{X}_{R2} < \overline{X}_{N2} \)
• \(\overline{X}_{N1} < \overline{X}_{N2} \)

Despite having so many combinations, the null hypothesis for this scenario is still relatively simple, just like all null hypotheses.

Exercise \(\PageIndex{1}\)

State the null hypothesis in words and symbols.

Answer

Null Hypothesis: Students will have a similar average scores on the Mindset Quiz in all four conditions: Beginning of Remedial English, End of Remedial English, Beginning of Next English Class, and End of Next English Class

Symbols: \(\overline{X}_{R1} = \overline{X}_{R2} = \overline{X}_{N1} = \overline{X}_{N2} \)

Step 2: Find the Critical Values

This step might be easier after you’ve completed the ANOVA Summary Table because you will have the Degrees of Freedom for both groups, but we’ll keep following the steps as we’ve learned them.

Exercise \(\PageIndex{2}\)

Using the Critical Values of F Table or finding the link in the Common Critical Values page at the end of the textbook, what is the critical value at \(α\) = 0.05?

Answer

Critical F(3,33) = 2.92

The first Degree of Freedom (3), for the numerator (df_B) is found through: k – 1, with k being the number of groups.

The second Degree of Freedom (33), for the denominator (df_W) is found through \((k-1)\times(Ps-1)\), with Ps being the number of participants.

\[ (k-1) \times (Ps-1) = (4-1)\times(12-1) = 3 * 11 = 33 \nonumber \]

Take a breath, because now we will start with the messy calculations!

Step 3: Compute the Test Statistic

If you will never have to calculate the Sums of Squares by hand, skip this part and just fill in the ANOVA Summary Table (Table \(\PageIndex{5}\)) at the end of this section. If you are practicing the Sums of Squares, each Sum of Square will have its own Example. Heads up, to do all of these can take about an hour!

Table \(\PageIndex{2}\)- Raw Mindset Quiz Scores for Four Conditions

Participant	Pretest in Remedial English	Posttest in Remedial English	Pretest in Next English Level	Posttest in Next English Class	Total
A	30	48	32	42	152
B	31	39	35	32	137
C	33	35	39	38	145
D	35	32	40	50	157
E	36	37	42	48	163
F	38	33	42	51	164
G	39	45	43	38	165
H	42	52	45	48	187
I	43	39	45	48	175
J	47	51	46	51	195
K	47	54	47	51	199
L	49	55	51	48	203
Sum:	470	520	507	545	2042
N:	12	12	12	12	48

Let’s go!

Between Groups SS

This is the same formula that we learned in the Between Groups ANOVA.

Example \(\PageIndex{3}\)

Calculate the Between Groups Sums of Squares.

Solution

\[S S_{B}=\sum_{EachGroup} \left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} \times (n_{group}) \right] \nonumber \]

The \(\sum_{EachGroup}\) means that you do everything following that for each intervention level, then add them all together. Let’s start with what’s inside the brackets for the Mindset Quiz at the beginning of the remedial English class (R1).

\[\text{R1} =\left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]

\(\overline{X}_{group}\) is asking for the mean of the group that we're looking at, and \(\overline{X}_{T}\) is asking for the total mean, the mean for all 48 scores. Both of these means were provided in Table \(\PageIndex{1}\). The number of scores in the group that we're looking at right now is what \(n_{group}\) is asking.

So let's plug those all in!

\[\text{R1} =\left[ \left(39.17 – 42.54\right)^{2} \times(12) \right] \nonumber \]

\[\text{R1} =\left[ \left(-3.37 \right)^{2} \times(12) \right] \nonumber \]

The mean of the group minus the mean of the sample should be negative in this scenario, but that negative sign goes away when we square it:

\[\text{R1} =\left[ \left(11.36 \right) \times(12) \right] \nonumber \]

\[\text{R1} =\left[ 136.28 \right] \nonumber \]

And let’s do that process three more times, once for the End of the Remedial English class’s semester, once for the Beginning of the Next English Class, and once for the End of the Next English class. Then we’ll add those four numbers together to get the SS_B.

\[\text{R2} =\left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]

\[\text{R2} =\left[ \left(43.33 – 42.54\right)^{2} \times(12) \right] \nonumber \]

\[\text{R2} =\left[ \left(0.79 \right)^{2} \times(12) \right] \nonumber \]

\[\text{R2} =\left[ \left(0.62 \right) \times (12) \right] \nonumber \]

\[\text{R2} =\left[ 7.44 \right] \nonumber \]

If you are doing all of your calculations in Excel or somehow saving all of the decimal points for each answer, this calculation will be a little higher (7.49). Don’t worry, the rounding differences will mostly wash out by the end of the formula. We will be using the answers provided if you type the two numbers after the decimal point into a calculator.

Why don’t you try to do it on your own for the Next English class conditions?

\[\left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]
\[\text{N1} =\left[0.96 \right] \nonumber \]
\[\text{N2} =\left[99.48 \right] \nonumber \]

Next step, add them all together! It’s easy to forget this step, but the Sum of Squares ends up to be one number, so when you get lost or forget the next step, look back at the full formula:

\[\sum_{EachGroup} \left[ \left(\overline{X}_{group}-\overline{X}_{T}\right)^{2} \times(n_{group}) \right] \nonumber \]
\[\sum_{EachGroup} = \left[ 136.32 + 7.44 + 0.96 + 99.48 \right] \nonumber \]
\[\sum_{EachGroup}= 244.20 \nonumber \]

Again, if you had saved more than two decimals, you would end up with \(S S_{B}=244.31 \). And if you used that number in the ANOVA Summary Table, the calculated F-value would be nearly identical to the same calculations but with only the two numbers after the decimal.

You did it! Only three more Sums of Squares to go!

Participant SS

This one is new!

Example \(\PageIndex{4}\)

Calculate the Participant Sum of Squares.

Solution

\[SS_{Ps} = \left[\sum{\left(\dfrac{(\sum{X_{Ps}})^2}{k}\right)}\right] -\dfrac{\left((\sum{X})^2\right)}{N} \nonumber \]

This one is easiest to do in a table, as shown in Table \(\PageIndex{3}\).

Table \(\PageIndex{3}\)- Raw Mindset Quiz Scores and Calculations for Participant Sum of Squares

Participant	Pretest in Remedial English	Posttest in Remedial English	Pretest in Next English Level	Posttest in Next English Class	Sum of Ps	Squared	Divide by k
A	30	48	32	42	152	23,104.00	5776.00
B	31	39	35	32	137	18,769.00	4692.25
C	33	35	39	38	145	21,025.00	5256.25
D	35	32	40	50	157	24,649.00	6162.25
E	36	37	42	48	163	26,569.00	6642.25
F	38	33	42	51	164	26,896.00	6724.00
G	39	45	43	38	165	27,225.00	6806.25
H	42	52	45	48	187	34,969.00	8742.25
I	43	39	45	48	175	30,625.00	7656.25
J	47	51	46	51	195	38,025.00	9506.25
K	47	54	47	51	199	39,601.00	9900.25
L	49	55	51	48	203	41,209.00	10302.25
Sum:	\(\sigma = 470\)	\(\sigma = 520\)	\(\sigma = 507\)	\(\sigma = 545\)	N/A	N/A	\(\sigma = 88,166.50\)
N:	n=12	n=12	n=12	n=12	N/A	N/A	48 scores

What’s happening in Table \(\PageIndex{3}\)? The first column is labeling each participant. When you see a column like this, then you know that the scores are related, meaning that you should run a dependent t-test or a Repeated Measures ANOVA. The two rows on the bottom (“Sum:” and “N:”) show the sum of each column, and the number of scores in that column. This makes is really easy to find the mean! The next four columns are the scores for the four conditions of the IV.

Then, we get some columns to help us calculate the Participant Sum of Squares. The Sum of Ps column just adds the four scores for each individual participant. For example,

\[\text{Sum of Participant A} = 30 + 48 + 32 = 42 = 152 \nonumber \]

The next column, Squared, is the individual participant’s summed scores squared \(\text{Participant A} = 152^2 = 23104.00 \). That number is then divided by k, the number of groups; we have four groups in this scenario. So, the full process for each individual participant is:

\[\text{Participant A} = \sum{X_{Participant A}} = 152 = 152^2 = 23104.00 = \dfrac{23,104}{4} = 5776.00 \nonumber \]

Once that process is completed for each participant (so, 12 times in this scenario), we add all of the numbers in the last column (the column that divides by k). That provides the bracketed information in the formula:

\[SS_{Ps} = \left[\sum{\left(\dfrac{(\sum{X_{Ps}})^2}{k}\right)}\right] -\dfrac{\left((\sum{X})^2\right)}{N} \nonumber \]

The next set of calculations (\(\dfrac{(\sum{X})^2}{N} \)) works with all of the scores. The sum of all of the scores was provided in Table \(\PageIndex{2}\) (\(\sum{X} = 2042\) ). That score is squared, then divided by the total number of scores (not the total number of people); this is 48 for this scenario because we have 12 people in four conditions (\(N = Ps \times k = 12 \times 4 = 48 \)).

\[\dfrac{(2042)^2}{48} \nonumber \]

\[\dfrac{\left(4,169,764.00\right)}{48} = 86,870.08 \nonumber \]

The final step for the Participant Sum of Square is to subtract this squared average of the total (the stuff that we just calculated) from the individual participants’ sum of squared averages (the stuff we calculated on the table) .

\[SS_{Ps} = \left[\sum{\left(\dfrac{(\sum{X_{Ps}})^2}{k}\right)}\right] -\dfrac{\left((\sum{X})^2\right)}{N} = 88,166.50 - 86,870.08 = 1,296.42 \nonumber \]

Note that Sums of Squares should always be positive, so if your answer is not positive, then you did something wrong...

We still can’t do the Within Groups (Error) Sum of Squares because we need the Total Sum of Squares to get it.

Total SS

This is also the same formula that we used in the Between Groups ANOVA.

Example \(\PageIndex{5}\)

Calculate the Total Sum of Squares.

Solution

\[S S_{T}=\sum \left[ \left(X - \overline{X}_{T}\right)^{2} \right] \nonumber \]

This formula is also saying to subtract a mean from each score, but this time we should be subtracting the Total mean (\(\overline{X}_T = 42.54\), found in Table \(\PageIndex{1}\)). This is again easiest to compute in a table.

Table \(\PageIndex{4}\) shows the Total mean subtracted from each score in the column to the right of the raw scores, then that is squared in the column to the next right. The squared values are then summed for all of the scores.

Table \(\PageIndex{4}\)- Total Mean Subtracted from Each Score, Squared, then Summed

IV Levels	Mindset Quiz Scores	minus Total mean	squared
R1	30	-12.54	157.25
R1	31	-11.54	133.17
R1	33	-9.54	91.01
R1	35	-7.54	56.85
R1	36	-6.54	42.77
R1	38	-4.54	20.61
R1	39	-3.54	12.53
R1	42	-0.54	0.29
R1	43	0.46	0.21
R1	47	4.46	19.89
R1	47	4.46	19.89
R1	49	6.46	41.73
R2	48	5.46	29.81
R2	39	-3.54	12.53
R2	35	-7.54	56.85
R2	32	-10.54	111.09
R2	37	-5.54	30.69
R2	33	-9.54	91.01
R2	45	2.46	6.05
R2	52	9.46	89.49
R2	39	-3.54	12.53
R2	51	8.46	71.57
R2	54	11.46	131.33
R2	55	12.46	155.25
N1	32	-10.54	111.09
N1	35	-7.54	56.85
N1	39	-3.54	12.53
N1	40	-2.54	6.45
N1	42	-0.54	0.29
N1	42	-0.54	0.29
N1	43	0.46	0.21
N1	45	2.46	6.05
N1	45	2.46	6.05
N1	46	3.46	11.97
N1	47	4.46	19.89
N1	51	8.46	71.57
N2	42	-0.54	0.29
N2	32	-10.54	111.09
N2	38	-4.54	20.61
N2	50	7.46	55.65
N2	48	5.46	29.81
N2	51	8.46	71.57
N2	38	-4.54	20.61
N2	48	5.46	29.81
N2	48	5.46	29.81
N2	51	8.46	71.57
N2	51	8.46	71.57
N2	48	5.46	29.81
Sum:	\(\sigma = 2042\)	N/A	\(\sigma=2240\)

\[S S_{T}=\sum \left[ \left(X - \overline{X}_{T}\right)^{2} \right] = 2240 \nonumber \]

Okay, now we can do the Within Groups (Error) Sum of Squares! The easiest of all!

Within Groups (Error) SS

We get this Sum of Squares indirectly, through subtraction.

Example \(\PageIndex{6}\)

Calculate the Within (Error) Sum of Squares.

Solution

\[SS_{WG} = SS_{T} - SS_{BG} - S_{P} \nonumber \]

\[SS_{WG} = 2240 - 244.20 - 1296.42 = 699.40 \nonumber \]

If you saved more decimal points for all of the equations, you might end up with something like 699.19; that’s fine!

Yay! You finished the Sum of Squares! Unfortunately, because the Within Group (Error) Sum of Squares is calculated indirectly, we cannot do a computation check for the Sum of Squares. :(

We’re not done yet!

Now, plug those four Sums of Squares into the ANOVA Summary Table (Table \(\PageIndex{5}\)) so that you can fill out the rest of the table to calculate the final ANOVA F-value.

Table \(\PageIndex{5}\)- ANOVA Summary Table with SS only

Source	\(SS\)	\(df\)	\(MS\)	\(F\)
Between	144.72
Participants	1296.40
Error	699.40
Total	2240.00

Exercise \(\PageIndex{3}\)

Fill out the rest of the ANOVA Summary Table.

Answer

Table \(\PageIndex{6}\)- ANOVA Summary Table with Formulas

Source	\(SS\)	\(df\)	\(MS\)	\(F\)
Between	144.72	\(k-1 = 4 – 1 = 3\)	\(\frac{S S_{B}}{d f_{B}} = \frac{144.72}{3} = 48.24 \)	\(\frac{MS_{B}}{MS_{E}} = \frac{48.24}{21.19} = 2.28\)
Participants	1296.40	\(P-1 = 12 -1 = 11\)	N/A	N/A
Error	699.40	\((k-1)*(p-1) 3 * 11 = 33\)	\(\frac{S S_{E}}{d f_{E}} = \frac{699.40}{33} = 21.19 \)	N/A
Total	2240.00	\(N-1 = 48 – 1 = 47\)	N/A	N/A

*Your answer might also be 3.84 if you use a spreadsheet that keeps all of the decimals.

You can still do a computation check with the degrees of freedom:

\[df_{T}=df_{B} + + df_{Ps} = df_{E} = 3 + 11 + 33 = 47 \nonumber \]

Since we did that part correctly, we can move on to the next step of the process with this final ANOVA Summary Table (Table \(\PageIndex{7}\)):

Table \(\PageIndex{7}\)- Completed ANOVA Summary Table for Mindset by English Class

Source	\(SS\)	\(df\)	\(MS\)	\(F\)
Between	144.72	3	48.24	2.28*
Participants	1296.40	11
Error	699.40	33	21.19
Total	2240.00	47

*Your answer might also be 3.84 if you use a spreadsheet that keeps all of the decimals.

Step 4: Make the Decision

Based on the completed ANOVA Summary Table (Table \(\PageIndex{7}\)), our calculated F-score is 3.84. If you remember all the way back to Step 2, we found our critical F-score to be 2.92.

(Critical \(<\) Calculated) \(=\) Reject null \(=\) At least one mean is different from at least one other mean. \(= p<.05\)

(Critical \(>\) Calculated) \(=\) Retain null \(=\) All of the means are similar. \(= p>.05\)

Exercise \(\PageIndex{4}\)

Should we retain or reject the null hypothesis? Are we saying that the means are similar, or that at least one mean is different from at least one other mean?

Answer

We reject the null hypothesis because our critical value is smaller than our calculated value. This means that at least one time period's average Mindset Quiz score was different from at least one other time period's average Mindset Quiz score groups.

Example \(\PageIndex{6}\)

Should we conduct pairwise comparisons? Why or why not?

Solution

Since we rejected the null hypothesis that all of the means are similar, we know that at least one mean is different from at least one other mean. But since we don't know yet which means are different from which other means, we need to conduct pairwise comparisons to find if the differences in the means match our research hypothesis.

To determine which time period’s means are different from each other, we can either calculate the critical value (in this case, we’ll use Tukey’s HSD) or we can start by calculating the differences between each mean from each other. Since we have been finding the critical value (Step 2 in our process) before calculating the test statistic (Step 3 in our process), let’s keep that pattern.

Example \(\PageIndex{7}\)

Using Tukey’s HSD, what is the critical value for differences between each pair of means?

Solution

The formula for Tukey’s HSD

\[ HSD = q * \sqrt{\dfrac{MSw}{n_{group}}} \nonumber \]

requires information from the ANOVA Summary Table, as well as the q-value from a table. We’ve been using a table of critical q-values from RealStatistics.com: https://www.real-statistics.com/statistics-tables/studentized-range-q-table/ Just make sure to use the Alpha = 0.05 set of tables, and use k (the number of groups) not the Between Groups Degrees of Freedom.

Using that table of critical q-values, we find with k = 4 and the Degrees of Freedom from the Within Groups (Error) row (df_WG = 33) that the critical q-value is 3.825. Let’s plug that into our formula:

\[ HSD = \left(q \times \sqrt{\dfrac{MSw}{n_{group}}}\right) \nonumber \]

\[ HSD = \left(3.825 \times \sqrt{\dfrac{21.19}{12}}\right) \nonumber \]

\[ HSD = \left(3.825 \times \sqrt{1.77}\right) \nonumber \]

\[ HSD = \left(3.825 \times 1.33 \right) \nonumber \]

\[ HSD = 5.08 \nonumber \]

Now that we have our critical value, let’s calculate each pair of mean differences.

Example \(\PageIndex{8}\)

What is the difference between each pair of means?

Solution

Using the means from Table \(\PageIndex{1}\), I subtract each mean from each other mean:

\[ \bar{X}_{R1} - \bar{X}_{R2} = 39.17 - 43.33 = -4.16 \nonumber \]

\[ \bar{X}_{R1} - \bar{X}_{N1} = 39.17 - 42.25 = -3.08 \nonumber \]

\[ \bar{X}_{R1} - \bar{X}_{N2} = 39.17 - 45.42 = -6.25 \nonumber \]

\[ \bar{X}_{R2} - \bar{X}_{N1} = 43.33 - 42.25 = 1.08\nonumber \]

\[ \bar{X}_{R2} - \bar{X}_{N2} = 43.33 - 45.42 =-2.09 \nonumber \]

\[ \bar{X}_{N1} - \bar{X}_{N2} = 42.25 - 45.42 = -3.17 \nonumber \]

Comparing the absolute value of each mean difference to the critical mean difference of Tukey’s HSF=5.08, we find that only one pair of means is significantly different from each other; the mean of the Beginning of Remedial English (\( \bar{X}_{R1} = 39.17 \)) is statistically lower than the mean of the End of the Next English Class (\( \bar{X}_{N2} = 45.42 \)).

Write-Up

Okay, here’s the big finish!

Exercise \(\PageIndex{5}\)

Write a conclusion to describe the results of the analysis. Don’t forget to include the four components necessary in any report of results.

Answer

The researchers hypothesized that students’ average Mindset Quiz scores will be higher in the beginning of the semesters than at the end, and higher in the next class than in the first (remedial) class. This research hypothesis was partially supported (F(3,33)=3.84, p<0.05) because the mean of the Beginning of Remedial English (\(M_{R1} = 39.17\)) is statistically lower than the mean of the End of the Next English Class (\(M_{N2} = 45.42\)). All other means were similar (\(M_{R2} = 43.33, M_{N1} = 42.25\)). This suggests that it takes a long time to change mindset, or that the journaling activity was not a strong intervention.

One addition to this write-up that isn’t part of the four required components is that last sentence that’s more of a conclusion about the whole study. The class that you are taking right now isn’t just to teach you how to calculate statistics, or even to interpret them, but to think about what the results mean. If I was a college administrator, and the English professors asked for funding for journals or a paid journaling account to improve mindset, I might not fund them. I passionately want students to understand that success is possible if they keep trying and learning from their mistakes, but this data is not strong support that the journaling activity as a way to change students’ minds.

You did it! Take a break and reward yourself!

Next up, we’ll look at what to do if you don’t think that your distribution is normally distributed when you have three or more related groups…