11.3: The Foundations of Repeated-Measures ANOVA Formula

The omnibus test is performed using the ANOVA F-formula. When we test a hypothesis using repeated measures ANOVA, it is because we expect that a substantial proportion of variability will be attributed to the conditions relative to how much is attributed to random error. Therefore, the focus of the computations is how much variation is attributed to the conditions relative to how much is attributed to random error. The repeated measures ANOVA formula uses sums of squares to measure variability. Though the focus is on two sources, the variability can come from three important sources: different conditions, differences among participants, and random error. Therefore, the sums of squares must be parsed out and attributed to these three different sources so that the proportions attributed to conditions vs. random error can be compared.

The formula for repeated measures ANOVA can be written as follows:

\[F=\dfrac{M S S_{\text {between }}}{M S S_{\text {error }}} \nonumber \]

The formula can be understood as follows:

\[F=\dfrac{\text { variation between conditions }}{\text { error variance }} \nonumber \]

This is similar to the independent groups ANOVA (see Chapter 10) but with one very important difference: the denominator for repeated measures ANOVA is a computation of random error instead of the sum of squares within (\(S S_w\)). The numerator for repeated-measures ANOVA focuses on differences between conditions and the denominator focuses on random, unsystematic differences observed, also known as the error variance.

The obtained value, \(F\), tells us the ratio of difference between conditions relative to random error. Another way to say this is that it tells us how different the means for conditions are from one another after removing individual variation that is consistent and taking into account random variation that is not attributed to the conditions nor to the individuals. The goal is to see how much of the difference observed is attributed to the conditions relative to how much is attributed to random, unsystematic variation alone. The variation of interest between conditions can be thought of as systematic differences caused by the independent variable (i.e. the conditions) rather than individual differences (i.e. individual variation) or random error. The portion of variation between and attributed to the conditions is known as the treatment effects. However, when we observe variation between conditions, it will include not only the treatment effects but also some random, unsystematic variation that can occur known as random error. Because both treatment effects and random error exist between groups, random error must be accounted for in the denominator of the ANOVA formula. To summarize, we can understand the formula’s main construction as follows:

\[F=\dfrac{\text { variation between conditions }}{\text { error variance }}=\dfrac{\text { treatment effects }+ \text { random error }}{\text { random error }} \nonumber \]

The final \(F\)-value for repeated-measures ANOVA, thus, indicates the ratio of variability that is between conditions relative to that which is attributed to random error. The lowest an \(F\)-value can be for an ANOVA is 1.00. When \(F\) = 1.00, all of the observed variability is attributed to random error and not to the different conditions. This is in keeping with the null hypothesis. However, the greater the \(F\)-value, the greater the ratio of variability that is uniquely attributed to the conditions (i.e. the IV) rather than to random differences.

When there is no treatment effect, the numerator and denominator will be equal and the result will be \(F\) = 1.00. Thus, the lowest \(F\)-value possible is 1.00 which occurs when no treatment effect is observed. However, the greater the treatment effect, the greater the \(F\)-value will be.

When this formula is written out to summarize the computational elements, it is written as follows:

\[F=\dfrac{M S S_b}{M S S_e}=\dfrac{S S_{b \div} d f_b}{S S_{e \div} d f_e} \nonumber \]

Keep in mind that between calculations for repeated measures ANOVA refer to differences between conditions (i.e. differences between groups of data for the same participants) not differences between different groups of participants. Therefore, in repeated measures ANOVA there can be differences that occur among the participants within each group but there are no differences that occur from having different people in each group.

Sources of Variation in Repeated Measures ANOVA

Let’s take a moment to understand the different sources of variation for repeated-measures ANOVA before looking at the way things are calculated so we can understand the rationale for those calculations. There are three things to which variation can be attributed and estimated in ANOVA:

1. Treatment Effects (\(S S_{\text {treatment }}\)): variation attributed to the independent variable (i.e. variation attributed to exposure to different conditions);
2. Individual Variation (\(S S_{\text {participants }}\) or \(S S_p\)): variation attributed to pre-existing, systematic differences among individuals that they bring with them to each condition;
3. Error Variance (\(S S_{\text {error }}\) or \(S S_e\); also known as Residual Error): unexplained variation not accounted for or attributed to either of the aforementioned forms.

It is important to note that the proportions of variance attributed to each of these sources cannot all be directly computed. Instead, some sources can be more directly estimated while others must be deduced by partitioning them out during computations.

Parsing out Sources of Variation

For repeated-measures ANOVA, we must parse out the variation attributed to several sources before the F-formula can be used. This can be done using four important computations of variability:

1. \(S_{\text {betweenconditions }}\left(S S_b\right)\): This includes variation between groups that is attributed to the conditions/treatment (\(S S_t\)) and any random error (\(S S_e\)) together. This can also be referred to as \(SS\) between treatments or simply \(SS\) between.
2. \(S S_{\text {within }}\left(S S_w\right)\): This includes variation that exists within group caused by both participant differences that are systematic (\(S S_p\)) and random differences (error) which is non-systematic (\(S S_e\)).
   • If you compute and sum \(S S_p\) and \(S S_e\), you will get \(S S_w\)
3. \(S_{\text {participants }}\left(S S_p\right)\): This includes variation due to differences between participants that they bring with them to each condition.
   • This form of error is part of \(S S_w\) but is not part of \(S S_b\).
4. \(S S_{\text {error }}\left(S S_e\right)\): This includes variation that is random and is not attributed to the treatment (i.e. conditions) nor to differences between participants.
   • \(S S_e\) exists within both \(S S_b\) and \(S S_w\).

\(S S_b\) is short for the “sum of squares between conditions.” \(S S_b\) is the portion of variance attributed to the independent variable (i.e. the different conditions) plus random error (\(S S_e\)). \(S S_w\) is short for “sum of squares within conditions.” There are two sources of variability within conditions that together comprise the \(S S_w: 1\). The sum of squares for participants (\(S S_p\)) which is the portion of variability attributed to pre-existing differences among people and 2. The sum of squares error (\(S S_e\)) which is the variability that is attributed to randomness. Notice that both \(S S_b\) and \(S S_w\) include random error but only \(S S_w\) includes error attributed to differences between people. Therefore, we want to remove the error attributed to pre-existing differences among participants (\(S S_p\)) from the \(S S_w\) when creating our denominator; this is so that both the numerator and denominator will account for the same sources of error (namely, just the \(S S_e\)).

The sum of squares total (\(S S_T\)) can also be computed but is not a necessity so we will only briefly review it here. \(S S_T\) includes variability from all three sources (i.e. treatment effects, individual variation, and error variance) without partitioning them. \(S S_T\) can be computed directly or found by summing \(S S_b\) and \(S S_w\) because these together contain the variability for all three sources.

Reading Review 11.2

1. What two sources of variance together make up the \(SS\) between?
2. What two sources of variance together make up the \(SS\) within?
3. What source of variance is included in both the \(SS\) between and the \(SS\) within?
4. Which \(SS\) includes any variance attributed to the independent variable?