10.4: Using the ANOVA Formula

Formula Component Summary

Now that we have taken some time to understand the construction of the one-way ANOVA formula, let’s focus on how to actually use it, starting with identifying all of its parts.

In order to solve for \(F\), four things must first be known:

\(SS_w\) = sum of squares within groups

\(df_w\) = degrees of freedom within groups

\(SS_b\) = sum of squares between groups

\(df_b\) = degrees of freedom between groups

Formula Steps

The steps are shown in order and categorized into two sections:

1. preparation and
2. solving.

Preparation steps refer to any calculations that need to be done before values can be plugged into the formula. For the one-way ANOVA this includes finding several preparatory components needed for the formula (listed in the section above). Once those are known, the steps in section B can be used to yield the obtained value for the formula. The symbol for the obtained value for ANOVA is \(F\). Follow these steps, in order, to find \(F\).

Section A: Preparation

1. Find \(k\) (the number of independent groups being compared)
2. Find the \(n\) for each group
3. Find \(N\) (the total sample size across all independent groups being compared)
4. Find \(\bar{x}\) for each group
5. Find the \(\bar{x}_{\text {grand }}\) (grand mean)

Note

We will need the mean and standard deviation for each group when reporting results in APA-format. Therefore, it can be useful to compute the standard deviation for each group during the preparatory steps even though they are not needed for the ANOVA formula.

Section B: Solving

The values from the preparatory steps must now be used to find the four main components of the ANOVA formula before the \(F\)-value can be found.

1. Find the \(SS_w\) for each independent group then sum them to get the total \(SS_w\): \[SS_w=\Sigma(x-\bar{x})^2 \nonumber \]
   1. Subtract the mean for Group 1 from each raw score in Group 1 to find this groups deviations.
   2. Square the deviations for Group 1.
   3. Sum the squared deviations to get the \(SS_{w 1}\) (which is the Sum of Squares Within for Group 1).
   4. Repeat steps 1a through 1c for each group until the \(SS_w\) for each group is known.
   5. Add the \(SS_w\) (Sum of Squares Within) for all groups together to get the total \(SS_w\).
2. Find the \(df_w\) by subtracting the number of groups (\(k\)) from the total of sample sizes (\(N\)). \[df_w=N-k \nonumber \]
3. Find the total \(SS_b\) across all groups: \[S S_b=\Sigma n_i\left[\left(\bar{x}_i-\bar{x}_{\text {grand }}\right)^2\right] \nonumber \]
   1. Subtract the grand mean (\(\bar{x}_{\text {grand }}\)) from the mean for Group 1 (\(\bar{x}_1\))
   2. Square the group deviation from step 3a.
   3. Multiply the squared deviation for the group (which is the result of step 3b) by the sample size for Group 1 (\(n_1\)) to find the Sum of Squares Between for Group 1 (\(SS_b\)).
   4. Repeat steps 1a through 1c for each group until the \(SS_b\) for each group is known.
   5. Add the \(SS_b\) (Sum of Squares Between) for all groups together to get the total \(SS_b\).
4. Find the \(df_b\) by subtracting 1 from the number of groups (\(k\)). \[d f_b=k-1 \nonumber \]
5. Write the ANOVA formula with the four values found in the above steps (i. e. \(SS_w\), \(df_w\), \(SS_b\), and \(df_b\)) plugged into their respective locations. \[F=\dfrac{SS_b \div d f_b}{SS_w \div d f_w} \nonumber \]
6. Solve for \(MSS_b\) by dividing \(SS_b\) by \(df_b\). This gives you the numerator for the \(F\) formula. \[MSS_b=SS_b \div df_b \nonumber \]
7. Solve for \(MSS_w\) by dividing \(SS_w\) by \(df_w\). This gives you the denominator for the \(F\) formula. \[MSS_w=SS_w \div d f_w \nonumber \]
8. Finally, divide \(MSS_b\) (which is the result of step 6) by \(MSS_w\) (which is the result of step 7) to get the obtained \(F\)-value. \[F=\dfrac{MSS_b}{MSS_w} \nonumber \]

Example of How to Test a Hypothesis by Computing \(F\).

Let us assume that a researcher believed that children in three different conditions would differ in their mean number of aggressive behaviors toward a toy. Suppose that aggression was measured in number of physically aggressive acts (such as individual hits, kicks, and throws) towards a toy for three different groups of children. Group 1 has been shown an adult acting aggressively toward the toy (such as by hitting, kicking, and throwing it), Group 2 has been shown an adult playing non-aggressively with the toy (such as by picking it up, sitting it down, and pretending to share with it), while Group 3 is not shown the toy or any interactions with it. Assume that Data Set 10.1 includes data from the three independent samples. Let’s use this information to follow the steps in hypothesis testing.

Data Set 10.1. Aggressive Acts by Children in Three Independent Viewing Groups (\(n\) = 21)

Group 1	Group 2	Group 3
5	3	0
7	1	1
6	2	0
4	4	3
8	2	1
6	2	1
6	0	1

Steps in Hypothesis Testing

In order to test a hypothesis, we must follow these steps:

1. State the hypothesis.

A summary of the research hypothesis and corresponding null hypothesis in sentence and symbol format are shown below. However, researchers often only state the research hypothesis using a format like this: It is hypothesized that the mean acts of aggression will be different among children in three different conditions. The format shown below can also be used instead. Remember, a one-tailed test is used when conducting the omnibus test in ANOVA, regardless of whether the hypothesis is stated directionally.

Non-Directional Hypothesis for a One-Way ANOVA

Research hypothesis	The mean of the groups are not all equal to each other.	\(H_A: \mu_1 \neq \mu_2 \neq \mu_3 \ldots\)
Null hypothesis	The mean of the groups are all equal to each other.	\(H_0: \mu_1=\mu_2=\mu_3 \ldots\)

2. Choose the inferential test (formula) that best fits the hypothesis.

The means of three independent samples are being compared so the appropriate test is a one-way ANOVA.

3. Determine the critical value.

In order to determine the critical value for a one-way ANOVA, three things must be identified:

1. the alpha level,
2. the Degrees of Freedom Between (\(df_b\)), and,
3. the Degrees of Freedom Within (\(df_w\)).

The alpha level is often set at .05 unless there is reason to adjust it such as when multiple hypotheses are being tested in one study or when a Type I Error could be particularly problematic. The default alpha level can be used for this example because only one hypothesis is being tested and there is no clear indication that a Type I Error would be especially problematic. Thus, alpha can be set to 5%, which can be summarized as \(\alpha\) = .05.

The \(df_b\) and the \(df_w\) must also be calculated. These will be used to find the critical value and these will also be important pieces in the ANOVA formula. Let’s find each for Data Set 10.1. There are three groups (\(k\) = 3) and each has a sample size of 7 (\(n_1 = 7, n_2 = 7\), and \(n_3 = 7\)) The total sample size across the three groups is 21 (\(N\) = 21).

\[df_w=N-k=21-3=18 \nonumber \]

\[df_b=k-1=3-1=2 \nonumber \]

These three pieces of information are used to locate the critical value for the test. The full tables of the critical values for \(F\)-tests are located in Appendix E. Below is an excerpt of the section of the \(F\)-tables that fits the current hypothesis and data. Under the conditions of an alpha level of .05, \(df_b\) = 2, and \(df_w\) = 18, the critical value is 3.555.

Critical Values Table

	Degrees of Freedom Between (\(df_b\))
		1	2	3	4	5	6	7	8
Degrees of Freedom Within (\(df_w\))	18	4.414	3.555	3.160	2.928	2.773	2.661	2.577	2.510

The critical value represents the value which must be exceeded in order to declare a result statistically significant. The obtained value (which is called \(F\) in an ANOVA) is the amount of evidence present. \(F\)-values will always be positive so we do not have to worry about any negative signs when comparing the obtained value for F to the critical value; only the magnitude of the obtained \(F\)-value must be considered. Thus, in order for the result to significantly support the hypothesis is needs to exceed the critical value of 3.555.

4. Calculate the test statistic.

A test statistic can also be referred to as an obtained value. The formula needed to find the test statistics, known as \(F\) for this scenario, is as follows:

\[F=\dfrac{MSS_b}{MSS_w} \nonumber \]

But we must remember that the MSS in each section stands for the mean sum of squares and that each of these is actually comprised of two parts: an \(SS\) and a \(df\). Thus, it is more helpful to break the formula out to show those pieces as follows:

\[F=\dfrac{SS_b \div df_b}{SS_w \div df_w}=\dfrac{MSS_b}{MSS_w} \nonumber \]

Section A: Preparation

Start each inferential formula by identifying and solving for the pieces that must go into the formula. For the one-way ANOVA, this preparatory work is as follows:

1. Find \(k\) (the number of independent groups being compared).

This value is found using Data Set 10.1 and is summarized as \(k\) = 3

2. Find the \(n\) for each group.

This value is found using Data Set 10.1 for each group and is summarized as follows:

\(n_1\) = 7

\(n_2\) = 7

\(n_3\) = 7

3. Find \(N\) (the total sample size across all independent groups being compared). This value is found using Data Set 10.1 and is summarized as follows:

\[\begin{aligned}
& N=n_1+n_2+n_3 \\
& N=7+7+7 \\
& N=21
\end{aligned} \nonumber \]

4. Find \(\bar {x}\) for each group.

These values are found using Data Set 10.1 and are summarized as follows:

\(\bar{X}_1\) = 6.00

\(\bar{X}_2\) = 2.00

\(\bar{X}_3\) = 1.00

5. Find the \(\bar{x}_{\text {grand }}\) (grand mean).

This value is found using Data Set 10.1 and is summarized as follows: 21= 63

\[\bar{x}_{\text {grand }}=\dfrac{5+7+6+4+8+6+6+3+1+2+4+2+2+0+0+1+0+3+1+1+1}{21}=\dfrac{63}{21}=3.00 \nonumber \]

Now that the pieces needed for the formula have been found, we can move to Section B.

Section B: Solving

The values from the preparatory steps must now be used to find the four main components of the ANOVA formula before the \(F\)-value can be solved. The calculations for steps 1a through 1d are shown in the calculations table below.

1. Find the \(SS_w\) for each independent group then sum them to get the total \(SS_w\):
   1. Subtract the mean for Group 1 from each raw score in Group 1 to find their deviations.
   2. Square the deviations for Group 1.
   3. Sum the squared deviations to get the \(SS_w 1\) (which is the Sum of Squares Within for group 1).
   4. Repeat steps 1a through 1c for each group until the \(SS_w\) for each group is known.

Note

The summarized calculations are shown below for reference. For a detailed review of how to calculate a sample mean, see Chapter 3. For a detailed review of how to calculate sums of squares within, see Chapter 4.

Calculations for Descriptive Statistics and \(SS_w\) for each Group in Data Set 10.1

Group 1	\(X- \bar{X}_1\)	\((X- \bar{X}_1)^2\)	Group 2	\((X- \bar{X}_2)^2\)	\((X- \bar{X}_2)^2\)	Group 3	\(X- \bar{X}_3\)	\((X- \bar{X}_3)^2\)
5	(-1)	1	3	1	1	0	(-1)	1
7	1	1	1	(-1)	1	1	0	0
6	0	0	2	0	0	0	(-1)	1
4	(-2)	4	4	2	4	3	2	4
8	2	4	2	0	0	1	0	0
6	0	0	2	0	0	1	0	0
6	0	0	0	(-2)	4	1	0	0
\(n_1=7\)	\(SS_w1=10.00\)		\(n_2=7\)	\(SS_w2=10.00\)		\(n_3=7\)	\(SS_w3=10.00\)
\(\bar{X}_1=6.00\)			\(\bar{X}_2=2.00\)			\(\bar{X}_3=1.00\)

e. Add the \(SS_w\) (Sum of Squares Within) for all groups together to get the total \(SS_w\). \[\begin{gathered}
S S_w=S S_{w 1}+S S_{w 2}+S S_{w 3} \\
S S_w=10.00+10.00+6.00 \\
S S_w=26.00
\end{gathered} \nonumber \]

2. Find the \(df_w\) by subtracting the number of groups (\(k\)) from the total of sample sizes (\(N\)). \[\begin{gathered} d f_w=N-k=21-3=18 \\d f_w=18 \end{gathered} \nonumber \]
3. Find the total \(SS_b\) across all groups. The formula for \(SS_b\) for each group is as follows: \[S S_{\text {b_group }}=n_{\text {group }}\left[\left(\bar{x}_{\text {group }}-\bar{x}_{\text {grand }}\right)^2\right] \nonumber \] We must use this formula to find the \(SS_b\) for each group. We will fill in the formula then break down the three steps to using it. If the information is filled in for Group 1, we get the following:\[SS_{b 1}=7\left[(6.00-3.00)^2\right] \nonumber \] Now we can use order of operations to find the Sum of Squares Between for Group 1 as follows:
   1. Subtract the grand mean (\(\bar{x}_{\text {grand }}\)) from the mean for Group 1 (\(\bar{x}\)) to get the group deviation. \[\begin{gathered} \bar{x}_1-\bar{x}_{\text {grand }} \\6.00-3.00 \\3.00\end{gathered} \nonumber \]
   2. Square the group deviation from step 3a. \[3.00^2=9.00 \nonumber \]
   3. Multiply the squared deviation for the group (which is the result of step 3b) by the sample size for Group 1 (\(n_1\)) to find the Sum of Squares Between for Group 1 (\(SS_b\)). \[\begin{gathered} S S_{b 1}=7(9.00) \\S S_{b 1}=63.00\end{gathered} \nonumber \]
   4. Repeat steps 1a through 1c for each group until the \(SS_b\) for each group is known.

      Group 2 Calculations	Group 2 Calculations
      \(SS_{b2}=n_2\left[\left(bar{x}_2-\bar{x}_{\text {grand }}\right)^2\right]\)	\(SS_{b3}=n_3\left[\left(bar{x}_3-\bar{x}_{\text {grand }}\right)^2\right]\)
      \(SS_{b 2}=7\left[(2.00-3.00)^2\right]\)	\(SS_{b 3}=7\left[(1.00-3.00)^2\right]\)
      \(SS_{b 2}=7\left[(-1.00)^2\right]\)	\(SS_{b 3}=7\left[(-2.00)^2\right]\)
      \(SS_{b 2}=7(1.00)\)	\(SS_{b 3}=7(4.00)\)
      \(SS_{b 2}=7.00\)	\(SS_{b 3}=28.00\)

   5. Add the \(SS_b\) (sum of squares between) for all groups together to get the total \(SS_b\). \[\begin{gathered} SS_b=SS_{b 1}+SS_{b 2}+SS_{b 3} \\ SS_b=63.00+7.00+28.00 \\SS_b=98.00 \end{gathered} \nonumber \]
4. Find the \(df_b\) by subtracting 1 from the number of groups (\(k\)). \[\begin{gathered} df_b=k-1=3-1=2 \\df_b=2 \end{gathered} \nonumber \]
5. Write the ANOVA formula with the four values found in the prior steps (i. e. \(SS_w\), \(df_w\), \(SS_b\), and \(df_b\)) plugged into their respective locations. \[\begin{aligned} F & =\dfrac{S S_b \div d f_b}{S S_w \div d f_w} \\ F & =\dfrac{98.00 \div 2}{26.00 \div 18}\end{aligned} \nonumber \]
6. Solve for \(MSS_b\) by dividing \(SS_b\) by \(df_b\). This gives you the numerator for the \(F\) formula. \[F=\dfrac{49.00}{26.00 \div 18} \nonumber \]
7. Solve for \(MSS_w\) by dividing \(SS_w\) by \(df_w\). This gives you the denominator for the \(F\) formula. \[F=\dfrac{49.00}{\mathbf{1.4444 \ldots}} \nonumber \]
8. Finally, divide \(MSS_b\) (which is the result of step 6) by \(MSS_w\) (which is the result of step 7) to get the obtained \(F\)-value. \[F=\mathbf{33.9230 \ldots} \nonumber \] The obtained value for this test is 33.92 when rounded to the hundredths place.

5. Apply a decision rule and determine whether the result is significant.

Assess whether the obtained value for \(F\) exceeds the critical value as follows: The critical value is 3.555.

The obtained \(F\)-value is 33.92.

The obtained \(F\)-value exceeds (i.e. is greater than) the critical value. Therefore, the criteria has been met to declare the result significant. This result is significant and supports the hypothesis.

Note

If the hypothesis had a direction, the directions of each group by group comparison would need to be checked in the post-hoc analyses before concluding that the hypothesis had been supported. However, the current hypothesis was simply that groups would differ; thus, we are able to conclude that the hypothesis was supported before checking the post-hoc results. In this scenario, we will still use the post-hoc analyses to determine which group mean(s) were different from which other group mean(s) so that we can provide more detailed information in our APA-formatted summary of the results.

6. Calculate the effect size and/or other relevant secondary analyses.

When it is determined that the result is significant, effect sizes should be computed. Because the result was determined to be significant in step 5, the effect size is needed before proceeding to step 7.

The effect size that is appropriate for a one-way ANOVA is a calculation of the percent of variance observed that was systematic (and, thus, was uniquely between groups) which is reported in decimal form. The symbol for this effect size is \(\eta^2\) (which is named “eta squared”). The formula is as follows:

\[\eta^2=\dfrac{SS_b}{SS_T} \nonumber \]

To calculate this, two things need to be known:

1. the Sum of Squares Between groups (\(SS_b\)) and
2. the Sum of Squares Total (\(SS_T\)).

We have already found \(SS_b\) in earlier steps but still need to find \(SS_T\). \(SS_T\) refers to the total sum of squared deviations observed overall for the ANOVA; this is found by summing the \(SS_b\) and the \(SS_w\). We already know \(SS_w\) from earlier steps. We can summarize the two things we do know and use them to find the one we need as follows:

\[\begin{gathered}
S S_b=98.00 \\
S S_w=26.00 \\
S S_T=S S_b+S S_w=98.00+26.00=124.00
\end{gathered} \nonumber \]

Now that we have the pieces needed, we can use this formula to find the proportion of variance that was uniquely accounted for between groups. The calculations for the current data would be as follows:

\[\begin{aligned}
\eta^2 & =\dfrac{98.00}{124.00} \\
\eta^2 & =0.7903 \ldots
\end{aligned} \nonumber \]

Effect sizes, like most values, are rounded and reported to the hundredths place. Thus, this effect size is reported as \(\eta^2=0.79\).

This translates to 79% of the variance being accounted for by differences between groups rather than within groups. The largest the effect size can be is 1.00 because that would mean 100% of the variance was accounted for by between group differences. In keeping, the lowest an effect size can be is 0.00 because that would mean that 0% of the variance was accounted for by between group differences. Thus, the closer the effect size is to 1.00, the larger it is, and the closer the effect size is to 0.00, the smaller the effect size is. The current effect size of 0.79 would be considered a large effect.

Post-Hoc Analyses for One-Way ANOVA

Comparing the means of three or more independent groups requires two processes:

1. An omnibus test to know whether all means are approximately equal vs. whether at least one mean is significantly different that at least one other mean and,
2. A post-hoc analysis which can specify which means are significantly different than which other means. Post-hoc analyses are warranted when an omnibus result is significant. The omnibus test was significant for Data Set 10.1 and, thus, hos-hoc testing is necessary.

Post-hoc tests for ANOVA are pairwise tests. This means that groups will be compared in pairs (meaning two at a time). The commonly used version of this for data that meet the assumptions for a one-way ANOVA is known as Tukey’s Honestly Significant Difference (\(HSD\)) post-hoc test. To use this by hand, an \(HSD\) value is calculated and represents the minimum difference that must be observed between any two groups in order to declare the means of the pair significantly different from one another. The \(HSD\) formula is as follows:

\[HSD=q \sqrt{\dfrac{MS_w}{n}} \nonumber \]

Three things are needed to find an \(HSD: MS_w\), \(n\), and \(q\). We already solved for two of those during omnibus testing but still must find \(q\). In this formula, \(q\) is known as the studentized range statistic. We can locate \(q\) using the table in Appendix F. To find \(q\), we need to know \(k\) and \(df_w\). Using these, we can find \(q = 3.609\) in Appendix F.

\[H S D=3.609 \sqrt{\frac{1.4444 \ldots}{7}}=1.6394 \ldots \approx 1.64 \nonumber \]

Then, the pairwise differences must be calculated to compare them to the \(HSD\) threshold. The pairwise differences in means for Data Set 10.1 are as follows:

Descriptive Statistics by Group
Group 1: \(M\) = 6.00, \(SD\) = 1.29	Group 2: \(M\) = 2.00, \(SD\) = 1.29	Group 3: \(M\) = 1.00, \(SD\) = 1.00
Pairwise Comparisons
Group 1 vs. Group 2	Group 1 vs. Group 3	Group 2 vs. Group 3
6.00 – 2.00 = 4.00	6.00 – 1.00 = 5.00	2.00 – 1.00 = 1.00
4.00 > 1.64, \(p\) < .05	5.00 > 1.64, \(p\) < .05	1.00 < 1.64, \(p\) > .05

The pairwise comparisons indicate that the means of Group 1 and Group 2 are significantly different, the means of Group 1 and Group 3 are significantly different, but that the means of Group 2 and Group 3 are not significantly different from one another.

7. Report the results in American Psychological Associate (APA) format.

Results for inferential tests are often best summarized using a paragraph that states the following:

1. the hypothesis and specific inferential test used,
2. the main results of the test and whether they were significant,
3. any additional results that clarify or add details about the results,
4. whether the results support or refute the hypothesis.

APA format requires a specific organization be used for reporting the results of a test. This includes a specific format for reporting relevant symbols and details for the formula and data used. Following this, the results for our hypothesis with Data Set 10.1 can be written as shown in the summary example below.

APA Formatted Summary Example

A one-way ANOVA was used to test the hypothesis that the mean acts of aggression would be different among children in three different conditions. Consistent with the hypothesis, the mean acts of aggression were different among the conditions, \(F(2, 18) = 33.92\), \(p\) < .05. The effect size of 0.79 was large. Tukey’s \(HSD\) post-hoc results indicate that the mean acts of aggression was significantly higher in group exposed to aggressive acts (\(M\) = 6.00; \(SD\) = 1.29) than the group exposed to neutral acts (\(M\) = 2.00; \(SD\) = 1.29) and the group which was not exposed to either (\(M\) = 1.00; \(SD\) = 1.00), \(p\) < .05. However, the means were not significantly different between Group 2 and Group 3, \(p\) > .05.

As always, the APA-formatted summary provides a lot of detail in a particular order. For a brief review of the structure for the APA-formatted summary of the omnibus test results, see the summary below.

Summary of APA-Formatted Results for the One-Way ANOVA

In your APA write up for a one-way ANOVA you should state:

1. Which test was used and the hypothesis which warranted its use.
2. Whether the aforementioned hypothesis was supported or not. To do so properly, three components must be reported:
   1. The mean and standard deviation for each group
   2. The test results in an evidence string as follows: \(F\left(df_b, df_w\right) \) = obtained value
   3. The significance portion of the evidence string for the omnibus test as \(p\) < .05 if significant or \(p\) > .05, \(ns\) if not significant
   4. The effect size (Note: this part is only required if the omnibus result was significant).
   5. The significance for the post-hoc pairwise comparisons as \(p\) < .05 if significant or \(p\) > .05, \(ns\) if not significant (Note: this part is only required if the omnibus result was significant).

Anatomy of the Evidence String

The following breaks down what each part represents in the evidence string for the omnibus results in the APA-formatted paragraph above:

Symbol for the test	Degrees of Freedom	Obtained Value	\(p\)-Value
\(F\)	(2, 18)	= 33.92,	\(p\) < .05.

Reading Review 10.3

1. How is \(SS_b\) calculated?
2. What two things are reported inside the parentheses of the evidence string?
3. What information is needed to find the critical value for a one-way ANOVA?
4. Which two things only need to be included in the results summary paragraph when the omnibus results are significant?
5. How can an effects size be calculated for a one-way ANOVA?