12.2: Factorial Designs

In factorial analysis, just like the fractals we see in nature, we can add multiple branchings to every experimental group, thus exploring combinations of factors and their contribution to the meaningful patterns we see in the data. In this chapter, we will tackle two-way Analysis of Variance (ANOVA) and explore conceptually how factorial analysis works.

What is Factorial Analysis?

To understand when you need two-way ANOVA and how to set up the analyses, you first need to understand the corresponding research design terminology. We will also need to define and interpret main effects and interaction effects, both of which can be analyzed in a factorial research design. Later, we will approach the detection and interpretation of interaction effects, which will really help you see the extraordinary complexity of information factorial analyses can offer.

Factorial analyses, such as a two-way ANOVA, are required when we analyze data from a more complex experimental design than we have seen up until now, like an experiment (or quasi-experiment) that includes two or more independent variables (or factors).

Figure \(\PageIndex{1}\): Example two-way ANOVA design with factors dose and administration time.

Examples of designs requiring two-way ANOVA (in which there are two factors) might include the following: a drug trial with three doses as well as the time of administration (Figure 1), or a memory test using four different colors of stimuli and also three different lengths of word lists (Figure \(\PageIndex{2}\)).

Figure \(\PageIndex{2}\): Example two-way ANOVA design with factors stimuli colors and lengths of word lists.

As we saw in the chapter on one-way ANOVA, the total variability among scores in a dataset can be separated out, or partitioned, into two buckets. As demonstrated in Figure \(\PageIndex{3}\), the first bucket, often called between-groups variability or treatment effect, refers to the systematic differences between treatment groups or some other grouping. These are the differences among scores we are investigating in research questions about group differences — the explained differences. The other bucket, often called within-groups variability or error variability (different from Type I and Type II error), refers to the random, unsystematic differences that cannot be explained by the independent variables. These are the unexplained individual differences that represent the noise in the data, obscuring the signal or pattern we are investigating.

Figure \(\PageIndex{3}\): Sources of variability: total, between, and within.

You will recall from the ANOVA chapter that we called independent variables "factors" and the groups within each of those factors "levels." Let's consider a visual example (in Figure \(\PageIndex{4}\) in which the goal is to separate color swatches according to some factor, such that the colors within each grouping are more uniform. If we first sort the colors according to the hue (we can think of this as similar to a factor), let’s say into green or blue hues, then we have explained some of the overall variability. But if we add brightness (a second factor), then we can explain even more of the differences among the color swatches, making each grouping a little more uniform.

Figure \(\PageIndex{4}\): Sorting shirts by hue and brightness.

Clearly, there is still some work to be done; for example, if in the hue factor we included a third level of “red,” the uniformity would have been much improved. And with factorial analysis, there is technically no limit to the number of factors or the number of levels we can employ to explain away the variability in the data. In general, researchers are aiming to explain as much variance in the dependent variable as possible through multiple factors and/or multiple levels of those factors. This is what we will be able to do with two-way ANOVA and factorial designs. In practice, however, a large number of factors or levels of factors can become unwieldy to interpret.

To grasp factorial research designs, it becomes even more important to develop comfort with the concepts of factors and levels so that you can identify and describe the design and thus the requisite analytical setup. Let us suppose that we have a research study that measures the effect of a placebo, a low dose and a high dose of the drug, and also takes into account the timing of administration. The first factor could be succinctly identified as “drug dose,” and the second factor as “timing”. In another example, perhaps we show participants words in black, red, blue or yellow, and we also take into account whether the word list presented is long, medium, or short. What would you call each of those two factors?

What if, in a drug study, you notice that men seem to react differently than women? If you have that information (male/female), you can use it in your ANOVA and see if you can put more variance in your “good” (or between) bucket.

Figure \(\PageIndex{5}\): 3 × 2 ANOVA with factors drug dose and sex. (Copyright Nicole Vittoz, licensed under CC BY-NC 4.0.)

In the design illustrated in Figure \(\PageIndex{5}\), we see that it is a 3 × 2 ("three by two") ANOVA. There are three levels in the first factor (drug dose), and there are two levels in the second factor (sex). This notation that identifies the number of levels in each factor with a multiplier between them helps us see clearly how many samples are needed to realize the research design. In this example, we would need six samples in total, each of which would need to have a good enough sample size to allow for the central limit theorem to justify the normality assumption (N ≥ 30). That is a lot of participants! However, we could learn much more by including both factors, if indeed the sex of the participant is associated with a different response to the drug.

Figure \(\PageIndex{6}\): 4 × 3 ANOVA with factors word color and length of list. (Copyright Nicole Vittoz, licensed under CC BY-NC 4.0.)

We can see an example of a 4 × 3 two-way ANOVA in Figure \(\PageIndex{6}\), with our example of word color and length of list. Altogether, this design would require 12 samples.

And just for the sake of showing you the potential of factorial analyses, you could also impose a third factor on the design: the age of the participants. In this case, you have a 4 × 3 × 2 design, requiring 12 samples. At 30 participants each, that would be 30 × 12 = 360 people! You can appreciate how each factor exponentially increases the practical demands (costs) of the research study. For this reason, a cost-benefit analysis must be carefully applied in factorial research design, such that the minimum complexity is used to answer the key research questions sufficiently.

In a two-way ANOVA, just as in a one-way ANOVA, we calculate various types of Sums of Squares (SS). The SS total is broken down into SS_between and SS_within. However, with a two-way ANOVA, the SS_between must be further broken down, because there are now two different factors that can have a main effect (i.e., can explain some of the total variance). Also, with more than one factor, there can be an interaction between the two that itself uniquely accounts for some of the variance. So now, we have SS_row (the first factor), SS_column (the second factor), and SS_interaction. For each SS, you can also see the matching degrees of freedom.

Video: Introduction to Two Way ANOVA (Factorial Analysis)

Introduction to Two Way ANOVA (Factorial Analysis) on YouTube.

ANOVA Summary Table for Two-Way Designs

Table \(\PageIndex{1}\) shows the full ANOVA summary table expanded to accommodate the three subtypes of between-groups variability. (X is the individual score. N is the number of scores. M_O is the overall mean.)

Table \(\PageIndex{1}\): ANOVA summary table for a two-way design.

Source	SS	df	MS	F
Between (Row)	\(\sum{(N_\text{row}(M_\text{row}-M_O)^2)}\)	\(\text{rows}-1\)	\(\Large\frac{SS_r}{df_r}\)	\(\Large\frac{s_r^2}{s_w^2}\)
Between (Column)	\(\sum{(N_\text{col}(M_\text{col}-M_O)^2)}\)	\(\text{columns}-1\)	\(\Large\frac{SS_c}{df_c}\)	\(\Large\frac{s_c^2}{s_w^2}\)
Between (Interaction)	\(\sum{(N_\text{cell}(M_\text{cell}-M_O)^2)}\\-SS_\text{row}-SS_\text{col}\)	\((\text{rows}-1)(\text{columns}-1)\)	\(\Large\frac{SS_i}{df_i}\)	\(\Large\frac{s_i^2}{s_w^2}\)
Within	\(\sum{(X-M_\text{cell})^2)}\)	\(N-\text{cells}\)	\(\Large\frac{SS_w}{df_w}\)
Total	\(\sum{(X-M_O)^2)}\)	\(N-1\)

Note that all of the Sums of Squares and degrees of freedom still should add up to the total. As you can see, there will now be three F-test results from this one omnibus analysis, one for each of the between-groups terms. Each can be compared to the appropriate critical value (section 16.3) to determine the statistical significance of the degree to which that factor (or interaction) accounts for variance in the dependent variable that was measured in the study.

The row and column means, the averages of cell means going across or down this matrix, are often referred to as marginal means (because they are noted at the margins of the data matrix). To help you interpret the formulas as they reference row means, column means, and cell means, Table \(\PageIndex{2}\) shows how to locate these numbers in a 2 × 2 two-way ANOVA scenario.

Table \(\PageIndex{2}\): Example cell, row, and column means in a 2 × 2 table.

	Column A	Column B	Row Means
Row A	Cell	Cell	\(M_\text{RowA}\)
Row B	Cell	Cell	\(M_\text{RowB}\)
Column Means	\(M_\text{ColA}\)	\(M_\text{ColB}\)

When it comes to hypothesis testing, a two-way ANOVA can best be thought of as three hypothesis tests in one. For each factor, and also for the interaction of the two, you need to identify populations and hypotheses, cutoffs, calculate the SS between, degrees of freedom, variance between, and F-test results. All three will share the same error terms, the SS, degrees of freedom, and variance within groups. As you can imagine, the complexity of calculating such an analysis could be daunting, but a systematic, organized approach using the ANOVA table keeps it well under control.

As with one-way ANOVA, if any factor has more than two levels, you may need to calculate pairwise (post hoc) contrasts for that factor to determine where exactly a significant difference among group means lies. Even with a 2×2 ANOVA, the interaction effect has four possible pairwise comparisons to investigate, and that would require a planned contrast or post-hoc test. The same rules apply to such analyses as before: they may only be conducted if there is a significant overall ANOVA result, and the familywise risk of Type I error must be controlled.

Considerations for Factorial Designs

Many researchers new to the trade are keen to include as many factors as possible in their research design, and to include lots of levels just in case it is informative. This is an understandable impulse, given how much effort and expense can go into designing and conducting a research study. We want to gather as much information as possible from that effort! However, as we saw before, the more factors we add in, the more participants we need to ensure a decent sample size in each cell of our data matrix.

There is another important element to consider, as well. For each factor we add in, we add interaction terms. If we were ambitious enough to include three factors in our research design, we would have the potential for interaction effects among each pair of the factors, but we would also potentially see a three-way interaction effect.

In a three-way ANOVA involving factors A, B, and C, one must analyze the following interactions:

• A x B
• B x C
• A x C
• A x B x C

The interpretation of all these interactions becomes very challenging. For this reason, solid advice to researchers is to limit ourselves to two factors for any given analysis, unless there is a very strong hypothesis regarding a three-way interaction.

Test Your Knowledge

Question \(\PageIndex{1}\)