13.2.1: Example with Main Effects and Interactions

You might realize by now the Dr. MO’s student researchers did a couple studies on mindset. Remember growth mindset? The idea that mistakes help you learn, and that more time and effort leads to brain development? One group of student researchers collected data on improving growth mindset that should be analyzed with a factorial ANOVA. Here it is:

Scenario

The student researchers looked at the Difference scores of 106 students at Dr. MO’s community college. The Difference scores were calculated by subtracting each students’ Post-Test Mindset Quiz score (measured at the end of the semester) from their own Pre-Test Mindset Quiz score (measured at the beginning of the same semester); thus, positive scores mean that the student improved their mindset. In this study, we had two variables. One was the intervention, which had two levels: No Intervention (comparison control group) and Intervention (in which faculty tried different activities to improve mindset). We also collected information on what department these activities were happening in, and had enough data to analyze students in English classes and students in Psychology classes.

Can you identify the groups and variables in this scenario?

Example \(\PageIndex{1}\)

Answer the following questions to understand the variables and groups that we are working with.

1. Who is the sample?
2. Who do might be the population?
3. What are the IVs and levels for each IV?
4. What is the DV (quantitative variable being measured)?
5. Is this a 2x2 factorial design? If not, what kind of design is it?
6. List out each of the combinations of the levels of the IVs:

Solution

1. The sample was 106 community college students taking an English class or a Psychology class.
2. The population could be all community colleges students, or maybe all community college students taking English or Psychology. Or maybe all community college students in a general education course?
3. One IV is Intervention, with the levels being Yes or No. The other IV is Department, with the levels being English or Psychology.
4. The DV is the Difference score from the Mindset Quiz pre-test to the Mindset Quiz post-test.
5. Yes, this is a 2x2 factorial design because there are two IVs (so there are two numbers __x__); the first IV (Intervention) has two levels and the second IV (Department) has two levels.
6. List out each of the combinations of the levels of the IVs:
   1. Intervention in English
   2. Intervention in Psychology
   3. No Intervention in English
   4. No Intervention in Psychology

Can you plug these IVs into a Punnett’s Square grid?

Example \(\PageIndex{2}\)

Complete a grid showing the factorial design IVs and DVS.

Solution

Table \(\PageIndex{1}\)- 2x2 Factorial Grid of Intervention by Department

IV Levels	IV1- Yes Intervention	IV1- No Intervention
IV2: English Department	2x2: Students who experienced an intervention in an English class.	2x2: Students who did not experience an intervention but who were in an English class.
IV2: Psychology Department	2x2: Students who experienced an intervention in a Psychology class.	2x2: Students who did not experience an intervention but who were in a Psychology class.

You could have put Department as IV1 (columns) and Intervention as IV2 (rows). It’s really about what makes the most sense to you; Dr. MO wanted what she thinks is the main influence of mindset on the top.

Participants in each “cell” of this design have a unique combination of IV conditions.

Three Effects

With a 2x2 factorial design, you have three effects to look at. Remember, “effects” are the results of the DV, what was measured. Here are the three effects that you need to look at:

1. The main effect of the one IV: How does one IV affect the DV (independent of the other IV)
2. The main effect of the other IV: How does the other IV affect the DV (independent of the first IV)
3. The interaction of the two IVs -- how they jointly affect the DV

Example \(\PageIndex{3}\)

In our mindset scenario, what are these three effects?

Solution

1. The main effect of the intervention; did the intervention improve Mindset Quiz scores?
2. The main effect of the department; did which class you were in affect Mindset Quiz scores?
3. The interaction of the intervention by department; is one department more likely to improve Mindset Quiz scores than another department when there’s an intervention? Another way to think about these variables is to ask whether the departments started out with different average Mindset Quiz scores, so that even in the No Intervention condition the Difference score would be statistically significantly different between the two departments.

Let’s look at these main effects in Table \(\PageIndex{2}\), in which the marginal means were included. Marginal means are, you guessed, it the means on the margins of the table. These means on the margin show the means for each level of each IV, which are the main effects. The marginal means do not show the combination of the IVs’ levels, so they do not show an interaction.

Table \(\PageIndex{2}\)- 2x2 Factorial Grid of Intervention by Department with Marginal Means

IV Levels	IV1- Yes Intervention	IV1- No Intervention	Marginal Means of IV2
IV2: English Department	2x2: Students who experienced an intervention in an English class.	2x2: Students who did not experience an intervention but who were in an English class.	DV = 2.49
IV2: Psychology Department	2x2: Students who experienced an intervention in a Psychology class.	2x2: Students who did not experience an intervention but who were in a Psychology class.	DV = 1.34
Marginal Means of IV1	DV = 2.08	DV = 1.76

Again, you could have put Department as IV1 (columns) and Intervention as IV2 (rows). And again, participants in each “cell” of this design have a unique combination of IV conditions.

Main Effects

Let’s go through the marginal means for Table \(\PageIndex{2}\).

Example \(\PageIndex{1}\)

What are the marginal means for the Intervention?

Solution

Intervention Yes \(=\) 2.08

Intervention No \(=\)1.76

We’ll look at statistical significant later, but just based on these means, it looks like the students who experienced an Intervention had a larger Difference score than students who did not have an intervention. This is what was expected; students who experienced an intervention had a higher Mindset Quiz score at the end of the semester (post-test), after the interventions, than at the beginning of the semester (pretest). This is your main effect of Intervention. When we do the statistical analyses, we’ll follow the same process for null hypothesis significance testing:

Critical \(< |Calculated| = \) Reject null \(=\) means are different \(=\) main effects \(= p<.05\)

Critical \(> |Calculated| =\) Retain null \(=\) means are similar \(=\) no main effects \(= p>.05\)

Let’s move on to the other independent variable.

Exercise \(\PageIndex{1}\)

Based on Table \(\PageIndex{2}\), what are the marginal means for department? Which department seemed to have a higher Difference score?

Answer

Marginal Means:

• English=2.49
• Psychology=1.34

There seems to be a main effect of department such that students in English had a higher Difference score than students in Psychology classes.

We can’t be say much more than that without looking at actual statistical results. Instead, we will look at the individual cells of our grid to see if there was an interaction between Department and Intervention on the Difference scores.

Interaction

Table \(\PageIndex{3}\) has the means for each combination of each IV level in the individual cells. Looking at Table \(\PageIndex{3}\), how do the inside cells seem relate to each other?

Table \(\PageIndex{3}\)- 2x2 Factorial Grid of Intervention by Department with Marginal Means & Cell Means

IV Levels	IV1- Yes Intervention	IV1- No Intervention	Marginal Means of IV2
IV2: English Department	DV=3.70	DV=1.29	DV=2.49
IV2: Psychology Department	DV=0.46	DV=2.23	DV=1.34
Marginal Means of IV1	DV=2.08	DV=1.76

Again, you could have put Department as IV1 (columns) and Intervention as IV2 (rows). And again, participants in each “cell” of this design have a unique combination of IV conditions.

Example \(\PageIndex{5}\)

Answer the following questions related the cell means in Table \(\PageIndex{3}\),

1. Was one cell substantially higher than the others?
2. Was one cell substantially lower than the others?

Solution

1. It looks like students in English class who experienced an intervention had higher Difference scores.
2. It looks like students in a Psychology class who experienced an intervention had a really low difference score.

This pattern of cell means is your interaction of Intervention and Department: Experiencing the intervention interacts with the department to affect Differences in Mindset Quiz scores such that the intervention seems to improve mindset for those in English but not in Psychology (the Psychology students who experienced an intervention were essentially unchanged). What’s strange is that Psychology students in the control condition (No Intervention) might have actually improved their Mindset Quiz scores more than both English students with no intervention and Psychology student with the intervention! You might be wondering why these strange effects for the Psychology students, and Dr. MO has no answer for you. This is real data, and there’s no good explanation for this with the variables that we have. ::shrug::

As you can see, the main effects of each IV can relate to the interaction in several different ways. Let’s look at that next.