14.5: Effect Size

Last updated
Save as PDF

Page ID: 4031

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

There’s a few different ways you could measure the effect size in an ANOVA, but the most commonly used measures are η2 (eta squared) and partial η². For a one way analysis of variance they’re identical to each other, so for the moment I’ll just explain η². The definition of η² is actually really simple:

\(\eta^{2}=\dfrac{\mathrm{SS}_{b}}{\mathrm{SS}_{t o t}}\)

That’s all it is. So when I look at the ANOVA table above, I see that SS_b=3.45 and SS_tot=3.45+1.39=4.84. Thus we get an η2 value of

\(\ \eta^2 ={ 3.45 \over 4.84}=0.71\)

The interpretation of η2 is equally straightforward: it refers to the proportion of the variability in the outcome variable (mood.gain) that can be explained in terms of the predictor (drug). A value of η2=0 means that there is no relationship at all between the two, whereas a value of η²=1 means that the relationship is perfect. Better yet, the η² value is very closely related to a squared correlation (i.e., r²). So, if you’re trying to figure out whether a particular value of η² is big or small, it’s sometimes useful to remember that

\(\ \eta = {\sqrt{SS_b \over SS_{tot}}}\)

can be interpreted as if it referred to the magnitude of a Pearson correlation. So in our drugs example, the η² value of .71 corresponds to an η value of \(\ \sqrt{.71}\) =.84. If we think about this as being equivalent to a correlation of about .84, we’d conclude that the relationship between drug and mood.gain is strong.
The core packages in R don’t include any functions for calculating η². However, it’s pretty straightforward to calculate it directly from the numbers in the ANOVA table. In fact, since I’ve already got the SSw and SSb variables lying around from my earlier calculations, I can do this:

SStot <- SSb + SSw          # total sums of squares
eta.squared <- SSb / SStot  # eta-squared value
print( eta.squared )

## [1] 0.7127623

However, since it can be tedious to do this the long way (especially when we start running more complicated ANOVAs, such as those in Chapter 16 I’ve included an etaSquared() function in the lsr package which will do it for you. For now, the only argument you need to care about is x, which should be the aov object corresponding to your ANOVA. When we do this, what we get as output is this:

etaSquared( x = my.anova )

##         eta.sq eta.sq.part
## drug 0.7127623   0.7127623

The output here shows two different numbers. The first one corresponds to the η² statistic, precisely as described above. The second one refers to “partial η²”, which is a somewhat different measure of effect size that I’ll describe later. For the simple ANOVA that we’ve just run, they’re the same number. But this won’t always be true once we start running more complicated ANOVAs.²⁰⁷