12.5: Reading Articles – ANOVA
- Page ID
- 57596
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)↵
We revisit the steps for reading journal articles. First, always identify your research question. Guiding questions include: What’s the goal of the study? What’s the study trying to contribute to the field? What are the hypotheses?
Second, identify the logistics of the statistics. Guiding questions include: What is the scoring protocol, what do high and low scores mean? What groups are being compared?
Third, review the ANOVA results. Highlight relevant F-test results. Report the means and the F-test results. If there is a significant F-test, look for the accompanying post-hoc tests to determine which groups are higher or lower than the other. Do the findings confirm the hypotheses? Note any possible Type I or II errors, any effect sizes worth noting, and any sample size/power issues.
Fourth, the conclusion. Were the authors successful in accomplishing their goal? Guiding questions: Comment on what it means that the hypothesis was supported. It is also worthwhile to comment on which hypotheses were not supported. Did that help us understand the issue? Did the findings tell you something new about the issue under investigation?
We’ll use the article by Erermis, S., et al. (2004) as an example: First question: What is the purpose of the study? For this study, the researchers wanted to know if obesity was associated with mental health difficulties. Apparently, the connection between obesity and mental health difficulties is weak or inconsistent, and there is no evidence that obesity is related to psychological difficulties that are not at the level of a mental health diagnosis.
Second question: The three groups are clinical obesity, non-clinical obesity, and normal weight controls. Clinical obesity adolescents were sampled from an outpatient clinic for pediatric endocrinology. The non-clinically obese and normal-weight adolescents were sampled from a high school. Their variables were the scales on the Child Behavior Checklist (CBCL), and higher scores indicate more levels of the scale’s construct. Table 1 presents their findings.
| CBCL subscales | Clinical obese group ( n=30 ) | Nonclinical obese group ( n=30 ) | Normal weight controls ( n=30 ) | F | P | Comparison between groups |
|---|---|---|---|---|---|---|
| Activity | 42.5+-9.1 | 43.4+-7.7 | 46.9+-8.1 | 2.31 | 0.104 | NS |
| School competence | 43.2+-12.1 | 43.3+-4.5 | 45.0+-5.9 | 0.44 | 0.644 | NS |
| Total competence | 41.8+-13.9 | 41.5+-6.9 | 45.1+-6.9 | 1.2 | 0.304 | NS |
| Aggressiveness | 59.2+-8.4 | 55.1+-6.4 | 54.7+-6.1 | 3.69 | 0.028 | 1 > 2,3 |
| Delinquent behavior | 57.1+-7.5 | 52.8+-3.8 | 54.3+-6.6 | 3.67 | 0.029 | 1 > 2 |
| Anxiety-depression | 65.1+-10.8 | 60.0+-8.2 | 56.8+-7.2 | 6.66 | 0.002 | 1 > 2,3 |
| Attention | 61.4+-9.1 | 57.5+-6.9 | 56.8+-6.9 | 3.11 | 0.049 | 1 > 3 |
| Social problems | 64.9+-8.0 | 60.3+-6.4 | 54.7+-6.4 | 15.65 | 0.0001 | 1 > 2,3;2 > 3 |
| Somatic problems | 62.5+-10.7 | 61.4+-8.6 | 53.5+-4.5 | 10.32 | 0.0001 | 1 > 3; 2 > 3 |
| Thought problems | 59.1+-7.9 | 57.1+-7.7 | 57.0+-6.8 | 0.74 | 0.477 | NS |
| Social withdrawal | 62.8+-9.2 | 55.5+-6.3 | 54.1+-6.6 | 11.59 | 0.0001 | 1 > 2,3 |
| Externalizing behavior | 57.9+-9.4 | 52.6+-7.8 | 51.4+-9.8 | 4.34 | 0.016 | 1 > 2,3 |
| Internalizing behavior | 65.8+-9.6 | 59.8+-8.6 | 53.0+-9.6 | 14.25 | 0.0001 | 1 > 2,3; 2 > 3 |
Third question: What are the results? As with all statistical tables, start in the upper left corner, work your way left to right, and up and down. Start with the first variable, “Activity.” The clinical obese group has a mean of 42.5, the non-clinical obese group has a mean of 43.4, and the normal weight controls have a mean of 46.9. The F-test is 2.31, and the p-value is .10. This result means there is no significant difference between the three groups in their activity level.
So, is this finding expected? Actually, it isn’t. Let us walk through this result. If I did not know any better, I would think that the clinically obese group would have a lower activity level than the normal weight control group, possibly also lower than the non-clinically obese group. It turns out that all three groups are similar in activity level. Yes, the clinical and non-clinical obese groups are slightly lower in activity level than the normal weight control group, but the difference is not significant. This finding negates a persistent myth: obese adolescents are lazy and do not do much, which contributes to their obesity. This finding does not support this persistent myth, which is probably both good and bad news. It is good news because this finding dispels the idea that obese youth are lazy, and a lack of activity contributes to their obesity. It might be bad news because increasing the activity of obese youth may not result in weight loss.
Let us take a significant result, for the variable aggressiveness. The clinical obese group has a mean of 59.2, the non-clinical obese group has a mean of 55.1, and the normal weight controls have a mean of 54.7. The F-test is 3.69, and the p-value is .028. This result means there is a significant difference between the three groups in their aggressiveness level. The F-test is significant, and all it says is that there is something going on in the mean scores among the groups. It does not say exactly what is going on.
For that information, we turn to the post-hoc tests. The researchers, as do most researchers in journal publications, list the post-hoc tests as comparisons among groups. For example, the post hoc test could be noted as 1 > 2 > 3, which means group 1 has a higher mean than group 2, and group 2 has a higher mean than group 3. In this case, the post hoc test was noted as 1 > 2, 3, which means that group 1, the clinical obese group, has a higher mean and more aggressiveness than both groups 2 and 3, the non-clinical obese group and the normal weight control group, respectively.
The effect size was not reported, but as a rough estimate, the difference between the clinical obese group and the non-clinical obese group, and the normal weight control group is about four points (59.2 – 55.1 = 4, 59.2 – 54.7 = 4), and the pooled average standard deviation for the three groups is about seven (clinical obese group, SD = 8.4; non-clinical obese group, SD = 6.4; normal weight control group, SD = 6.1, average SD ~ 7), so the effect size is about 4/7 about .6, which is a moderate effect size. The moderate effect size means that the difference between the clinical obese group and the non-clinical obese group and the normal weight controls is noticeable.
What does this result mean? Always think conceptually. First, it seems unexpected. There is no immediate idea why clinically obese adolescents would be more aggressive than normal weight controls. To think conceptually, always look for an explanation, and for explanations, try to think of a perspective, a framework, or a theory. For this result, from a social skill or bullying framework, perhaps clinically obese adolescents are being bullied by others for their obesity. It is possible that clinically obese adolescents need to be aggressive to defend themselves. This scenario is possible, but if that were true, we would see the non-clinically obese adolescents have similar levels of aggressiveness with the clinically obese adolescents, and certainly more than normal weight controls. But that result did not happen here. Clinically obese adolescents have more aggressiveness than non-clinically obese adolescents. For this unexpected result, let us try a different framework. Maybe a medical framework would help. The clinically obese adolescents were recruited from a pediatric endocrine clinic. Endocrinology addresses hormones. Maybe the hormones in clinically obese adolescents contribute to their increased aggression. Admittedly, I do not know the connection between obesity and aggressiveness for adolescents, but it is one possibility given that the clinically obese adolescent population was recruited from a medical context, specifically a clinic.
A quick review of the remaining results suggests the clinically obese adolescent group does have higher psychopathology than the non-clinical and normal weight controls. This result is consistent and suggests a pattern, which is that clinically obese adolescents have more psychopathology. It does raise the question if the pathology is related to being obese, but if so, we would have thought that both the clinical and non-clinical obese adolescent groups would be higher than the normal weight controls. An explanation may elude us for now, but the results do indicate that we need to pay attention to the emotional, social, and mental health needs of clinically obese adolescents.