11.7: Reading Articles That Involve a T-test

We turn to how to read articles that use a t-test. The first few points are common for any statistical test.

11.7.1: First – Identify your Research Question

What’s the goal of the study? What’s the study trying to contribute to the field? The research question is not always straightforward. The research question is about what the researchers are curious about. Identifying and understanding the research question is about understanding the objective of the study. Keep in mind that the research question should be grounded in theoretical considerations. There should be an explanation for why the authors wanted to examine the two groups.

It is not a good idea to say the rationale for conducting the study is to develop better treatment plans, or because there is a gap in the literature. All clinical explorations of an issue will have implications for treatment planning. The rationale is weak because there is nothing unique about developing new treatment plans. A gap in the literature is something that has not yet been studied. However, there are always gaps in the literature because not everything has been studied. Studying something because there is a gap in the literature is the antithesis of the “Mt. Everest” argument. You don’t study something simply because it was not there.

You need the conceptualization part of the research question to understand why the results turned out as they did or did not.

11.7.2: Second - What are the Hypotheses?

You want a tally of the hypotheses to organize your understanding of the statistics. You should identify the two groups in question and the outcome variables. Usually, there are several within a given study. Keep your hypothesis in this framework: “Group A is higher or lower than Group B.” Keeping your hypothesis consistent helps with organization.

11.7.3: Third - Logistics

The logistics include the details of how you interpret the numbers. First, review the scoring protocol for each measure in the method section. What do high and low scores mean? Do high or low scores mean something good or something bad? Are there any thresholds or critical values you need to be aware of? These notes will help you interpret what the means indicate. Usually, the mean scores, when compared to the range, only indicate one of three outcomes: the scores mean that the group has a low, medium, or high amount of the variable in question.

What are the two groups being compared? Any notes about how they were formed? What is the overall sample size, and what is the sample size for each group? Any oddities about the sample sizes or demographic characteristics of each group? These issues could be considered confounds when evaluating the validity of the results.

11.7.4: Fourth - Review the T-test Results

Report the means and the t-test results as significant or not significant. Look for patterns in the effects. Do the findings confirm the hypotheses?

When reviewing the results, follow these steps :

1. Read the t-test and the p value. Does the p value indicate significance? The only answers are yes and no.
2. If no then your statement is: here is no mean difference between the two groups. And nothing more needs to be stated. Remember, do not state, almost significant or approaching significance. There are no such statements.
3. If yes, then your statement is: there is a mean difference between the two groups. And nothing more needs to be stated. Remember, do not state, somewhat significant or highly significant.There are no such statements.
4. If yes, then state the pattern. For t-tests, there are only two patterns. Group A is higher than group B, or group A is lower than group B. Keep your groups consistent according to each hypothesis.
5. If yes,then determine if the pattern you observed is expected or unexpected. By expected, we mean the pattern confirms the hypothesis. By unexpected, we mean the pattern was a surprise; the hypothesis did not predict that pattern to happen. Was it expected that Group A would be higher or lower than Group B?

Then, examine the effect size. Most of the time, for t-tests, they are not reported. However, a crude way of estimating an effect size is to take the standard deviations of both groups, average them, and then divide the actual difference in the means by the combined, or pooled, average standard deviation.

There are only three answers to the question, What is the effect size? The answers are low, medium, and high. We usually do not see high-effect sizes. We usually see low or medium effect sizes. Are the effect sizes expected or unexpected? Do we want to see that much difference between the two groups?

If there are multiple t-tests and multiple tests, pool the results together and evaluate if the results are consistent and aligned with the research question. Do the results say something about the phenomenon under investigation?

Please note the following when reviewing statistics in articles.

There is no need for criticism of the method or the overall study. Stay with the numbers. Criticism of the study only distracts you from interpreting the numbers. Every study has its critics. Yes, some studies are worse than others. Until you decide the validity of the study is in question, put aside the criticism and stick with interpreting the numbers and the statistical results.

There is no need to act, think, or state things in a sophisticated manner. Keep it simple. Sounding academic or scholarly makes people confused. What this means is “if you hear hooves, it’s a horse, not a zebra.” This mantra is from medicine. This mantra is meant to communicate to doctors that you are likely not looking at an unusual “one-in-a-million,” case. The case is simple. You have a common cold, nothing more, no rare infectious disease. The same is true for statistics. In interpreting the statistics, it’s a horse. You will likely not encounter any strange anomaly, advanced interpretation, or anything hidden in the numbers.

Read everything straightforwardly. There are no magical, hidden, or steps meant to trip you up in reading statistics. “No rabbit hole spiral thinking” – This is my mantra. Do not engage in “what ifs” such as looking at demographics and trying to say, “Well, what if there were more of this, or more of that?” Or trying to add extra variables or contexts to explain the results. This kind of thinking can distract and lead you down the rabbit hole of Alice in Wonderland and take you away from the variables in question. Stay in the lane of the variables included in the statistical test.

So, now that I’ve told you what NOT to do when interpreting statistical tests, what should you do?

11.7.5: Conclusion

Always think CONCEPTUALLY, which is your explanation of why we are looking at these variables in the first place. Conceptually means how these variables are connected. What you want to do is think about why these variables would be connected, or what these variables represent.