11.5: Reading T-tests

Reading statistical analyses in journal articles follows the same format for all statistical tests. Review your research question and hypotheses, identify your independent and dependent variables, and read the statistical results by following the same structure as stated in the next section.

11.5.1: Determining Significance

When you read a t-test in a journal article or read a t-test in your SPSS statistical printout, you will get something that looks like this: t = 3.56, p = .03. Read this output as, “the t-test value is 3.56, and it is significant at the p < .05, which means there is a mean difference between the two groups”.

Here are the steps:

A t-test is significant if the t-test value is greater than the threshold value, and the p value is less than your alpha level. Usually, the t-test value threshold is 1.96, N >=30 per group, and p is set at .05.

Here are examples:

T-test = 1.03, p = .38. The test is not significant. The t-test is below 1.96, p greater than .05, the test is not significant. When it is not significant, the mean difference between the two groups is not large enough to be significant. The two groups are similar; nothing is going on here.

T-test = 4.39, p < .01. The test is significant. The t-test is above 1.96, p less than .05, the test is significant. When it is significant, you can claim that the mean difference between the two groups is large enough to be significant. Something is going on. There is something about the two groups that is associated with the difference in the mean scores.

Note that the sign of the t-test is not important. You will read t-tests that say 4.39 and -.4.39. The sign is just directional. If the t-test is positive, it means that the first group is higher than the second group. If the t-test is negative, it means that the first group is lower than the second group. There is no inherent value regarding the sign of the t-test. It is just a descriptive tag that signals the first group is higher or lower than the other group.

11.5.2: Statistics as Ratios – T-tests

Keep in mind that statistical tests are ratios. The ratios are the true variance/error variance. The value you obtain in a statistical test is the outcome of a ratio that compares the true variance to the error variance. If you are looking for the mean difference between two groups, what you want to see is the truth, or that the groups are different from each other, and that truth is bigger than the error, or that the groups are not truly different from each other.

Back to basic math. You get a large number when you divide a large number by a small one. You get a small number when you divide a small number by a large one. When you divide a number by the same number, you get one.

The value of the t-test, t = 4.39, is the outcome of true variance/error variance. If the value of the t-test is positive, that means the true variance is greater than the error variance. But we do not want the true variance to be just barely greater than the error variance. If the t-test is 1.96, almost 2.00, then the true variance is twice as large as the error variance. That is good because it clearly states that the true variance is larger than the error variance. That is why statistical tests are basically ratios. Recall that the hallmark of statistics is comparison. We are always comparing something to something. In this case, we are comparing two groups; we want to see if the difference, the truth, between the two groups, is larger than the error, or lack of difference, between the two groups.

When interpreting t-tests, as with most statistical tests, as the value of the statistical test goes up, the p value goes down. That is why sometimes you read only the p value in reports, because the p value is all you need. You do not get any more information if you have the t-value accompanying the p value. Actually, it is better to just read the p value and ignore the t-value. The t-test value is affected by several factors, including the overall sample size, the sample size in each group, the difference in the sample sizes between the two groups, and the distribution of the scores for each group. Determining the true variance and the error variance is not a straightforward way to calculate it. While we fuss over the t-test value, you are better off just interpreting the p value. Recall that the p value only indicates significance, and in determining significance, we are interested in just one question – is it significant or not? The only answers to that question are yes or no. There is no such thing as more or less significant. The t-test value itself does not really add any information about the significance of the test beyond the p value.

Of note, given what we previously learned about not placing too much stock in the p value, you might wonder why you read about t-tests with different levels of significance, such as p < .05, p < .01, or p < .001. The p < .05 and p < .01 could be expected if researchers adjusted for Type I or II errors. However, the researchers should report that they are using two different p values for that purpose. There is absolutely no value in having a t-test be significant at the p < .001 level and stating that the t-test is highly significant. There is no such thing. Do not pay attention to these variations when you see t-tests reported as having p values of .05 and .01. They all mean the same thing: something is significant or something is not significant. Nothing more.

11.5.3: Patterns in T-test Results

It is rare for a t-test to be conducted as a single test without it being used as part of a series of t-tests. You might examine an age difference between males and females, but that single t-test is part of an overall battery of statistical tests that examine differences between males and females for several variables. Usually, we conduct a series of t-tests, and when we do so, we look for consistency in the t-test results. Is one group consistently higher than the other across t-tests?

This situation commonly occurs when comparing two groups across a series of variables because you are interested in whether the two groups are similar or different. Suppose you are examining military veterans with PTSD and comparing them with first responders with PTSD. You conduct a series of t-tests with one group of military veterans with PTSD and the other group of first responders with PTSD. Military veterans are similar to first responders in outcome variables such as depression, anxiety, and a decrease in social support. But the military veterans have higher scores, indicating more severity, on scales for PTSD, anger management, and alcoholism, compared to first responders. In this case, this pattern of t-test results might make sense. While both groups have PTSD and will demonstrate depression, anxiety, and decrease in social support, perhaps the military experiences associated with PTSD might yield differences that are relevant to the military, such as anger. Because of the culture of alcohol use among the military, perhaps they will engage in more alcohol use compared to first responders. Here, the series of t-tests yields important information because of the pattern among the collective t-test results, rather than just one t-test result at a time.