17.2: Choosing the Test
- Page ID
- 17442
You might remember that we already talked about which statistical test to run at the end of the chapters on ANOVA. In that section, Dr. MO said that there are three things about your variables that determined which statistical analysis is most appropriate; these were:
- The type of DV (qualitative or quantitative or ranked/ordinal)
- The number of groups
- Whether the groups are independent or dependent or the population
In an Example, you described what types of variables (based on these three descriptions) fit all of the analyses that we had covered at that point:
- One-Sample t-test
- DV: Compares means, so quantitative
- Number of Groups: Compares a sample to the population
- Type of Group: The population
- Two-sample independent t-test
- DV: Compares means, so quantitative
- Number of Groups: Compares two samples
- Type of Group: Independent (unrelated)
- Two-sample dependent t-test
- DV: Compares means, so quantitative
- Number of Groups: Compares two samples
- Type of Group: Dependent (related)
- Between Groups ANOVA
- DV: Compares means, so quantitative
- Number of Groups: Compares two or more levels of an IV
- Type of Group: Independent (unrelated)
- Repeated Measures ANOVA (also called Within Groups ANOVA)
- DV: Compares means, so quantitative
- Number of Groups: Compares two or more levels of an IV
- Type of Group: Dependent (related)
- Mann-Whitney U
- DV: Ranked data, or we assume that the distribution is NOT normally distributed.
- Number of Groups: Compares two groups
- Type of Group: Independent (unrelated)
- Wilcoxon Match-Pair Signed-Rank test
- DV: Ranked data, or we assume that the distribution is NOT normally distributed.
- Number of Groups: Compares two groups
- Type of Group: Dependent (related)
- Kruskal-Wallis One-Way ANOVA
- DV: Ranked data, or we assume that the distribution is NOT normally distributed.
- Number of Groups: Compares two or more groups
- Type of Group: Independent (unrelated)
- Friedman’s test
- DV: Ranked data, or we assume that the distribution is NOT normally distributed.
- Number of Groups: Compares two or more groups
- Type of Group: Dependent (related)
But since then, we’ve also learned about correlations, regression, and Chi-Squares. What we learned about regression is mostly focused on extending what we learned about correlations, so we won’t look at them again right now. However, regression is one of the most powerful predictive tools that statisticians have, so you will learn a LOT more about this type of analysis if you take more advanced statistic courses. For now, let’s look at describe the three characteristics for each of the correlations and Chi-Square analyses that we have learned about.
Example \(\PageIndex{1}\)
Identify the three criteria (type of DV, number of groups, and type of group) for the following analyses:
- Pearson’s Correlation
- Spearman’s Rank Correlation
- Phi Correlation
- Chi-Square Goodness of Fit
- Chi-Square Test of Independence
- McNemar Test
Solution
- Pearson’s Correlation
- DV: Two quantitative variables (not two levels of one IV, they are two entirely different variables).
- Number of Groups: Compares two groups
- Type of Group: We are testing to see if they are linearly related.
- Spearman’s Rank Correlation
- DV: Two ranked (ordinal) variables.
- Number of Groups: Compares two groups
- Type of Group: We are testing to see if they are related.
- Phi Correlation
- DV: Two binary variables.
- Number of Groups: Compares two groups
- Type of Group: We are testing to see if they are related.
- Chi-Square Goodness of Fit
- DV: One qualitative variable with two or more levels
- Number of Groups: Compares two or more groups or categories
- Type of Group: Independent
- Chi-Square Test of Independence
- DV: Two qualitative variables (not two levels of one IV, they are two entirely different variables).
- Number of Groups: Compares two groups, each with two or more levels or categories.
- Type of Group: Independent
- McNemar Test
- DV: Two qualitative variables (not two levels of one IV, they are two entirely different variables).
- Number of Groups: Compares two groups, each with two or more levels or categories.
- Type of Group: Dependent
Another way to think about choosing the most appropriate test is through a flow chart or decision tree. A flow chart or decision tree could cover the three basic questions about the variables, and branch out depending on the answer.
Note
You might consider drawing a flow chart yourself with only the statistical tests that your professor has focused on.
For now, you can check out this PDF of a Decision Tree Handout created by Dr. MO to see what this flow chart might look like with most of the analyses that we’ve covered so far. Here is the link to the interactive website to help you practice when to use which kind of statistical analysis that was provided in that previous ANOVA chapter. To start, click on the kinds of analyses that you want to be tested on, then hit Submit.
You might think that you don’t need to learn about these things because statistical software will know which analysis to run. This is both true and not true. Sometimes, the software won’t let you run the analysis with the wrong type of variables. For example, if you try to have SPSS run a Chi-Square with quantitative variables, it will say that it can’t. But, if you try to run a Chi-Square with qualitative variables that are related, it will. But we know that one of the basic requirements of Chi-Square is that the groups must be independent (meaning that the same people can’t be in both groups, the groups can’t be related). So, you can’t trust the software to know what’s it’s doing. You have to be the one who knows what you’re doing!
What the software definitely won’t do is tell you what it means. Let’s turn to that idea next.
Next, let's finish the textbook by revisiting why you had to take this class. This will also be a great refresher on all that you've learned!