7.5: Choosing the Right Test
- Page ID
- 1764
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Learning Objectives
- This table is designed to help you decide which statistical test or descriptive statistic is appropriate for your experiment. In order to use it, you must be able to identify all the variables in the data set and tell what kind of variables they are.
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
|---|---|---|---|---|---|---|
| Exact test for goodness-of-fit | \(1\) | – | – | test fit of observed frequencies to expected frequencies | use for small sample sizes (less than \(1000)\) | count the number of red, pink and white flowers in a genetic cross, test fit to expected \(1:2:1 \) ratio, total sample \(<1000\) |
| Chi-square test of goodness-of-fit | \(1\) | – | – | test fit of observed frequencies to expected frequencies | use for large sample sizes (greater than \(1000\)) | count the number of red, pink and white flowers in a genetic cross, test fit to expected \(1:2:1\) ratio, total sample \(>1000\) |
| G–test of goodness-of-fit | \(1\) | – | – | test fit of observed frequencies to expected frequencies | used for large sample sizes (greater than \(1000\)) | count the number of red, pink and white flowers in a genetic cross, test fit to expected \(1:2:1\) ratio, total sample \(>1000\) |
| Repeated G–tests of goodness-of-fit | \(2\) | – | – | test fit of observed frequencies to expected frequencies in multiple experiments | - | count the number of red, pink and white flowers in a genetic cross, test fit to expected \(1:2:1\) ratio, do multiple crosses |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| Fisher's exact test | \(2\) | – | – | test hypothesis that proportions are the same in different groups | use for small sample sizes (less than \(1000\)) | count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, total sample \(<1000\) |
| Chi-square test of independence | \(2\) | – | – | test hypothesis that proportions are the same in different groups | use for large sample sizes (greater than \(1000\)) | count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, total sample \(>1000\) |
| G–test of independence | \(2\) | – | – | test hypothesis that proportions are the same in different groups | large sample sizes (greater than \(1000\)) | count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, total sample \(>1000\) |
| Cochran-Mantel-Haenszel test | \(3\) | – | – | test hypothesis that proportions are the same in repeated pairings of two groups | alternate hypothesis is a consistent direction of difference | count the number of live and dead patients after treatment with drug or placebo, test the hypothesis that the proportion of live and dead is the same in the two treatments, repeat this experiment at different hospitals |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| Arithmetic mean | – | \(1\) | – | description of central tendency of data | - | - |
| Median | – | \(1\) | – | description of central tendency of data | more useful than mean for very skewed data | median height of trees in forest, if most trees are short seedlings and the mean would be skewed by a few very tall trees |
| Range | – | \(1\) | – | description of dispersion of data | used more in everyday life than in scientific statistics | - |
| Variance | – | \(1\) | – | description of dispersion of data | forms the basis of many statistical tests; in squared units, so not very understandable | - |
| Standard deviation | – | \(1\) | – | description of dispersion of data | in same units as original data, so more understandable than variance | - |
| Standard error of the mean | – | \(1\) | – | description of accuracy of an estimate of a mean | - | - |
| Confidence interval | – | \(1\) | – | description of accuracy of an estimate of a mean | - | - |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| One-sample t–test | – | \(1\) | – | test the hypothesis that the mean value of the measurement variable equals a theoretical expectation | - | blindfold people, ask them to hold arm at \(45^{\circ}\) angle, see if mean angle is equal to \(45^{\circ}\) |
| Two-sample t–test | \(1\) | \(1\) | – | test the hypothesis that the mean values of the measurement variable are the same in two groups | just another name for one-way anova when there are only two groups | compare mean heavy metal content in mussels from Nova Scotia and New Jersey |
| One-way anova | \(1\) | \(1\) | – | test the hypothesis that the mean values of the measurement variable are the same in different groups | - | compare mean heavy metal content in mussels from Nova Scotia, Maine, Massachusetts, Connecticut, New York and New Jersey |
| Tukey-Kramer test | \(1\) | \(1\) | – | after a significant one-way anova, test for significant differences between all pairs of groups | - | compare mean heavy metal content in mussels from Nova Scotia vs. Maine, Nova Scotia vs. Massachusetts, Maine vs. Massachusetts, etc. |
| Bartlett's test | \(1\) | \(1\) | – | test the hypothesis that the standard deviation of a measurement variable is the same in different groups | usually used to see whether data fit one of the assumptions of an anova | compare standard deviation of heavy metal content in mussels from Nova Scotia, Maine, Massachusetts, Connecticut, New York and New Jersey |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| Nested anova | \(2+\) | \(1\) | – | test hypothesis that the mean values of the measurement variable are the same in different groups, when each group is divided into subgroups | subgroups must be arbitrary (model II) | compare mean heavy metal content in mussels from Nova Scotia, Maine, Massachusetts, Connecticut, New York and New Jersey; several mussels from each location, with several metal measurements from each mussel |
| Two-way anova | \(2\) | \(1\) | – | test the hypothesis that different groups, classified two ways, have the same means of the measurement variable | - | compare cholesterol levels in blood of male vegetarians, female vegetarians, male carnivores, and female carnivores |
| Paired t–test | \(2\) | \(1\) | – | test the hypothesis that the means of the continuous variable are the same in paired data | just another name for two-way anova when one nominal variable represents pairs of observations | compare the cholesterol level in blood of people before vs. after switching to a vegetarian diet |
| Wilcoxon signed-rank test | \(2\) | \(1\) | – | test the hypothesis that the means of the measurement variable are the same in paired data | used when the differences of pairs are severely non-normal | compare the cholesterol level in blood of people before vs. after switching to a vegetarian diet, when differences are non-normal |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| Linear regression | – | \(2\) | – | see whether variation in an independent variable causes some of the variation in a dependent variable; estimate the value of one unmeasured variable corresponding to a measured variable | - | measure chirping speed in crickets at different temperatures, test whether variation in temperature causes variation in chirping speed; or use the estimated relationship to estimate temperature from chirping speed when no thermometer is available |
| Correlation | – | \(2\) | – | see whether two variables covary | - | measure salt intake and fat intake in different people's diets, to see if people who eat a lot of fat also eat a lot of salt |
| Polynomial regression | – | \(2\) | – | test the hypothesis that an equation with \(X^2\), \(X^3\), etc. fits the \(Y\) variable significantly better than a linear regression | - | - |
| Analysis of covariance (ancova) | \(1\) | \(2\) | – | test the hypothesis that different groups have the same regression lines | first test the homogeneity of slopes; if they are not significantly different, test the homogeneity of the \(Y\)-intercepts | measure chirping speed vs. temperature in four species of crickets, see if there is significant variation among the species in the slope or \(Y\)-intercept of the relationships |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| Multiple regression | – | \(3+\) | – | fit an equation relating several \(X\) variables to a single \(Y\) variable | - | measure air temperature, humidity, body mass, leg length, see how they relate to chirping speed in crickets |
| Simple logistic regression | \(1\) | \(1\) | – | fit an equation relating an independent measurement variable to the probability of a value of a dependent nominal variable | - | give different doses of a drug (the measurement variable), record who lives or dies in the next year (the nominal variable) |
| Multiple logistic regression | \(1\) | \(2+\) | – | fit an equation relating more than one independent measurement variable to the probability of a value of a dependent nominal variable | - | record height, weight, blood pressure, age of multiple people, see who lives or dies in the next year |
| test | nominal variables | measurement variables | ranked variables | purpose | notes | example |
| Sign test | \(2\) | – | \(1\) | test randomness of direction of difference in paired data | - | compare the cholesterol level in blood of people before vs. after switching to a vegetarian diet, only record whether it is higher or lower after the switch |
| Kruskal–Wallis test | \(1\) | – | \(1\) | test the hypothesis that rankings are the same in different groups | often used as a non-parametric alternative to one-way anova | \(40\) ears of corn (\(8\) from each of \(5\) varieties) are ranked for tastiness, and the mean rank is compared among varieties |
| Spearman rank correlation | – | – | \(2\) | see whether the ranks of two variables covary | often used as a non-parametric alternative to regression or correlation | \(40\) ears of corn are ranked for tastiness and prettiness, see whether prettier corn is also tastier |