8.2: Experimental Design and Cause-Effect

The independent samples t-test is sometimes used to analyze data from an experiment. Not all inferential tests fit well with experimental designs. This is because different designs yield different kinds of data and, thus, vary in which inferential tests best suit them. A misconception some folks have is that the words “research,” “study,” and “experiment” are synonymous and all refer to any work where data are collected and used to test hypotheses. However, experiments are specific types of studies. Study and research are broad terms whereas experiment is specific to only some forms of research. Thus, all experiments are studies or research but not all research or studies are experiments. Instead, experiments fall under a specific subcategory of research known broadly as “experimental designs.” Experiments are unique and especially valuable forms or research so it is worth taking a moment to understand them.

The experimental design of research involves controlling, manipulating, or constraining one variable to see if it has an impact on another variable. Because of this (and some additional features), experimental designs are the only kinds of studies in which cause-effect relationships can be deduced. The additional considerations when using an experimental design are beyond the scope of this book so we will stay focused on the key feature: Controlling the independent variable. Recall (from Chapter 1) that in a cause-effect relationship, the causal variable is called the independent variable (IV) and the affected variable is called the dependent variable (DV). In order to see if the IV impacts the DV in an experiment, researchers control how participants in each group experience the IV and then measure the DV. Control in this context means the researcher gets to decide what happens for the IV for each case. A common form of experiment is the experimental vs. control group design. In this design there are two groups (as indicated in the name of the design): an experimental group and a control group. Participants are often randomly assigned to one of these two groups. What distinguishes these two groups is whether (or to what extent) they experience the IV. The experimental group receives/is exposed to the IV (or a higher amount of the IV) whereas the control group does not receive/is not exposed to the IV (or gets a markedly lower amount of the IV). A control group provides a baseline for comparison. If the IV impacts the DV, then the experimental and control groups should have different levels of the DV as a result of their different experiences with the IV.

Let’s walk through these basic components of an experiment using the hypothesis that those who receive study guides will have higher exam scores than those who do not. The IV would be the study guide. An experiment could be done where a group of students are randomly assigned to either receive a study guide or not. The DV would be exam scores. To gather data on the DV, participants in both the experimental and control groups could be asked to complete the same exam. The exam scores would then be used to compute the mean exam scores for each of the two groups. If the mean for the experimental group (those who received study guides) is significantly greater than the mean for the control group (those who did not receive study guides), the directional hypothesis would be supported. In order to test whether the means are significantly greater in one group than the other, an independent samples t-test must be used. This is just one of many examples of times when data between two different groups could be tested using an independent samples t-test.

Non-Experimental Designs

The independent samples t-test is flexible and can also be used to compare the means of two different groups when data were acquired using a non-experimental research design. Therefore, though it is often used to test data from experiments by comparing an experimental group to a control group, it can also be used to test data from pre-existing groups in non experimental research. For example, the independent samples t-test can be used to compare the sleep hours of teens to those of young adults. It could also be used to compare the mean commute times of people who work morning shifts to those who work night shifts or to compare the satisfaction of clients who worked with either of two different salespersons.

When to Use Causal Language

Because the independent samples t-test can be used with experimental designs that deduce cause-effect and with non-experimental designs which cannot be used to deduce cause-effect, it is necessary to use causal language only when appropriate. Causal language includes words and phrases such as “caused, “impacted,” or “resulted in” to connect the variables. For example, if we say “Study guides caused mean exam scores to differ” or “The availability of study guides resulted in higher exam scores” we are making a claim that a causal relationship exists. Causal language can also be subtler. For example, if we say “Study guides led to differences in scores,” “Study guides made scores higher,” or even “Those who get study guides will get higher scores, on average” we are making causal claims. This kind of language should only be used when an experiment has been performed, not simply anytime two groups are compared and/or when independent samples t-tests are used. Therefore, it is best to use non causal language as a default and to only switch to using causal language when it is known that an experimental design was used and that causal language is appropriate.

Here are examples of how the claims from the prior paragraph could be rewritten without using causal language: Differences in scores were observed for those who did and did not receive study guides; The mean exam score was higher for the group which received study guides than for the group which did not receive study guides; Those who got study guides had higher scores, on average, than those who did not.

Reading Review 8.1

1. What is the general research hypothesis that can be tested using a one-tailed, independent samples t-test where Group 1 is expected to have the lower mean? Provide both sentence and symbol formats.
2. What is meant by homogeneity of variances?
3. How many grouping variables can be used each time an independent samples t-test is performed?
4. Under what conditions can causal language be used when performing or interpreting results using the independent samples t-test?