3.3: Designed Experiments
- Page ID
- 58814
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)In many classical scientific experiments, we envision a scientist who sets up the conditions of the experiment and then observes what happens as a result. Within the framework of the scientific method, the conditions of the experiment are stated in the hypothesis. That is, the hypothesis usually states that if the conditions are set in a certain way, then a certain outcome will be observed. The purpose of the experiment is then to observe if the hypothesized outcome occurs, which would provide evidence that the hypothesis is correct. Alternatively, if the hypothesized outcome does not occur, then this would refute the hypothesis. In this type of experiment the conditions set by the researchers are the factors, and what is observed to occur is the response. This type of experiment is called a designed experiment.
A designed experiment is an experiment where the factors are controlled and set by the researcher prior to observing the outcome.
Designed experiments are common in scientific applications because the factors are usually relatively easy to control. For example, one can study the effect on the acceleration of a marble rolling down an inclined plane at different angles by setting the inclined plane at several angles and observing how long it takes the marble to get to the bottom. In social science research these experiments are less common, but they do occur. The most common designed experiments in social justice research are usually psychological experiments that attempt to assess how a stimulus, taken to be a controlled factor, affects the attitude of an individual about a social issue, which is taken to be the response. For example, researchers might measure the empathy of two groups of individuals after viewing two different versions of a documentary. Of interest would be to observe whether there is evidence that the viewpoint shown in the documentary influences the individuals observed in the study. Because the researchers control which version of the documentary each individual views, this is a designed experiment.
A related type of experiment compares individuals who are treated in a special way to those who receive a standard treatment. For example, a researcher may divide a group of students into two groups. One of the groups receives a standard study guide for the final exam in a specific course, while the second group of students is given an enhanced version of the study guide. The purpose of the study is then to observe whether the enhanced study guide improves student performance on the final exam. In a study using this type of framework, the group that receives the standard treatment is called the control group and the group that receives the special treatment is called the treatment group.
In an experiment that compares a standard treatment to a special treatment, the group that receives the standard treatment is called the control group.
In an experiment that compares a standard treatment to a special treatment, the group that receives the special treatment is called the treatment group.
The idea of this type of study is to compare the special treatment, usually a new or potential improvement on the standard treatment, to a standard or established treatment. By including individuals who are given the standard treatment (the control group) we have a baseline for comparison to those given the new or special treatment (the treatment group). Without this baseline observation it is impossible to determine if the special treatment is making any difference. For example, if the researcher wants to determine if an enhanced study guide improves student performance, a measure of student performance using the standard study guide must be used for comparison.
This idea can be expanded to compare several treatments to a control group, or several treatment groups to one another. So, for example, a researcher could compare several types of study guides to observe evidence about which of these approaches is the best, or if any of the approaches is better than the standard study guide.
A crucial question related to the use of a designed experiment is to determine how the individuals in a study are assigned to the treatment and control groups. Considering the example of comparing the two study guides for the final exam, the researchers could give the enhanced study guide to those students whose grades are in the bottom half of the class while giving the standard study guide to those students whose grades are in the top half of the class. This may seem like a good idea in that the enhanced study guide should help the students who are having trouble with the class. Unfortunately, from the perspective of the scientific method, this would be a poor choice. If, for example, the grades for the students who had the enhanced study guide were lower on average than for the students who had the standard study guide, then it would be impossible to know if this was because the enhanced study guide did not work or if it was because the students tended not to do well in the class to begin with. Issues with determining what individuals will be in the treatment group and which individuals will be in the control group will be considered in greater detail in the next section. The best way to choose the participants for each group is to do it based on a mechanism that is completely unrelated to student performance. The easiest method for implementing such a selection methodology, randomization, will be covered later in the book.