Skip to main content
Statistics LibreTexts

2.4: Threats to Validity

  1. Last updated
  2. Save as PDF
  • Page ID
    7211

    • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    To understand the pros and cons of various designs and to be able to better judge specific designs, we identify specific threats to internal and external validity. Before we do so, it is important to note that a (perhaps the) primary challenge to establishing internal validity in the social sciences is the fact that most of the phenomena we care about have multiple causes and are often a result of some complex set of interactions. For example, X may be only a partial cause of Y or X may cause Y, but only when Z is present. Multiple causation and interactive effects make it very difficult to demonstrate causality, both internally and externally. Turning now to more specific threats, Table 2.1 identifies common threats to internal validity and Table 2.2 identifies common threats to external validity.

    Figure \(\PageIndex{1}\): Common Threats to Internal Validity
    Threat
    History Any event that occurs while the experiment is in progress might be an alternation; using a control group mitigates this concern.
    Maturation Normal changes over time (e.g., fatigue or aging) might affect the dependent variable; using a control group mitigates this concern
    Selection Bias If randomization is not used to assign participants, the groups may not be equivalent
    Experimental Mortality If groups lost participants (e.g., due to dropping out of the experiment) they may not be equivalent.
    Testing A pre-test may confound the influence of the experimental treatment; using a control group mitigates this concern
    Instrumentation Changes or difference in the process of measurements might alternatively account for differences
    Statistical Regression The natural tendency for extreme scores to regress or move towards the mean

    Figure \(\PageIndex{2}\): Common Threats to External Validity

    tbl2.png

    This page titled 2.4: Threats to Validity is shared under a CC BY 4.0 license and was authored, remixed, and/or curated by Jenkins-Smith et al. (University of Oklahoma Libraries) via source content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request.