Skip to main content
Statistics LibreTexts

7.5.2: Summary of p-values and NHST

  • Page ID
    22072
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    We use p-values (probability) to determine if this could have happened by chance. We are comparing the mean of our sample to the mean of the population to ask “Is it likely that my sample is from the population?”

    Since we can’t ever collect data from the whole population, we’re forced to make inferences from the available sample data, and the probability of events happening that we know from the Standard Normal Distribution. We know that any score that is more 3 or more standard deviations from the mean (z = 3) is very rare. So, we compare the mean of a sample to the population mean (after standardizing so that we can use the Standard Normal Distribution) to see if the mean of the sample is extreme enough to say that the probability of the sample being from the population is below 5%.

    The p-value tells us the probability of getting an effect this different if the sample is the same as the population.

    If the probability (p-value) is small enough (p< .05), then we conclude that the sample mean probably is from a different population.

    Exercise \(\PageIndex{1}\)

    What does the null hypothesis say?

    Answer

    The null hypothesis says that there is no difference between the groups, that the mean of the sample is similar to the mean of the pouplation.

    Example \(\PageIndex{1}\)

    If we’re saying that the sample is probably different from the population, are we rejecting or retaining (failing to reject) the null hypothesis?

    Solution

    If we’re saying that the sample is probably different from the population, we reject the null hypothesis.

    Interpreting p-values

    Without having to understand everything about probability distributions and the Standard Normal Distribution, what do the p-values tell us?

    Small p-values

    A small p-value means a small probability that the two means are similar.

    Suggesting that the means are different…

    We conclude that:

    • The means are different.
    • The sample is not from the population.

    Large p-values

    A large p-value means a large probability that the two means are similar.

    We conclude that

    • The means are similar.
    • The sample is from the population.
    Note

    In other words:

    Reject null \(= (\bar{X} \neq \mu) = p<.05\)

    Retain null \( = (\bar{X} = \mu) = p>.05\)

    If the probability is less than 5% that you would get a sample mean that is this different from the population mean if the sample really is from the population, the sample is probably not from that population. If you took 100 samples from a population, less than 5 of the samples’ means would be this different from the population’s mean if the samples were different in reality. So, the sample is probably not from the population. Probably.

    You have a 5% chance that your results are wrong. You have a (small) chance that your sample mean is this different from the population mean, but the sample is still actually from the population.

    Statisticians are okay with being wrong 5% of the time!

    Let's practice!​​​​​​

    Example \(\PageIndex{2}\)

    For each:

    1. Determine whether to report “p<.05 ” or “p>.05”
    2. Determine whether to retain or reject the null hypothesis.
    3. Determine whether the sample mean and population mean are similar or different.

    Hint:

    Reject null \(= (\bar{X} \neq \mu) = p<.05\)

    Retain null \( = (\bar{X} = \mu) = p>.05\)

    1. p < .05

    2. p = .138

    3. p = .510

    Solution

    1. p < .05
      1. “p<.05 ”
      2. Reject the null hypothesis.
      3. The sample mean and population mean are different.
    2. p = .138
      1. “p>.05” (because 0.138 > 0.05, 0.138 is bigger than 0.05)
      2. Retain the null hypothesis.
      3. The sample mean and population mean are similar.
    3. p = .510
      1. “p>.05” (We’re comparing to 0.05, or 5%, not .50, or 50%)
      2. Retain the null hypothesis.
      3. The sample mean and population mean are similar.

    Now, try it yourself:

    Exercise \(\PageIndex{2}\)

    For each:

    1. Determine whether to report “p<.05 ” or “p>.05”
    2. Determine whether to retain or reject the null hypothesis.
    3. Determine whether the sample mean and population mean are similar or different.

    Hint:

    Reject null \(= (\bar{X} \neq \mu) = p<.05\)

    Retain null \( = (\bar{X} = \mu) = p>.05\)

    1. p > .05

    2. p = .032

    3. p = .049

    Answer
    1. p > .05
      1. “p>.05 ”
      2. Retain the null hypothesis.
      3. The sample mean and population mean are similar.
    2. p = .032
      1. “p<.05 ” or “p>.05” (because 0.32 < .05, 0.032 is less than 0.05)
      2. Reject the null hypothesis.
      3. The sample mean and population mean are different.
    3. p = .049
      1. “p<.05 ” (even though it’s close, .049 is smaller than 0.05)
      2. Reject the null hypothesis.
      3. The sample mean and population mean are different.

    This page titled 7.5.2: Summary of p-values and NHST is shared under a CC BY-SA 4.0 license and was authored, remixed, and/or curated by Michelle Oja.

    • Was this article helpful?