Skip to main content
Statistics LibreTexts

8.10: Different Significance Level

  • Page ID
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

    ( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\id}{\mathrm{id}}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\kernel}{\mathrm{null}\,}\)

    \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\)

    \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\)

    \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    \( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

    \( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

    \( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vectorC}[1]{\textbf{#1}} \)

    \( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

    \( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

    \( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    Finally, let’s take a look at an example phrased in generic terms, rather than in the context of a specific research question, to see the individual pieces one more time. This time, however, we will use a stricter significance level, \(α\) = 0.01, to test the hypothesis.

    Step 1: State the Hypotheses

    We will use 60 as an arbitrary null hypothesis value:

    \(H_0\): The average score does not differ from the population

    \(H_0: \mu = 60\)

    We will assume a two-tailed test:

    \(H_A\): The average score does differ

    \(H_A: μ ≠ 60\)

    Step 2: Find the Critical Values

    We have seen the critical values for \(z\)-tests at \(α\) = 0.05 levels of significance several times. To find the values for \(α\) = 0.01, we will go to the standard normal table and find the \(z\)-score cutting of 0.005 (0.01 divided by 2 for a two-tailed test) of the area in the tail, which is \(z*\) = ±2.575. Notice that this cutoff is much higher than it was for \(α\) = 0.05. This is because we need much less of the area in the tail, so we need to go very far out to find the cutoff. As a result, this will require a much larger effect or much larger sample size in order to reject the null hypothesis.

    Step 3: Calculate the Test Statistic

    We can now calculate our test statistic. We will use \(σ\) = 10 as our known population standard deviation and the following data to calculate our sample mean:

    61 62
    65 61
    58 59
    54 61
    60 63

    The average of these scores is \(M\)= 60.40. From this we calculate our \(z\)-statistic as:

    \[z=\dfrac{60.40-60.00}{10.00 / \sqrt{10}}=\dfrac{0.40}{3.16}=0.13 \nonumber \]

    Step 4: Make the Decision

    Our obtained \(z\)-statistic, \(z\) = 0.13, is very small. It is much less than our critical value of 2.575. Thus, this time, we fail to reject the null hypothesis. Our conclusion would look something like:

    Based on the sample of 10 scores, we cannot conclude that there is no effect causing the mean (\(M\)= 60.40) to be statistically significantly different from 60.00, \(z\) = 0.13, \(p\) > 0.01.

    Notice two things about the end of the conclusion. First, we wrote that \(p\) is greater than instead of p is less than, like we did in the previous two examples. This is because we failed to reject the null hypothesis. We don’t know exactly what the \(p\)-value is, but we know it must be larger than the \(α\) level we used to test our hypothesis. Second, we used 0.01 instead of the usual 0.05, because this time we tested at a different level. The number you compare to the \(p\)-value should always be the significance level you test at.

    Finally, because we did not detect a statistically significant effect, we do not need to calculate an effect size.

    Contributors and Attributions

    • Foster et al. (University of Missouri-St. Louis, Rice University, & University of Houston, Downtown Campus)

    This page titled 8.10: Different Significance Level is shared under a CC BY-NC-SA 4.0 license and was authored, remixed, and/or curated by Foster et al. (University of Missouri’s Affordable and Open Access Educational Resources Initiative) via source content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request.