Skip to main content
Statistics LibreTexts

6.2: The Sampling Distribution of Sample Means

  • Page ID
    7108
  • To see how we use sampling error, we will learn about a new, theoretical distribution known as the sampling distribution. In the same way that we can gather a lot of individual scores and put them together to form a distribution with a center and spread, if we were to take many samples, all of the same size, and calculate the mean of each of those, we could put those means together to form a distribution. This new distribution is, intuitively, known as the distribution of sample means. It is one example of what we call a sampling distribution, we can be formed from a set of any statistic, such as a mean, a test statistic, or a correlation coefficient (more on the latter two in Units 2 and 3). For our purposes, understanding the distribution of sample means will be enough to see how all other sampling distributions work to enable and inform our inferential analyses, so these two terms will be used interchangeably from here on out. Let’s take a deeper look at some of its characteristics. 

    The sampling distribution of sample means can be described by its shape, center, and spread, just like any of the other distributions we have worked with. The shape of our sampling distribution is normal: a bell-shaped curve with a single peak and two tails extending symmetrically in either direction, just like what we saw in previous chapters. The center of the sampling distribution of sample means – which is, itself, the mean or average of the means – is the true population mean, \(μ\). This will sometimes be written as \(\mu_{\overline{X}}\) to denote it as the mean of the sample means. The spread of the sampling distribution is called the standard error, the quantification of sampling error, denoted \(\mu_{\overline{X}}\). The formula for standard error is: 

    \[\sigma_{\overline{X}}=\dfrac{\sigma}{\sqrt{n}} \]

    Notice that the sample size is in this equation. As stated above, the sampling distribution refers to samples of a specific size. That is, all sample means must be calculated from samples of the same size \(n\), such \(n\) = 10, \(n\) = 30, or \(n\) = 100. This sample size refers to how many people or observations are in each individual sample, not how many samples are used to form the sampling distribution. This is because the sampling distribution is a theoretical distribution, not one we will ever actually calculate or observe. Figure \(\PageIndex{1}\) displays the principles stated here in graphical form.

    fig 6.2.1.png
    Figure \(\PageIndex{1}\): The sampling distribution of sample means

    Two Important Axioms

    We just learned that the sampling distribution is theoretical: we never actually see it. If that is true, then how can we know it works? How can we use something that we don’t see? The answer lies in two very important mathematical facts: the central limit theorem and the law of large numbers. We will not go into the math behind how these statements were derived, but knowing what they are and what they mean is important to understanding why inferential statistics work and how we can draw conclusions about a population based on information gained from a single sample. 

    Central Limit Theorem

    The central limit theorem states:

    Theorem \(\PageIndex{1}\)

    For samples of a single size \(n\), drawn from a population with a given mean \(μ\) and variance \(σ^2\), the sampling distribution of sample means will have a mean \(\mu_{\overline{X}}=\mu\) and variance \(\sigma _{X}^{2}=\dfrac{\sigma ^{2}}{n}\). This distribution will approach normality as \(n\) increases.

    From this, we are able to find the standard deviation of our sampling distribution, the standard error. As you can see, just like any other standard deviation, the standard error is simply the square root of the variance of the distribution. 

    The last sentence of the central limit theorem states that the sampling distribution will be normal as the sample size of the samples used to create it increases. What this means is that bigger samples will create a more normal distribution, so we are better able to use the techniques we developed for normal distributions and probabilities. So how large is large enough? In general, a sampling distribution will be normal if either of two characteristics is true:

    1. the population from which the samples are drawn is normally distributed or
    2. the sample size is equal to or greater than 30.

    This second criteria is very important because it enables us to use methods developed for normal distributions even if the true population distribution is skewed.

    Law of Large Numbers

    The law of large numbers simply states that as our sample size increases, the probability that our sample mean is an accurate representation of the true population mean also increases. It is the formal mathematical way to state that larger samples are more accurate.

    The law of large numbers is related to the central limit theorem, specifically the formulas for variance and standard error. Notice that the sample size appears in the denominators of those formulas. A larger denominator in any fraction means that the overall value of the fraction gets smaller (i.e 1/2 = 0.50, 1/3 – 0.33, 1/4 = 0.25, and so on). Thus, larger sample sizes will create smaller standard errors. We already know that standard error is the spread of the sampling distribution and that a smaller spread creates a narrower distribution. Therefore, larger sample sizes create narrower sampling distributions, which increases the probability that a sample mean will be close to the center and decreases the probability that it will be in the tails. This is illustrated in Figures \(\PageIndex{2}\) and \(\PageIndex{3}\). 

    fig 6.2.2.png
    Figure \(\PageIndex{2}\): Sampling distributions from the same population with \(μ\) = 50 and \(σ\) = 10 but different sample sizes (\(N\) = 10, \(N\) = 30, \(N\) = 50, \(N\) = 100)
    fig 6.2.3.png
    Figure \(\PageIndex{3}\): Relation between sample size and standard error for a constant \(σ\) = 10

    Contributors

    • Foster et al. (University of Missouri-St. Louis, Rice University, & University of Houston, Downtown Campus)