6.6: Introduction to Mixed Models

Treatment designs can comprise both fixed and random effects. When we have this situation the treatment design is referred to as a mixed model. Mixed models are by far the most commonly encountered treatment designs. The three situations we now have are often referred to as Model I (fixed effects only), Model II (random effects only), and Model III (mixed) ANOVAs. In designating the effects of a mixed model as mixed or random, the following rule will be useful.

Rule! Any interaction or nested effect containing at least one random factor is random.

Below are the ANOVA layouts of two basic mixed models with two factors.

Factorial

In the simplest case of a balanced mixed model, we may have two factors, A and B, in a factorial design in which factor A is a fixed effect and factor B is a random effect.

The statistical model is similar to what we have seen before: \[y_{ijk} = \mu + \alpha_{i} + \beta_{j} + \left(\alpha \beta\right)_{ij} + \epsilon_{ijk}\] where \(i = 1, 2, \ldots, a\), \(j=1, 2, \ldots, b\), and \(k = 1,2, \ldots, n\).

Here, \(\sum_{i} \alpha_{i} = 0\), \(\beta_{j} \sim \mathcal{N} \left(0, \sigma_{\beta}^{2}\right)\), \((\alpha \beta)_{i,j} \sim \mathcal{N} \left(0, \frac{a-1}{a} \sigma_{\alpha \beta}^{2}\right)\), \(\sum_{i} (\alpha \beta)_{i,j} = 0\) and \(\epsilon_{ijk} \sim \mathcal{N} \left(0, \sigma^{2}\right)\). Also, \(\beta_{j}\), \((\alpha \beta)_{ij}\), and \(\epsilon_{ij}\) are pairwise independent.

In this case, we have the following ANOVA.

Source	DF	EMS
A	\((a-1)\)	\(\sigma^{2} + nb \frac{\sum \alpha_{i}^{2}}{a-1} + n \sigma_{\alpha \beta}^{2}\)
B	\((b-1)\)	\(\sigma^{2} + na \sigma_{\beta}^{2}\)
A × B	\((a-1)(b-1)\)	\(\sigma^{2} + n \sigma_{\alpha \beta}^{2}\)
Error	\(ab(n-1)\)	\(\sigma^{2}\)
Total	\(abn-1\)

The \(F\)-tests are set up based on the EMS column above and we can see that we have to use different denominators in testing significance for the various sources in the ANOVA table:

Source	EMS	F
A	\(\sigma^{2} + nb \frac{\sum \alpha_{i}^{2}}{a-1} + n \sigma_{\alpha \beta}^{2}\)	MSA / MSAB
B	\(\sigma^{2} + na \sigma_{\beta}^{2}\)	MSB / MSE
A × B	\(\sigma^{2} + n \sigma_{\alpha \beta}^{2}\)	MSAB / MSE
Error	\(\sigma^{2}\)
Total

As a reminder, the null hypothesis for a fixed effect is that the \(\alpha_{i}\)'s are equal, whereas the null hypothesis for the random effect is that the \(\sigma_{\beta}^{2}\)'s are equal to zero.

Note

The denominator for the \(F\)-test for the main effect of factor A is now the MS for the A × B interaction. For Factor B and the A × B interaction, the denominator is the MSE.

Nested

In the case of a balanced nested treatment design, where A is a fixed effect and B(A) is a random effect, the statistical model would be: \[y_{ijk} = \mu + \alpha_{i} + \beta_{j(i)} + \epsilon_{ijk}\] where \(i = 1,2, \ldots, a\), \(j = 1, 2, \ldots, b\), and \(k = 1, 2, \ldots, n\).

Here, \(\sum_{i} \alpha_{i} = 0\), \(\beta_{j(i)} \sim \mathcal{N} \left(0, \sigma_{\beta}^{2}\right)\), and \(\epsilon_{ijk} = \mathcal{N} \left(0, \sigma^{2}\right)\).

We have the following ANOVA for this model:

Source	DF	EMS
A	\((a-1)\)	\(\sigma_{\epsilon}^{2} + n \sigma_{\beta(\alpha)}^{2} + bn \frac{\sum \alpha_{i}^{2}}{a-1}\)
B(A)	\(a(b-1)\)	\(\sigma_{\epsilon}^{2} + n \sigma_{\beta(\alpha)}^{2}\)
Error	\(ab(n-1)\)	\(\sigma_{\epsilon}^{2}\)
Total	\(abn-1\)

Here is the same table with the \(F\)-statistics added. Note that the denominators for the \(F\)-test are different.

Source	EMS	F
A	\(\sigma_{\epsilon}^{2} + n \sigma_{\beta(\alpha)}^{2} + bn \frac{\sum \alpha_{i}^{2}}{a-1}\)	MSA / MSB(A)
B(A)	\(\sigma_{\epsilon}^{2} + n \sigma_{\beta(\alpha)}^{2}\)	MSB(A) / MSE
Error	\(\sigma_{\epsilon}^{2}\)
Total

\(F\)-Calculation Facts

As can be seen from the examples above and also from sections 6.3-6.6, when significance testing in random or mixed models, the denominator of the \(F\)-statistic is no more the MSE value and has to be aptly chosen. Recall that the \(F\)-statistic for testing the significance of a given effect is the ratio with the numerator equal to the MS value of the effect, and the denominator is also an MS value of an effect included in the ANOVA model. Furthermore, the \(F\)-statistic has a non-central distribution when \(H_{a}\) is true and a central \(F\)-distribution when \(H_{0}\) is true.

The non-centrality parameter of the non-central F distribution when \(H_{a}\) is true depends on the type of effect (fixed vs random), and equals \sum_{i=1}^{T} \alpha_{i}^{2}\) for a fixed effect and \(\sigma_{trt}^{2}\) for a random effect. Here \(\alpha_{i} = \mu_{i} - \mu\), where \(\mu_{i} \ (i=1, 2, \ldots, T)\) is the \(i^{th}\) level of the fixed effect and \(\mu\) is the overall mean while \(\sigma_{trt}^{2}\) is the variance component associated with the random effect. Also, MS under true \(H_{a}\) equals to MS under true \(H_{0}\) plus non-centrality parameter, so that \[F \text{-statistic} = \frac{\text{MS when } H_{0} \text{ is true + non-centrality parameter}}{\text{MS when } H_{0} \text{ is true}}\]

The above identity can be used to identify the correct denominator (also called the error term) with the aid of EMS expressions displayed in the ANOVA table.

Rule! The \(F\)-statistic denominator is the MS value of the source which has an EMS containing all EMS terms in the effect except the non-centrality parameter.