11.3: More on Covariance Structures

Variance Components (VC)

\[\begin{bmatrix} \sigma_{1}^{2} & \cdots & \cdots & \vdots \\ \vdots & \sigma_{1}^{2} & \cdots & \vdots \\ \vdots & \cdots & \sigma_{1}^{2} \vdots \\ \cdots & \cdots & \cdots & \sigma_{1}^{2} \end{bmatrix}\]

The variance component structure (VC) is the simplest, where the correlations of errors within a subject are presumed to be 0. This structure is the default setting in proc mixed, but is not a reasonable choice for most repeated measures designs. It is included in the exploration process to get a sense of the effect of fitting other structures.

Compound Symmetry (CS)

\[\sigma^{2} \begin{bmatrix} 1.0 & \rho & \rho & \rho \\ & 1.0 & \rho & \rho \\ & & 1.0 & \rho \\ & & & 1.0 \end{bmatrix} = \begin{bmatrix} \sigma_{b}^{2} + \sigma_{e}^{2} & \sigma_{b}^{2} & \sigma_{b}^{2} & \sigma_{b}^{2} \\ & \sigma_{b}^{2}+\sigma_{e}^{2} & \sigma_{b}^{2} & \sigma_{b}^{2} \\ & & \sigma_{b}^{2}+\sigma_{e}^{2} & \sigma_{b}^{2} \\ & & & \sigma_{b}^{2}+\sigma_{e}^{2} \end{bmatrix}\]

The simplest covariance structure that includes within-subject correlated errors is compound symmetry (CS). Here we see correlated errors between time points within subjects, and note that these correlations are presumed to be the same for each set of times, regardless of how distant in time the repeated measures are made.

First Order Autoregressive AR(1)

\[\sigma^{2} \begin{bmatrix} 1.0 & \rho & \rho^{2} & \rho^{3} \\ & 1.0 & \rho & \rho^{2} \\ & & 1.0 & \rho \\ & & & 1.0 \end{bmatrix}\]

The autoregressive (Lag 1) structure considers correlations to be highest between adjacent times, and a systematically decreasing correlation with increasing distance between time points. For one subject, the error correlation between time 1 and time 2 would be \(\rho^{t_{2}-t_{1}}\). Between time 1 and time 3 the correlation would be less, and equal to \(\rho^{t_{3}-t_{1}}\). Between time 1 and 4, the correlation is lesser, as \(\rho^{t_{4}-t_{1}}\), and so on. Note that this structure is only applicable for evenly spaced time intervals for the repeated measure; so that consecutive correlations are \(\rho\) raised to powers of 1, 2, 3, etc.

Spatial Power

\[\sigma^{2} \begin{bmatrix} 1.0 & \rho^{\left| \frac{t_1-t_2}{t_1-t_2} \right|} & \rho^{\left| \frac{t_1-t_3}{t_1-t_2} \right|} & \rho^{\left| \frac{t_1-t_4}{t_1-t_2} \right|} \\ & 1.0 & \rho^{\left| \frac{t_2-t_3}{t_1-t_2} \right|} & \rho^{\left| \frac{t_2-t_4}{t_1-t_2} \right|} \\ & & 1.0 & \rho^{\left| \frac{t_3-t_4}{t_1-t_2} \right|} \\ & & & 1.0 \end{bmatrix}\]

When time intervals are not evenly spaced, a covariance structure equivalent to the AR(1) is the spatial power (SP(POW)). The concept is the same as the AR(1) but instead of raising the correlation to powers of 1, 2, 3, …, the correlation coefficient is raised to a power that is the actual difference in times (e.g. \(t^{2}-t^{1}\) for the correlation between time 1 and time 2). It is clear that this method requires having quantitative values for the variable time in the data so that it can be specified for the calculation of the exponents in the SP(POW) structure. If an analysis is run wherein the repeated measures are equally spaced in time, the AR(1) and SP(POW) structures yield identical results.

Unstructured Covariance

\[\begin{bmatrix} \sigma_{1}^{1} & \sigma_{12} & \sigma_{13} & \sigma_{14} \\ & \sigma_{2}^{2} & \sigma_{23} & \sigma_{24} \\ & & \sigma_{3}^{2} & \sigma_{34} \\ & & & \sigma_{4}^{2} \end{bmatrix}\]

The Unstructured covariance structure (UN) is the most complex because it is estimating unique correlations for each pair of time points. As there are too many parameters (all distinct correlations), the estimates most times will not be computable. SAS for instance returns an error message indicating that there are too many parameters to estimate with the data.

Choosing the Best Covariance Structure

The fit statistics used for model selection can also be utilized in choosing the best covariance matrix. The model selections most commonly supported by software are -2 Res Log Likelihood, Akaike’s information criterion - corrected (AICC), and Bayesian Information Criteria (BIC). These statistics are functions of the log likelihood and can be compared across different models as well as different covariance structures provided the fixed effects part is the same in each model. The smaller the criterion statistics value is, the better the model is, and if they are close, the simpler model is preferred.

BIC tends to choose simpler models compared to AICC. Choosing a model that is too simple however inflates the Type I error rate. Therefore, if controlling Type I error is of importance, AICC may be the better criterion. On the other hand, if loss of power is of more concern, BIC might be preferable (Guerin and Stroup 2000).

The MIXED procedure in SAS outputs the 3 criterion statistics when using the type = option in the Repeated statement.

In addition to using the above fit statistics, graphical approaches are also available, and see Graphical Approach for more details. Combining information from both approaches to make the final choice may also prove to be beneficial.