Simultaneous Inference

We want to find confidence interval for more than one parameter simultaneously. For example we might want to find confidence interval for \(\beta_0 \) and \( \beta_1 \).

Bonferroni Joint Confidence Intervals

The confidence coefficients for individual parameters are adjusted to the higher 1 - \( \alpha \) so that the confidence coefficient for the collection of parameters must be at least 1 - \( \alpha \). This is based on the following inequality:

Theorem (Bonferroni's Inequality)

\[P( \beta_0\cap\beta_1)\geq1-P(\beta_0^c)-P(\beta_1^c) \label{Bonferroni}\]

for any two events \(\beta_0 \) and \( \beta_1 \), where \(\beta_0^c \) and \(\beta_1^c \) are complements of events \( \beta_0 \) and \( \beta_1 \), respectively.

We take, \( \beta_0 =\) the event that confidence interval for \(\beta_0 \) covers \(\beta_0 \); and, \( \beta_1 =\) the event that confidence interval for \( \beta_1 \) covers \( \beta_1 \);

So, if \(P(\beta_0) = 1-\alpha_1 \), and \(P(\beta_1) = 1-\alpha_2 \), then \(P(\beta_0\cap\beta_1)\geq1-\alpha_1-\alpha_2 \), by Bonferroni's inequality (Equation \ref{Bonferroni}). Note that \(\beta_0\cap\beta_1\) is the event that confidence intervals for both the parameters cover the respective parameters. Therefore we take \(\alpha_1 = \alpha_2 = \alpha/2 \) to get joint confidence intervals with confidence coefficient at least \(1 - \alpha \),

\(b_0 \pm t(1-\alpha/4;n-2) s(b_0) \) and \(b_1 \pm t(1-\alpha/4;n-2) s(b_1) \) for \(\beta_0\) and \(\beta_1\), respectively.

Bonferroni Joint Confidence Intervals for Mean Response

We want to find the simultaneous confidence interval for \(E(Y|X = X_h) = \beta_0 + \beta_1X_h\) for g different values of \(X_h\). Using Bonferroni's inequality for the intersection of g different events, the confidence intervals with confidence coefficient (at least) \(1-\alpha\) are given by

\[\widehat{Y_h} \pm t(1-\alpha/2g; n-2)s(\widehat{Y_h}).\]

Confidence band for regression line : Working-Hotelling procedure

The confidence band

\[\widehat{Y_h} \pm\sqrt{2F(1-\alpha;2,n-2)}s(\widehat{Y_h})\]

contains the entire regression line (for all values of \(X\)) with confidence level \(1-\alpha\). The Working-Hotelling procedure for obtaining the \(1-\alpha\) simultaneous confidence band for the mean responses, therefore, is to use these confidence limits for the g different values of \(X_h\).

Simultaneous prediction intervals

Recall that, the standard error of prediction for a new observation \(Y_{h(new)}\) with \(X = X_h\), is $$s(Y_{h(new)}-\widehat{Y_h}) = \sqrt{MSE(1+\frac{1}{n}+\frac{(X_h-\overline{X})^2}{\sum_i(X_i-\overline{X})^2})}$$ In order to predict the new observations for g different values of X, we may use one of the two procedures:

• Bonferroni procedure : $$\widehat{Y_h}\pm t(1-\alpha/2g;n-2)s(Y_{h(new)}-\widehat{Y_h})$$.
• Scheffe procedure : $$\widehat{Y_h}\pm \sqrt{gF(1-\alpha;g,n-2)}s(Y_{h(new)}-\widehat{Y_h})$$.

Remark : A point to note is that except for the Working-Hotelling procedure for finding simultaneous confidence intervals for mean response, in all the other cases, the confidence intervals become wider as g increases.

Which method to choose : Choose the method which leads to narrower intervals. As a comparison between Bonferroni and Working-Hotelling (for finding confidence intervals for the mean response), the following can be said :

• If g is small, Bonferroni is better.
• If g is large, Working-Hotelling is better (the coefficient of \(s(\widehat{Y_h})\) in the confidence limits remains the same even as g becomes large).

Housing data as an example

Fitted regression model : \(\widehat{Y_h} = 28.981 + 2.941X, n=19, s(b_0) = 8.5438, s(b_1) = 0.5412, MSE = 11.9512.\)

• Simultaneous confidence intervals for \(\beta_0\) and \(\beta_1\) : For 95% simultaneous C.I., \(t(1-\alpha/4;n-2)=t(0.9875;17) = 2.4581\). The intervals are (for \(\beta_0 and \beta_1\), respectively)

\(28.981 \pm 2.4581 \times 8.5438 \equiv 28.981 \pm 21.002, 2.941 \pm 2.4581 \times 0.5412 \equiv 2.941 \pm 1.330\)

• Simultaneous inference for mean response at g different values of X : Say g = 3.

And the values are

$$X_h$$	14	16	18.5
$$\widehat{Y_h}$$	70.155	76.037	83.390
$$s(\widehat{Y_h})$$	1.2225	0.8075	1.7011

\(t(1-0.05/2g;n-2) = t(0.99167;17) = 2.655, \sqrt{2F(0.95;2,n-2)} = \sqrt{2 \times 3.5915} = 2.6801 \)

The 95% simultaneous confidence intervals for the mean responses are given in the following table:

$$X_h$$	14	16	18.5
Bonferroni	$$70.155 \pm 3.248$$	$$76.037 \pm 2.145$$	$$83.390 \pm 4.520$$
Working-Hotelling	$$70.155 \pm 3.276$$	$$76.037 \pm 2.164$$	$$83.390 \pm 4.559$$

• Simultaneous prediction intervals for g different values of X : Again, say g = 3 and the values of 14,16 and 18.5. In this case, \(\alpha = 0.05, t(1-\alpha/2g;n-2) = t(0.99167; 17) = 2.655\). And \(\sqrt{gF(1-\alpha;g,n-2)} = \sqrt{3F(0.95;3,17)} = \sqrt{3 \times 3.1968} = 3.0968\). The standard errors and simultaneous 95% C.I. are given in the following table:

$$X_h$$	14	16	18.5
$$\widehat{Y_h}$$	70.155	76.037	83.390
$$s(Y_{h(new)}-\widehat{Y_h})$$	3.6668	3.5501	3.8529
Bonferroni	$$70.155 \pm 9.742$$	$$76.037 \pm 9.432$$	$$83.390 \pm 10.237$$
Scheffe	$$70.155 \pm 11.355$$	$$76.037 \pm 10.994$$	$$83.390 \pm 11.932$$

Contributors

• Anirudh Kandada(UCD)