# Simultaneous Inference

- Page ID
- 240

We want to find confidence interval for more than one parameter simultaneously. For example we might want to find confidence interval for **\(\beta_0 \) **and **\( \beta_1 \)**.

## Bonferroni Joint Confidence Intervals

The confidence coefficients for individual parameters are adjusted to the higher 1 - \( \alpha \) so that the confidence coefficient for the collection of parameters must be at least 1 - \( \alpha \). This is based on the following inequality:

Theorem (Bonferroni's Inequality)

\[P( \beta_0\cap\beta_1)\geq1-P(\beta_0^c)-P(\beta_1^c) \label{Bonferroni}\]

for any two events \(\beta_0 \) and \( \beta_1 \), where \(\beta_0^c \) and \(\beta_1^c \) are complements of events \( \beta_0 \) and \( \beta_1 \), respectively.

We take, \( \beta_0 =\) the event that confidence interval for \(\beta_0 \) covers \(\beta_0 \); and, \( \beta_1 =\) the event that confidence interval for **\( \beta_1 \) **covers **\( \beta_1 \)**;

So, if \(P(\beta_0) = 1-\alpha_1 \), and \(P(\beta_1) = 1-\alpha_2 \), then \(P(\beta_0\cap\beta_1)\geq1-\alpha_1-\alpha_2 \), by *Bonferroni's inequality *(Equation \ref{Bonferroni}). Note that \(\beta_0\cap\beta_1\) is the event that confidence intervals for both the parameters cover the respective parameters. Therefore we take \(\alpha_1 = \alpha_2 = \alpha/2 \) to get joint confidence intervals with confidence coefficient at least \(1 - \alpha \),

\(b_0 \pm t(1-\alpha/4;n-2) s(b_0) \) *and* \(b_1 \pm t(1-\alpha/4;n-2) s(b_1) \) for \(\beta_0\) and \(\beta_1\), respectively.

## Bonferroni Joint Confidence Intervals for Mean Response

We want to find the simultaneous confidence interval for \(E(Y|X = X_h) = \beta_0 + \beta_1X_h\) for *g *different values of \(X_h\). Using *Bonferroni's inequality *for the intersection of *g* different events, the confidence intervals with confidence coefficient (at least) \(1-\alpha\) are given by

\[\widehat{Y_h} \pm t(1-\alpha/2g; n-2)s(\widehat{Y_h}).\]

## Confidence band for regression line : Working-Hotelling procedure

The confidence band

\[\widehat{Y_h} \pm\sqrt{2F(1-\alpha;2,n-2)}s(\widehat{Y_h})\]

contains the entire regression line (for all values of \(X\)) with confidence level \(1-\alpha\). The Working-Hotelling procedure for obtaining the \(1-\alpha\) simultaneous confidence band for the mean responses, therefore, is to use these confidence limits for the *g* different values of \(X_h\).

## Simultaneous prediction intervals

Recall that, the standard error of prediction for a new observation \(Y_{h(new)}\) with \(X = X_h\), is $$s(Y_{h(new)}-\widehat{Y_h}) = \sqrt{MSE(1+\frac{1}{n}+\frac{(X_h-\overline{X})^2}{\sum_i(X_i-\overline{X})^2})}$$ In order to predict the new observations for *g* different values of *X*, we may use one of the two procedures:

**Bonferroni procedure :**$$\widehat{Y_h}\pm t(1-\alpha/2g;n-2)s(Y_{h(new)}-\widehat{Y_h})$$.**Scheffe****procedure :**$$\widehat{Y_h}\pm \sqrt{gF(1-\alpha;g,n-2)}s(Y_{h(new)}-\widehat{Y_h})$$.

**Remark : **A point to note is that except for the Working-Hotelling procedure for finding simultaneous confidence intervals for mean response, in all the other cases, the confidence intervals become wider as *g *increases.

**Which method to choose :** Choose the method which leads to narrower intervals. As a comparison between Bonferroni and Working-Hotelling (for finding confidence intervals for the mean response), the following can be said :

- If
*g*is small, Bonferroni is better. - If
*g*is large, Working-Hotelling is better (the coefficient of \(s(\widehat{Y_h})\) in the confidence limits remains the same even as*g*becomes large).

## Housing data as an example

Fitted regression model : \(\widehat{Y_h} = 28.981 + 2.941X, n=19, s(b_0) = 8.5438, s(b_1) = 0.5412, MSE = 11.9512.\)

**Simultaneous confidence intervals for \(\beta_0\) and \(\beta_1\) :**For 95% simultaneous C.I., \(t(1-\alpha/4;n-2)=t(0.9875;17) = 2.4581\). The intervals are (for \(\beta_0 and \beta_1\), respectively)

**\(28.981 \pm 2.4581 \times 8.5438 \equiv 28.981 \pm 21.002, 2.941 \pm 2.4581 \times 0.5412 \equiv 2.941 \pm 1.330\)**

**Simultaneous inference for mean response at**Say*g*different values of X :*g*= 3.

And the values are

$$X_h$$ | 14 | 16 | 18.5 |

$$\widehat{Y_h}$$ | 70.155 | 76.037 | 83.390 |

$$s(\widehat{Y_h})$$ | 1.2225 | 0.8075 | 1.7011 |

\(t(1-0.05/2g;n-2) = t(0.99167;17) = 2.655, \sqrt{2F(0.95;2,n-2)} = \sqrt{2 \times 3.5915} = 2.6801 \)

The 95% simultaneous confidence intervals for the mean responses are given in the following table:

$$X_h$$ | 14 | 16 | 18.5 |

Bonferroni | $$70.155 \pm 3.248$$ | $$76.037 \pm 2.145$$ | $$83.390 \pm 4.520$$ |

Working-Hotelling | $$70.155 \pm 3.276$$ | $$76.037 \pm 2.164$$ | $$83.390 \pm 4.559$$ |

**Simultaneous prediction intervals for***g***different values of**Again, say*X*:*g*= 3 and the values of 14,16 and 18.5. In this case, \(\alpha = 0.05, t(1-\alpha/2g;n-2) = t(0.99167; 17) = 2.655\). And \(\sqrt{gF(1-\alpha;g,n-2)} = \sqrt{3F(0.95;3,17)} = \sqrt{3 \times 3.1968} = 3.0968\). The standard errors and simultaneous 95% C.I. are given in the following table:

$$X_h$$ | 14 | 16 | 18.5 |

$$\widehat{Y_h}$$ | 70.155 | 76.037 | 83.390 |

$$s(Y_{h(new)}-\widehat{Y_h})$$ | 3.6668 | 3.5501 | 3.8529 |

Bonferroni | $$70.155 \pm 9.742$$ | $$76.037 \pm 9.432$$ | $$83.390 \pm 10.237$$ |

Scheffe | $$70.155 \pm 11.355$$ | $$76.037 \pm 10.994$$ | $$83.390 \pm 11.932$$ |

## Contributors

- Anirudh Kandada(UCD)