Skip to main content
Statistics LibreTexts

Simultaneous Inference

  • Page ID
    240
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)

    ( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\id}{\mathrm{id}}\)

    \( \newcommand{\Span}{\mathrm{span}}\)

    \( \newcommand{\kernel}{\mathrm{null}\,}\)

    \( \newcommand{\range}{\mathrm{range}\,}\)

    \( \newcommand{\RealPart}{\mathrm{Re}}\)

    \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)

    \( \newcommand{\Argument}{\mathrm{Arg}}\)

    \( \newcommand{\norm}[1]{\| #1 \|}\)

    \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)

    \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    \( \newcommand{\vectorA}[1]{\vec{#1}}      % arrow\)

    \( \newcommand{\vectorAt}[1]{\vec{\text{#1}}}      % arrow\)

    \( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vectorC}[1]{\textbf{#1}} \)

    \( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)

    \( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)

    \( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    \(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)

    We want to find confidence interval for more than one parameter simultaneously. For example we might want to find confidence interval for \(\beta_0 \) and \( \beta_1 \).

    Bonferroni Joint Confidence Intervals

    The confidence coefficients for individual parameters are adjusted to the higher 1 - \( \alpha \) so that the confidence coefficient for the collection of parameters must be at least 1 - \( \alpha \). This is based on the following inequality:

    Theorem (Bonferroni's Inequality)

    \[P( \beta_0\cap\beta_1)\geq1-P(\beta_0^c)-P(\beta_1^c) \label{Bonferroni}\]

    for any two events \(\beta_0 \) and \( \beta_1 \), where \(\beta_0^c \) and \(\beta_1^c \) are complements of events \( \beta_0 \) and \( \beta_1 \), respectively.

    We take, \( \beta_0 =\) the event that confidence interval for \(\beta_0 \) covers \(\beta_0 \); and, \( \beta_1 =\) the event that confidence interval for \( \beta_1 \) covers \( \beta_1 \);

    So, if \(P(\beta_0) = 1-\alpha_1 \), and \(P(\beta_1) = 1-\alpha_2 \), then \(P(\beta_0\cap\beta_1)\geq1-\alpha_1-\alpha_2 \), by Bonferroni's inequality (Equation \ref{Bonferroni}). Note that \(\beta_0\cap\beta_1\) is the event that confidence intervals for both the parameters cover the respective parameters. Therefore we take \(\alpha_1 = \alpha_2 = \alpha/2 \) to get joint confidence intervals with confidence coefficient at least \(1 - \alpha \),

    \(b_0 \pm t(1-\alpha/4;n-2) s(b_0) \) and \(b_1 \pm t(1-\alpha/4;n-2) s(b_1) \) for \(\beta_0\) and \(\beta_1\), respectively.

    Bonferroni Joint Confidence Intervals for Mean Response

    We want to find the simultaneous confidence interval for \(E(Y|X = X_h) = \beta_0 + \beta_1X_h\) for g different values of \(X_h\). Using Bonferroni's inequality for the intersection of g different events, the confidence intervals with confidence coefficient (at least) \(1-\alpha\) are given by

    \[\widehat{Y_h} \pm t(1-\alpha/2g; n-2)s(\widehat{Y_h}).\]

    Confidence band for regression line : Working-Hotelling procedure

    The confidence band

    \[\widehat{Y_h} \pm\sqrt{2F(1-\alpha;2,n-2)}s(\widehat{Y_h})\]

    contains the entire regression line (for all values of \(X\)) with confidence level \(1-\alpha\). The Working-Hotelling procedure for obtaining the \(1-\alpha\) simultaneous confidence band for the mean responses, therefore, is to use these confidence limits for the g different values of \(X_h\).

    Simultaneous prediction intervals

    Recall that, the standard error of prediction for a new observation \(Y_{h(new)}\) with \(X = X_h\), is $$s(Y_{h(new)}-\widehat{Y_h}) = \sqrt{MSE(1+\frac{1}{n}+\frac{(X_h-\overline{X})^2}{\sum_i(X_i-\overline{X})^2})}$$ In order to predict the new observations for g different values of X, we may use one of the two procedures:

    • Bonferroni procedure : $$\widehat{Y_h}\pm t(1-\alpha/2g;n-2)s(Y_{h(new)}-\widehat{Y_h})$$.
    • Scheffe procedure : $$\widehat{Y_h}\pm \sqrt{gF(1-\alpha;g,n-2)}s(Y_{h(new)}-\widehat{Y_h})$$.

    Remark : A point to note is that except for the Working-Hotelling procedure for finding simultaneous confidence intervals for mean response, in all the other cases, the confidence intervals become wider as g increases.

    Which method to choose : Choose the method which leads to narrower intervals. As a comparison between Bonferroni and Working-Hotelling (for finding confidence intervals for the mean response), the following can be said :

    • If g is small, Bonferroni is better.
    • If g is large, Working-Hotelling is better (the coefficient of \(s(\widehat{Y_h})\) in the confidence limits remains the same even as g becomes large).

    Housing data as an example

    Fitted regression model : \(\widehat{Y_h} = 28.981 + 2.941X, n=19, s(b_0) = 8.5438, s(b_1) = 0.5412, MSE = 11.9512.\)

    • Simultaneous confidence intervals for \(\beta_0\) and \(\beta_1\) : For 95% simultaneous C.I., \(t(1-\alpha/4;n-2)=t(0.9875;17) = 2.4581\). The intervals are (for \(\beta_0 and \beta_1\), respectively)

    \(28.981 \pm 2.4581 \times 8.5438 \equiv 28.981 \pm 21.002, 2.941 \pm 2.4581 \times 0.5412 \equiv 2.941 \pm 1.330\)

    • Simultaneous inference for mean response at g different values of X : Say g = 3.

    And the values are

    $$X_h$$ 14 16 18.5
    $$\widehat{Y_h}$$ 70.155 76.037 83.390
    $$s(\widehat{Y_h})$$ 1.2225 0.8075 1.7011

    \(t(1-0.05/2g;n-2) = t(0.99167;17) = 2.655, \sqrt{2F(0.95;2,n-2)} = \sqrt{2 \times 3.5915} = 2.6801 \)

    The 95% simultaneous confidence intervals for the mean responses are given in the following table:

    $$X_h$$ 14 16 18.5
    Bonferroni $$70.155 \pm 3.248$$ $$76.037 \pm 2.145$$ $$83.390 \pm 4.520$$
    Working-Hotelling $$70.155 \pm 3.276$$ $$76.037 \pm 2.164$$ $$83.390 \pm 4.559$$
    • Simultaneous prediction intervals for g different values of X : Again, say g = 3 and the values of 14,16 and 18.5. In this case, \(\alpha = 0.05, t(1-\alpha/2g;n-2) = t(0.99167; 17) = 2.655\). And \(\sqrt{gF(1-\alpha;g,n-2)} = \sqrt{3F(0.95;3,17)} = \sqrt{3 \times 3.1968} = 3.0968\). The standard errors and simultaneous 95% C.I. are given in the following table:
    $$X_h$$ 14 16 18.5
    $$\widehat{Y_h}$$ 70.155 76.037 83.390
    $$s(Y_{h(new)}-\widehat{Y_h})$$ 3.6668 3.5501 3.8529
    Bonferroni $$70.155 \pm 9.742$$ $$76.037 \pm 9.432$$ $$83.390 \pm 10.237$$
    Scheffe $$70.155 \pm 11.355$$ $$76.037 \pm 10.994$$ $$83.390 \pm 11.932$$

    Contributors

    • Anirudh Kandada(UCD)

    This page titled Simultaneous Inference is shared under a not declared license and was authored, remixed, and/or curated by Debashis Paul.