Experimental Design and Introduction to Analysis of Variance (LN 3)
( \newcommand{\kernel}{\mathrm{null}\,}\)
An overview of experimental designs
1. Complete randomized design (CRD): treatments (combinations of the factor levels of the different factors) are randomly assigned to the experimental units.Examples: one factor, two-factor, multi-factor studies (factorial designs).
Example: in a study involving two factors -- temperature (at three levels) and concentration of a solvent (at two levels) on the yield of a chemical process, all combinations of temperature and solvent concentrations are considered.
Solvent conc. | Temperature | ||
low | medium | high | |
low | x | x | x |
high | x | x | x |
Table 1: Chemical yield study: Crossed factor design
3. Nested design: one factor is nested within another factor in a multi-factor study.
Plant | Operator | |||||
1 | 2 | 3 | 4 | 5 | 6 | |
1 | x | x | ||||
2 | x | x | ||||
3 | x | x |
Table 2: Production study: Nested design
4. Repeated measurement design: the same experimental unit receives all the treatment combinations. This helps to eliminate the effects of confounding factors associated with the experimental units.
5. Randomized complete block design (RCBD): every treatment appears with every other treatment in the same block the same number of times and every block receives the full suit of treatments (possibly with replications), and the treatments are randomly assigned to the experimental units within each block. Used when the block sizes are integer multiples of the number of treatments.
An overview of observational studies
1. Cross sectional studies : measurements are taken from one or more populations or subpopulations at a single time point or time interval; and exposure to a potential causal factor and the response are determined simultaneously.Example: A study of incomes by gender in the decade 1981-1990, stratified by geographical locations.
2. Cohort study (prospective): one or more groups are formed in a nonrandom manner according to some hypothesized causal factors, and then these groups are observed over time with respect to the outcome of interest. (What is going to happen ?) Example: A group of smokers and a group of non-smokers were followed since their early 30's, and their health indices were recorded over years until their late 60's.
3. Case control study (retrospective study): groups are defined on the basis of an observed outcome, and the differences among the groups at an earlier time point are identified as potential causal effects. (What has happened?) Example: study of lung cancer (outcome) and smoking (explanatory factor) based on data collected on lung cancer patients.
A single factor study
A food company wanted to test four different package designs for a new break-fast cereal. 20 stores with approximately the same sales condition (such as sales volume, price, etc) were selected as experimental units. Five stores were randomly assigned to each of the 4 package designs.- A balanced complete randomized design.
- A single, 4-level, qualitative factor: package design;
- A quantitative response variable: sales -- number of packets of cereal sold during the period of study;
- Goal: exploring relationship between package design and sales.
Data
The assignment of the different stores (indicated by letters A to T) to the package designs (D1 to D4) is given in the following table.Store IDs | ||||||
S1 | S2 | S3 | S4 | S5 | ||
D1 | A | B | C | D | E | |
Design | D2 | F | G | H | I | J |
D3 | K | L | M | N | O | |
D4 | P | Q | R | S | T |
The observed data on sales in the following table. Store O was dropped from the study because of a fire. As a result of this, the design is not balanced anymore.
Store IDs | ||||||
S1 | S2 | S3 | S4 | S5 | ||
D1 | 11 | 17 | 16 | 14 | 15 | |
Design | D2 | 12 | 10 | 15 | 19 | 11 |
D3 | 23 | 20 | 18 | 17 | Miss | |
D4 | 27 | 33 | 22 | 26 | 28 |
ANOVA for single factor study
A simple statistical model the data is as follows: Yij=μi+εij,j=1,…,ni; i=1,…,r; where:- r is the number of factor levels (treatments) and ni is the number of experimental units corresponding to the i-th factor level;
- Yij is the measurement for the j-th experimental unit corresponding to the i-th factor level;
- μi is the mean of all the measurements corresponding to the i-th factor level (unknown);
- εij's are random errors (unobserved).
Model Assumptions
The following assumptions are made about the previous model:- εij are independently and identically distributed as N(0,σ2).
- μi 's are unknown fixed parameters (so called fixed effects), so that E(Yij)=μi and Var (Yij)=σ2 . The above assumption is thus equivalent to assuming that Yij are independently distributed as N(μi,σ2).
Interpretations
- Factor level means (μi): in an experimental study, the factor level mean μi stands for the mean response that would be obtained if the i-th factor level were applied to the entire population from where the experimental units were sampled
- Residual variance (σ2) : refers to the variability among the responses if any given treatment were applied to the entire population.
Steps in the anyalysis of factor level means
- Determine whether or not the factor level means are all the same: μ1=⋯=μr
- What does μ1=⋯=μr mean? The factor has no effect on the distribution of the response variable.
- How to evaluate the evidence of this statement μ1=⋯=μr based on observed data ?
2. If the factor level means do differ, examine how they differ and what are the implications of these differences? (Chapter 17)
In the example, we want to answer whether there is any effect of package design on sales. First step is the obtain estimates of the factor level means.Estimation of μi
Define, the sample mean for the i-th factor level: ¯Yi⋅=1nini∑j=1Yij=1niYi⋅ where Yi⋅=∑nij=1Yij is the sum of responses for the i-th treatment group, for i=1,…,r; and the overall sample mean: ¯Y⋅⋅=1∑ri=1nir∑i=1ni∑j=1Yij=1∑ri=1nir∑i=1ni¯Yi⋅=1∑ri=1niY⋅⋅ . Then ¯Yi⋅ is an estimate of μi for each i=1,…,I. Under the assumptions, ¯Yi⋅ is an unbiased estimator of μi since E(¯Yi⋅)=1nini∑j=1E(Yij)=1nini∑j=1μi=μi.Store IDs | Total | Mean | ni | ||||||
S1 | S2 | S3 | S4 | S5 | ( Yi. ) | ( ¯Yi. | |||
D1 | 11 | 17 | 16 | 14 | 15 | 73 | 14.6 | 5 | |
Design | D2 | 12 | 10 | 15 | 19 | 11 | 67 | 13.4 | 5 |
D3 | 23 | 20 | 18 | 17 | Miss | 78 | 19.5 | 4 | |
D4 | 27 | 33 | 22 | 26 | 28 | 136 | 27.2 | 5 | |
Total | Y..=354 | ¯Y..=18.63 | 19 |
It is easy to see that Designs 3 and 4 lead to larger sales than Designs 1 and 2. How to quantify these differences? How large is large?
Pairwise comparison of factor level means
Suppose we want to compare Designs 1 and 2. We can formulate this as a hypothesis testing problem for the following hypothesis: H0:μ1=μ2 against Ha:μ1≠μ2. The standard test procedure is the two-sample z -test described below (assuming for the time being that σ is known).- Null hypothesis H0:μ1=μ2 tested against alternative hypothesis Ha:μ1≠μ2.
- The test procedure essentially asks the following question: is the observed difference ¯Y1⋅−¯Y2⋅ large enough to support the hypothesis Ha:μ1≠μ2 ?
- The z-test statistic for H0:μ1=μ2 vs. Ha:μ1≠μ2 is
Contributors:
- Valerie Regalia
- Debashis Paul