Key Concepts in Design of Experiments

Why This Matters

Design of Experiments (DOE) isn't just a checklist of statistical techniques—it's the backbone of how engineers make decisions under uncertainty. When you're tested on DOE concepts, you're being asked to demonstrate that you understand why certain experimental structures produce valid, efficient results. The principles here connect directly to broader statistical reasoning: controlling variability, isolating effects, and building models that actually predict real-world behavior.

The concepts in this guide fall into three big categories: validity principles (how we ensure results are trustworthy), design structures (how we organize experiments efficiently), and analysis frameworks (how we interpret what we find). Don't just memorize definitions—know which principle solves which problem. When an exam question describes a confounded experiment or asks how to reduce error variance, you should immediately recognize which DOE concept applies.

---

Foundational Validity Principles

These three concepts form the bedrock of experimental design. They work together to ensure that your conclusions reflect real treatment effects rather than artifacts of poor experimental structure.

Randomization

• Eliminates systematic bias by assigning experimental units to treatments through a random mechanism—this is your first line of defense against hidden confounders
• Balances uncontrolled variables across treatment groups, ensuring that unknown factors don't systematically favor one treatment over another
• Validates statistical inference because most hypothesis tests assume random assignment; without it, your $p$-values don't mean what you think they mean

Replication

• Enables error estimation by providing multiple independent observations under the same conditions—you can't calculate $s^2$ with a single data point
• Increases statistical power by reducing the standard error of treatment means, making it easier to detect real effects
• Distinguishes signal from noise because repeated measurements reveal whether observed differences are consistent or just random fluctuation

Blocking

• Controls known nuisance variability by grouping similar experimental units together before randomizing treatments within each block
• Reduces error variance by removing block-to-block differences from the residual error term, effectively sharpening your statistical lens
• Improves precision without more runs because you're accounting for variability that would otherwise inflate your error estimate

---

Effect Types and Interpretation

Understanding what you're actually measuring is critical. Main effects and interactions tell different stories about how your system behaves.

Main Effects

• Quantify the average impact of changing a single factor from its low to high level, calculated as $\bar{y}_{high} - \bar{y}_{low}$
• Represent the simplest treatment effects and are typically the first thing you examine when analyzing factorial data
• Can be misleading in isolation if significant interactions exist—a main effect might average out to zero even when the factor matters enormously at certain conditions

Interaction Effects

• Capture non-additive behavior where the effect of one factor depends on the level of another factor—the whole is different from the sum of parts
• Appear graphically as non-parallel lines when you plot response vs. Factor A at different levels of Factor B
• Drive practical decisions because real engineering systems rarely behave additively; ignoring interactions leads to suboptimal solutions

Confounding

• Occurs when effects are aliased meaning two or more effects cannot be estimated separately because they share the same contrast
• Arises from fractional designs where you sacrifice information about higher-order effects to reduce experimental runs
• Requires careful design choices to ensure that main effects aren't confounded with two-factor interactions you care about

---

Design Structures for Efficiency

Different experimental goals call for different design architectures. Choosing the right structure balances information gained against resources spent.

Factorial Designs

• Test all combinations of factor levels, providing complete information about main effects and all interactions
• Require $2^k$ runs for $k$ factors at two levels each, which grows quickly but yields maximum information per run
• Enable effect estimation without confounding because orthogonality is built into the full factorial structure

Orthogonality

• Ensures independent effect estimates where changing one factor's level doesn't systematically change another's across the design
• Simplifies calculations because orthogonal designs produce uncorrelated parameter estimates with minimum variance
• Appears in balanced designs where each factor level appears equally often with every level of other factors

Screening Experiments

• Identify the vital few factors from a large initial set using minimal experimental resources—the Pareto principle applied to experimentation
• Employ fractional factorial structures like $2^{k-p}$ designs that test only a fraction of all possible combinations
• Accept some confounding of higher-order interactions in exchange for dramatic reductions in required runs

Plackett-Burman Designs

• Screen up to $n-1$ factors in $n$ runs where $n$ is a multiple of 4, offering exceptional efficiency for initial factor screening
• Confound main effects with two-factor interactions which is acceptable when you assume interactions are negligible
• Produce resolution III designs meaning main effects are clear of each other but aliased with two-factor interactions

---

Response Surface Methods

When your goal shifts from understanding to optimization, these techniques help you find the sweet spot. They model curvature in the response surface that factorial designs miss.

Response Surface Methodology

• Fits polynomial models typically second-order equations of the form $y = \beta_0 + \sum \beta_i x_i + \sum \beta_{ii} x_i^2 + \sum \beta_{ij} x_i x_j$
• Locates optimal operating conditions by finding the stationary point of the fitted surface through calculus or numerical methods
• Proceeds sequentially from screening to steepest ascent to final optimization around the optimum region

Central Composite Designs

• Combine factorial points with axial points at distance $\alpha$ from the center, plus center point replicates
• Enable estimation of quadratic terms allowing you to model curvature and find true optima rather than edge solutions
• Offer rotatability when $\alpha = 2^{k/4}$, meaning prediction variance depends only on distance from the center, not direction

Box-Behnken Designs

• Avoid extreme corners by placing points at midpoints of edges rather than vertices of the experimental cube
• Require fewer runs than CCD for three or more factors while still estimating all quadratic terms
• Work well when corners are infeasible such as when extreme factor combinations would damage equipment or violate constraints

---

Optimization Experiments

• Seek the best factor settings rather than just understanding factor effects—the goal is actionable improvement
• Build on screening results by focusing resources on the factors identified as most influential
• Use RSM techniques to navigate toward optimal conditions through sequential experimentation

---

Analysis Framework

Analysis of Variance (ANOVA)

• Partitions total variability into components: $SS_{total} = SS_{treatments} + SS_{blocks} + SS_{error}$ for a blocked design
• Tests significance via F-ratios comparing treatment mean squares to error mean squares: $F = MS_{treatment}/MS_{error}$
• Underlies all DOE analysis whether you're analyzing a simple one-factor experiment or a complex response surface model

---

Quick Reference Table

---

Self-Check Questions

1. A chemical engineer notices that temperature affects yield differently depending on pressure level. Which DOE concept describes this phenomenon, and how would it appear on an interaction plot?

2. Compare randomization and blocking: both reduce bias, but they address different types of confounding variables. When would you use each, and can they be used together?

3. You need to screen 15 potential factors with a limited budget. Which design structure would you choose, and what information would you sacrifice compared to a full factorial?

4. Explain why a Box-Behnken design might be preferred over a Central Composite Design when optimizing a chemical process where extreme temperature-pressure combinations could cause safety issues.

5. An ANOVA table shows a significant main effect for Factor A but also a significant A×B interaction. Why might reporting only the main effect be misleading, and what should you examine instead?