Sensitivity Analysis Methods

Why This Matters

In causal inference, you're rarely working with perfect data—unmeasured confounding lurks behind nearly every observational study, threatening to invalidate your conclusions. Sensitivity analysis methods give you the tools to ask the critical question: How wrong could I be? Rather than pretending confounders don't exist, these techniques let you quantify exactly how strong an unmeasured variable would need to be to overturn your findings. You're being tested on your ability to defend causal claims under uncertainty, and that means knowing when your estimates are robust versus when they're hanging by a thread.

These methods connect directly to core causal inference principles: the ignorability assumption, selection bias, instrumental variable validity, and mediation pathways. Each sensitivity analysis approach addresses a specific vulnerability in your causal argument. Don't just memorize what each method does—understand which assumption it stress-tests and what kind of study design it applies to. When an exam question asks you to evaluate the credibility of a causal claim, your answer should include how you'd probe its weaknesses.

---

Quantifying Confounder Strength

These methods answer a direct question: How strong would an unmeasured confounder need to be to explain away my results? They translate abstract concerns about bias into concrete, interpretable quantities.

E-Value Method

• Quantifies the minimum confounder strength needed to reduce an observed effect to null—expressed as the required associations between confounder, treatment, and outcome
• Calculated from the observed risk ratio using the formula $E = RR + \sqrt{RR \times (RR - 1)}$, making it easy to compute and communicate
• Particularly valuable for policy discussions because it shifts the burden: critics must argue a confounder that strong plausibly exists

Cornfield Conditions

• Establishes necessary inequalities that any confounder must satisfy to fully explain an observed association—originally developed in smoking-lung cancer debates
• Focuses on the confounder's prevalence and its strength of association with both treatment and outcome simultaneously
• Provides a logical framework for dismissing implausible confounding explanations rather than just computing a single number

Ding and VanderWeele's Bounding Factor

• Extends E-value logic by providing tighter bounds when you have partial information about potential confounders
• Allows incorporation of auxiliary data about confounder-treatment or confounder-outcome relationships from external sources
• Produces a bounding factor that directly multiplies your confidence interval, showing how much uncertainty unmeasured confounding adds

---

Bounds and Thresholds for Matched Designs

When you've used matching or stratification to control observed confounders, these methods ask: What if I missed something? They're designed specifically for studies where you've already done the work of balancing covariates.

Rosenbaum Bounds

• Parameterizes hidden bias with $\Gamma$—the odds ratio of differential treatment assignment between matched units due to unobserved factors
• Tests whether significance holds as you increase $\Gamma$ from 1 (no hidden bias) upward, revealing the "breaking point" of your conclusions
• Standard tool for matched observational studies because it directly addresses the residual selection bias that matching cannot eliminate

Tipping Point Analysis

• Identifies the exact threshold where an unmeasured confounder would flip your result from significant to null (or reverse direction)
• Presents results as a two-dimensional plot showing combinations of confounder prevalence and effect size that would overturn findings
• Guides future research priorities by showing exactly what kind of confounder would matter—if it seems implausible, your result is robust

---

Design-Specific Sensitivity Methods

Different causal identification strategies have different vulnerabilities. These methods stress-test the specific assumptions that make each design work.

Instrumental Variable Sensitivity Analysis

• Probes violations of the exclusion restriction—what if the instrument affects the outcome through paths other than treatment?
• Quantifies how much direct effect the instrument would need to have on the outcome to invalidate your IV estimate
• Essential for weak instruments where even small violations of assumptions can produce large biases in the two-stage least squares estimator

Regression Discontinuity Sensitivity Analysis

• Tests robustness to bandwidth choice and functional form assumptions near the cutoff—results shouldn't change dramatically with reasonable alternatives
• Examines manipulation of the running variable by checking for suspicious bunching just above or below the threshold
• Validates the continuity assumption by testing whether other covariates show jumps at the cutoff (they shouldn't if the design is valid)

Mediation Sensitivity Analysis

• Addresses unmeasured mediator-outcome confounding—the Achilles' heel of mediation analysis that randomizing treatment doesn't solve
• Uses sensitivity parameters $\rho$ to represent the correlation between errors in mediator and outcome models
• Determines whether direct and indirect effects remain significant across plausible ranges of unmeasured confounding strength

---

Comprehensive Bias Modeling

Sometimes you face multiple potential biases simultaneously—selection, measurement error, unmeasured confounding. These methods handle the messy reality of real-world data.

Multiple Bias Modeling

• Simultaneously models several bias sources including confounding, selection bias, and misclassification in a single analytical framework
• Reveals how biases can compound or cancel—two moderate biases might produce a larger net effect than one strong bias alone
• Requires specifying bias parameters for each source, making assumptions transparent and open to critique

Probabilistic Bias Analysis

• Replaces point estimates of bias with probability distributions—acknowledging that you don't know the exact magnitude of confounding
• Incorporates prior knowledge or expert elicitation to specify plausible ranges for bias parameters rather than arbitrary single values
• Produces a distribution of corrected effect estimates showing the full range of conclusions consistent with your uncertainty about bias

---

Quick Reference Table

---

Self-Check Questions

1. You've conducted a matched observational study and found a significant treatment effect. Which sensitivity analysis method would you use to determine how much hidden bias could exist before your result becomes non-significant, and what parameter would you report?

2. Compare the E-value method and Rosenbaum bounds: What type of study is each best suited for, and what does each method's output tell you about unmeasured confounding?

3. A researcher using instrumental variables is worried that their instrument might have a small direct effect on the outcome. Which sensitivity analysis approach addresses this concern, and what assumption is being tested?

4. How does probabilistic bias analysis differ from traditional sensitivity analysis that uses fixed bias parameters? When would you prefer one approach over the other?

5. FRQ-style: You're reviewing a mediation analysis claiming that a job training program improves earnings primarily through increased self-efficacy. What specific unmeasured confounding threat does mediation sensitivity analysis address, and why can't randomization of treatment alone solve this problem?