10.3 Hybrid algorithms

Hybrid algorithms combine machine learning's flexibility with the statistical rigor needed for valid causal inference. They let you use powerful predictive models (random forests, neural nets, ensemble learners) to estimate nuisance parameters while still getting unbiased causal effect estimates with proper confidence intervals. The three workhorses are targeted maximum likelihood estimation (TMLE), augmented inverse probability weighting (AIPW), and double machine learning (DML).

Hybrid algorithms overview

The core problem these methods solve: traditional causal inference estimators (like plain IPW or outcome regression) require you to correctly specify parametric models. Get the model wrong, and your estimates are biased. But if you just plug in a machine learning model instead, you lose the ability to do valid statistical inference because ML introduces its own biases (regularization bias, overfitting).

Hybrid algorithms resolve this tension through two key ideas:

• Double robustness: The estimator remains consistent if either the propensity score model or the outcome regression model is correctly specified (not necessarily both).
• Cross-fitting: Nuisance parameters are estimated on separate data folds from those used to construct the final estimator, preventing overfitting bias from contaminating inference.

All three methods build on the efficient influence function (EIF), which provides the theoretical backbone for achieving optimal efficiency and valid confidence intervals.

Targeted maximum likelihood estimation (TMLE)

TMLE procedure

TMLE starts with initial estimates of the outcome regression and propensity score, then updates those estimates in a targeted way so the final estimator solves the efficient influence function equation. This targeting step is what distinguishes TMLE from simply plugging ML predictions into a formula.

The procedure works as follows:

1. Estimate the outcome regression $\hat{m}(A, X) = \hat{E}[Y \mid A, X]$ using a flexible ML method (e.g., Super Learner, an ensemble of multiple algorithms).

2. Estimate the propensity score $\hat{e}(X) = \hat{P}(A=1 \mid X)$ using another ML method.

3. Define a fluctuation submodel that tilts the initial outcome regression estimate along the direction of the EIF. This submodel is parameterized by a scalar $\epsilon$ and typically uses the "clever covariate" $H(A,X) = \frac{A}{\hat{e}(X)} - \frac{1-A}{1-\hat{e}(X)}$ as an offset in a logistic regression.

4. Fit the fluctuation parameter $\epsilon$ by maximizing the targeted likelihood (e.g., running a simple logistic regression of $Y$ on $H(A,X)$ with $\hat{m}(A,X)$ as an offset).

5. Update the initial estimate: $\hat{m}^*(A,X) = \text{expit}(\text{logit}(\hat{m}(A,X)) + \hat{\epsilon} \cdot H(A,X))$.

6. Compute the plug-in estimate: $\hat{\psi}_{TMLE} = \frac{1}{n}\sum_{i=1}^n [\hat{m}^*(1, X_i) - \hat{m}^*(0, X_i)]$ for the ATE.

The update step ensures the estimator solves the EIF estimating equation, which is what gives TMLE its double robustness and efficiency properties.

TMLE for causal effect estimation

TMLE can target different causal parameters by changing the EIF used in the fluctuation step:

• Average treatment effect (ATE): $\psi = E[Y(1) - Y(0)]$
• Average treatment effect on the treated (ATT): $\psi = E[Y(1) - Y(0) \mid A=1]$
• Conditional average treatment effect (CATE): $\psi(x) = E[Y(1) - Y(0) \mid X=x]$

A common choice for the initial ML estimator is Super Learner, which builds a weighted ensemble of candidate algorithms (GLMs, random forests, gradient boosting, etc.) using cross-validated risk to select the optimal combination. This reduces the risk of choosing a single poorly-specified model.

TMLE vs traditional methods

Augmented inverse probability weighting (AIPW)

AIPW estimator

AIPW combines IPW and outcome regression into a single estimator that is doubly robust. For estimating the ATE, the AIPW estimator for $E[Y(1)]$ is:

$\hat{\psi}_{AIPW} = \frac{1}{n} \sum_{i=1}^n \left(\frac{A_i Y_i}{\hat{e}(X_i)} - \frac{A_i - \hat{e}(X_i)}{\hat{e}(X_i)} \hat{m}(1, X_i)\right)$

where $\hat{e}(X_i)$ is the estimated propensity score and $\hat{m}(1, X_i)$ is the estimated outcome under treatment.

The intuition: start with the IPW estimator, then add an augmentation term that corrects for errors in the propensity score model using the outcome regression. If the propensity score is correct, the augmentation term has expectation zero and doesn't hurt. If the propensity score is wrong but the outcome model is right, the augmentation term corrects the bias. This is where double robustness comes from.

An analogous expression is constructed for $E[Y(0)]$, and the ATE estimate is the difference.

AIPW for missing data problems

AIPW extends naturally to missing data under the missing at random (MAR) assumption. The logic is the same: replace the treatment indicator $A$ with a missingness indicator $R$, the propensity score with the probability of being observed, and the outcome regression with the expected outcome given covariates.

• Estimate $P(R=1 \mid X)$, the probability of observing the outcome.
• Estimate $E[Y \mid X, R=1]$, the outcome regression among observed cases.
• The AIPW estimator reweights observed outcomes by the inverse probability of being observed and augments with the predicted outcome for everyone.

This gives consistent estimates as long as either the missingness model or the outcome model is correct.

AIPW vs IPW and outcome regression

• IPW is consistent only if the propensity score is correctly specified. It can also be highly variable when propensity scores are near 0 or 1 (positivity violations).
• Outcome regression is consistent only if the outcome model is correctly specified.
• AIPW is consistent if either model is correct, and it achieves the semiparametric efficiency bound when both are correct.

AIPW also tends to be more stable than IPW in practice because the augmentation term reduces variance, especially when propensity scores are extreme.

Efficient influence functions (EIF)

EIF definition and properties

The efficient influence function (EIF) is a central object in semiparametric efficiency theory. It characterizes the best possible estimator for a given target parameter in a nonparametric or semiparametric model.

The EIF has several defining properties:

• It is a mean-zero function of the observed data: $E[\text{EIF}(O; \psi_0, \eta_0)] = 0$, where $\psi_0$ is the true parameter and $\eta_0$ are the true nuisance parameters.
• It is the pathwise derivative of the target parameter functional, meaning it captures how the parameter changes under small perturbations of the data distribution.
• Its variance equals the semiparametric efficiency bound: no regular asymptotically linear (RAL) estimator can have smaller asymptotic variance than $\text{Var}(\text{EIF})/ n$.

For the ATE, the EIF takes the form:

$\text{EIF}(O; \psi, \eta) = \frac{A(Y - m(1,X))}{e(X)} - \frac{(1-A)(Y - m(0,X))}{1-e(X)} + m(1,X) - m(0,X) - \psi$

This expression shows exactly how each component (propensity score, outcome regression, observed data) contributes to the efficient estimator.

EIF in TMLE and AIPW

Both TMLE and AIPW are constructed so that they solve the EIF estimating equation:

$\frac{1}{n}\sum_{i=1}^n \widehat{\text{EIF}}(O_i; \hat{\psi}, \hat{\eta}) \approx 0$

• In TMLE, the targeting/fluctuation step forces this equation to hold exactly. The clever covariate in the fluctuation submodel is derived directly from the EIF.
• In AIPW, the estimator is literally the sample mean of the EIF evaluated at estimated nuisance parameters: $\hat{\psi}_{AIPW} = \psi_{\text{init}} + \frac{1}{n}\sum_i \widehat{\text{EIF}}(O_i)$.

Because both methods solve the same estimating equation, they are asymptotically equivalent when both nuisance models are consistently estimated. They can differ in finite samples, though, because TMLE respects the bounds of the outcome (e.g., staying in [0,1] for binary outcomes) while AIPW does not.

EIF-based confidence intervals

Once you have an estimator $\hat{\psi}$ that solves the EIF equation, you can construct confidence intervals using the estimated EIF values as if they were i.i.d. observations:

$\hat{\psi} \pm z_{\alpha/2} \sqrt{\frac{1}{n^2} \sum_{i=1}^n \widehat{\text{EIF}}(O_i; \hat{\psi}, \hat{\eta})^2}$

where $z_{\alpha/2}$ is the $1-\alpha/2$ quantile of the standard normal distribution.

These confidence intervals have correct asymptotic coverage as long as:

• The estimator is $\sqrt{n}$-consistent and asymptotically normal.
• The nuisance parameter estimators converge fast enough (faster than $n^{-1/4}$).
• The propensity score is bounded away from 0 and 1.

This is a major practical advantage: you get valid inference even though the nuisance parameters were estimated with black-box ML methods.

Double machine learning (DML)

DML framework

Double machine learning, introduced by Chernozhukov et al. (2018), provides a general framework for combining ML nuisance estimation with valid causal inference. The name "double" refers to two features: double robustness and the use of two stages of estimation.

The DML procedure:

1. Split the data into $K$ folds (typically $K = 5$ or $K = 10$).
2. For each fold $k$: use the remaining $K-1$ folds to train ML models for the nuisance parameters (propensity score $\hat{e}^{(-k)}$ and outcome regression $\hat{m}^{(-k)}$).
3. Predict on the held-out fold $k$ using these trained models.
4. Compute the EIF-based score for each observation in fold $k$ using the out-of-fold predictions.
5. Average the scores across all $n$ observations to get the final DML estimate.

The critical insight: by always predicting on data that was not used for training, cross-fitting prevents overfitting bias from leaking into the causal estimate. Without cross-fitting, regularization bias from ML methods (which converge slower than $n^{-1/2}$) would distort the asymptotic distribution of the estimator.

DML for treatment effect estimation

For the ATE, the DML estimator takes the form:

$\hat{\psi}_{DML} = \frac{1}{n} \sum_{i=1}^n \left[\hat{m}^{(-k_i)}(1, X_i) - \hat{m}^{(-k_i)}(0, X_i) + \frac{A_i(Y_i - \hat{m}^{(-k_i)}(1, X_i))}{\hat{e}^{(-k_i)}(X_i)} - \frac{(1-A_i)(Y_i - \hat{m}^{(-k_i)}(0, X_i))}{1-\hat{e}^{(-k_i)}(X_i)}\right]$

where $k_i$ denotes the fold that observation $i$ belongs to, and the $(-k_i)$ superscript means "trained without fold $k_i$."

DML estimators are $\sqrt{n}$-consistent and asymptotically normal under a key rate condition: the product of the convergence rates of the two nuisance estimators must be faster than $n^{-1/2}$. Concretely, if both the propensity score and outcome regression converge at rate $n^{-1/4}$ or faster, the DML estimator achieves $\sqrt{n}$-normality. This is a much weaker requirement than needing either model to converge at the parametric $n^{-1/2}$ rate.

DML vs traditional machine learning

Traditional ML optimizes for prediction accuracy (minimizing out-of-sample loss). DML optimizes for causal parameter estimation with valid inference. The differences matter:

• Prediction ML can be biased for causal effects due to regularization, confounding, and lack of double robustness. A well-tuned random forest predicts $Y$ accurately but doesn't tell you the causal effect of $A$.
• DML uses ML only for nuisance estimation, then constructs the causal estimate through the EIF. The Neyman orthogonality of the EIF makes the final estimate insensitive to small errors in the nuisance models.
• Inference: DML provides asymptotically valid confidence intervals and p-values. Standard ML does not.

Cross-fitting technique

Cross-fitting procedure

Cross-fitting is the sample-splitting strategy that makes hybrid algorithms work with ML. It's similar to cross-validation but serves a different purpose: cross-validation selects models, while cross-fitting prevents overfitting bias in causal estimation.

Step-by-step:

1. Randomly partition the data into $K$ equally-sized folds ($K = 5$ is common).
2. For fold $k = 1$: train the propensity score model and outcome regression model on folds 2 through $K$. Generate predicted values for observations in fold 1.
3. Repeat for each fold: train on all folds except $k$, predict on fold $k$.
4. Now every observation has out-of-fold predictions for both nuisance parameters.
5. Compute the EIF-based estimator using these out-of-fold predictions for all $n$ observations.

The result: each observation's nuisance parameter estimates come from a model that never saw that observation during training.

Cross-fitting in TMLE and DML

In DML, cross-fitting is built into the core procedure. The EIF scores are computed using out-of-fold nuisance estimates, and the final estimator averages these scores.

In TMLE, cross-fitting is used for the initial estimation step. The outcome regression and propensity score are estimated via cross-fitting, and then the targeting/fluctuation step can be performed using these out-of-fold estimates. The final TMLE estimate averages the targeted predictions across folds.

Both approaches achieve the same goal: ensuring that the ML-induced bias in nuisance estimation is orthogonal to the causal parameter of interest, so it vanishes in the asymptotic distribution.

Cross-fitting benefits and trade-offs

Benefits:

• Eliminates overfitting bias that would otherwise invalidate inference when using adaptive ML methods.
• Allows you to use the full sample for both nuisance estimation and causal effect estimation (unlike a simple train/test split, which wastes data).
• Enables the use of highly flexible ML methods (deep learners, boosted trees) without sacrificing inferential validity.

Trade-offs:

• Computational cost scales linearly with $K$, since you train nuisance models $K$ times.
• Each training set is only $(K-1)/K$ of the full sample, so nuisance estimates may be slightly less accurate than using all the data.
• The choice of $K$ involves a bias-variance trade-off: larger $K$ means larger training sets per fold (less bias in nuisance estimation) but more computation. In practice, $K = 5$ or $K = 10$ works well for most sample sizes.

Hybrid algorithms performance

Efficiency and robustness

The two defining properties of hybrid algorithms are:

• Semiparametric efficiency: When both nuisance models are consistently estimated, the estimator achieves the smallest possible asymptotic variance among all RAL estimators. This variance equals $\text{Var}(\text{EIF})/n$.
• Double robustness: The estimator remains $\sqrt{n}$-consistent if either the propensity score or the outcome regression (but not necessarily both) is consistently estimated.

These properties stem from the Neyman orthogonality of the EIF: the EIF score is locally insensitive to perturbations in the nuisance parameters around their true values. This is why small errors in ML-estimated nuisance parameters don't propagate into the causal estimate.

Finite sample properties

Asymptotic guarantees don't always translate directly to finite samples. Several factors affect real-world performance:

• Sample size: With small samples (say, $n < 500$), the $n^{-1/4}$ rate condition may not be effectively satisfied, and confidence interval coverage can be poor.
• Positivity violations: If some propensity scores are very close to 0 or 1, all three methods can become unstable. Trimming or truncating extreme propensity scores helps but introduces bias.
• ML model choice: The quality of the nuisance estimates matters. Poorly tuned ML models yield poor causal estimates even with double robustness, because double robustness protects against misspecification of one model, not both.
• Tuning and regularization: Overly aggressive regularization can slow nuisance convergence rates below the required $n^{-1/4}$ threshold.

Simulation studies and sensitivity analyses are essential for assessing performance in any specific application.

Asymptotic properties

Under regularity conditions, hybrid algorithms satisfy:

• $\sqrt{n}$-consistency: $\hat{\psi} - \psi_0 = O_p(n^{-1/2})$
• Asymptotic normality: $\sqrt{n}(\hat{\psi} - \psi_0) \xrightarrow{d} N(0, \text{Var}(\text{EIF}))$
• Semiparametric efficiency: The asymptotic variance equals the efficiency bound.

These results require:

• Nuisance estimators converge at rates such that the product of their errors is $o_p(n^{-1/2})$. The standard sufficient condition is that each converges at $o_p(n^{-1/4})$.
• The propensity score is bounded: $0 < \delta \leq e(X) \leq 1 - \delta < 1$ for some $\delta > 0$ (positivity).
• Standard regularity conditions (finite variance of the EIF, Donsker-type conditions or use of cross-fitting to avoid them).

Cross-fitting is particularly valuable because it relaxes the Donsker condition, which would otherwise restrict the complexity of the ML methods you can use.

Applications of hybrid algorithms

Observational studies

Observational studies are the primary use case for hybrid algorithms. Without randomization, treatment assignment depends on covariates, creating confounding. Hybrid algorithms address this by:

• Estimating the propensity score $P(A=1 \mid X)$ to model treatment assignment.
• Estimating the outcome regression $E[Y \mid A, X]$ to model the outcome mechanism.
• Combining both through the EIF to get a doubly robust causal estimate.

Classic applications include estimating the effect of job training programs on earnings (as in the LaLonde dataset), assessing medical treatment effectiveness from electronic health records, and evaluating policy interventions using administrative data. In all these settings, the true data-generating process is unknown, making double robustness especially valuable.

Randomized trials with non-compliance

Even in randomized experiments, non-compliance complicates causal inference. If some participants assigned to treatment don't actually take it (or vice versa), the intention-to-treat (ITT) estimate dilutes the true treatment effect.

Hybrid algorithms can estimate the complier average causal effect (CACE), the effect among those who would comply with their assignment, by:

• Using randomized assignment $Z$ as an instrumental variable.
• Estimating the compliance model and outcome model with ML.
• Applying TMLE or DML adapted for IV estimation.

This approach has been used in studies of school voucher programs, medication adherence trials, and behavioral interventions where dropout is common.

Longitudinal studies

Hybrid algorithms extend to longitudinal settings where treatments, confounders, and outcomes evolve over time. Time-varying confounding is the central challenge: a variable at time $t$ can be both a confounder for future treatment and a mediator of past treatment.

Adapted methods include:

• Longitudinal TMLE (LTMLE): Sequentially targets the outcome regression at each time point, working backward from the final outcome.
• Longitudinal IPW / marginal structural models: Weight each person-time observation by the cumulative product of inverse probability weights across time points.
• Sequential DML: Applies the DML framework to sequential decision problems.

These methods handle time-varying confounding and informative censoring (when dropout depends on covariates or treatment history) under sequential versions of the no unmeasured confounders assumption.