13.3 Delta Method and Asymptotic Distributions

The delta method is a powerful tool for approximating distributions of functions of random variables. It uses Taylor series expansion to simplify complex statistical problems, making it easier to work with transformed parameters and conduct inference.

This method is particularly useful when dealing with maximum likelihood estimators. By applying the delta method to MLEs, we can derive asymptotic distributions for transformed parameters, enabling us to construct confidence intervals and perform hypothesis tests on various statistical quantities.

Delta Method and Asymptotic Distributions

Concept of delta method

• Approximates distribution of function of random variables using Taylor series expansion
• Applies to differentiable functions and random variables with known asymptotic distributions
• Utilizes original estimator, applied function, and first-order Taylor expansion
• Results in asymptotic normality for asymptotically normal estimators
• Approximates variance using squared derivative and original estimator variance

Application of delta method

• Identify function $g(\theta)$ and estimator $\hat{\theta}$
• Determine $\hat{\theta}$ asymptotic distribution
• Calculate derivative $g'(\theta)$
• Apply formula: $g(\hat{\theta}) \sim N(g(\theta), [g'(\theta)]^2 \sigma^2)$ for $\hat{\theta} \sim N(\theta, \sigma^2)$
• Multivariate case uses gradient vector and matrix multiplication
• Interpret mean and variance of resulting asymptotic distribution

Asymptotic distribution of MLE

• Maximum likelihood estimation finds parameters maximizing likelihood function
• MLE asymptotically efficient and normally distributed: $\hat{\theta}_{MLE} \sim N(\theta, I(\theta)^{-1})$
• $I(\theta)$ represents Fisher information
• Applying delta method to MLE: $g(\hat{\theta}_{MLE}) \sim N(g(\theta), [g'(\theta)]^2 I(\theta)^{-1})$
• Enables inference on transformed parameters (relative risk, odds ratio)

Inference with delta method

• Construct confidence intervals: $g(\hat{\theta}) \pm z_{\alpha/2} \sqrt{[g'(\theta)]^2 \hat{\sigma}^2}$
• Formulate null and alternative hypotheses for testing
• Calculate test statistic using asymptotic distribution
• Wald test statistic: $\frac{g(\hat{\theta}) - g(\theta_0)}{\sqrt{[g'(\hat{\theta})]^2 \hat{\sigma}^2}}$
• Likelihood ratio test alternative for small samples
• Apply to test functions of parameters (difference in means, ratio of variances)