12.2 Consistent Estimators and Asymptotic Normality

Consistency and asymptotic normality are key concepts in statistical estimation. They describe how estimators behave as sample sizes grow, with consistency ensuring convergence to true values and asymptotic normality enabling inference through normal approximations.

These properties are crucial for understanding estimator behavior in large samples. By proving consistency and asymptotic normality, we can construct confidence intervals, perform hypothesis tests, and compare estimators, forming the foundation for many statistical techniques.

Consistency and Asymptotic Normality of Estimators

Consistency and asymptotic normality

• Consistency
  • Estimator converges in probability to true parameter value as sample size increases
  • $\lim_{n \to \infty} P(|\hat{\theta}_n - \theta| < \epsilon) = 1$ for any $\epsilon > 0$
  • Weak consistency converges in probability, strong consistency converges almost surely
• Asymptotic normality
  • Estimator approaches normal distribution as sample size increases
  • $\sqrt{n}(\hat{\theta}_n - \theta) \xrightarrow{d} N(0, \sigma^2)$
  • Enables construction of confidence intervals and hypothesis tests (t-tests, z-tests)

Proving estimator consistency

• Probability limit (plim)

  • Convergence in probability concept
  • $plim_{n \to \infty} X_n = X$ if $\lim_{n \to \infty} P(|X_n - X| < \epsilon) = 1$ for any $\epsilon > 0$

• Consistency proof steps

  1. Express estimator using sample statistics (sample mean, variance)
  2. Apply law of large numbers to sample statistics
  3. Use plim properties (linearity, continuity) to simplify
  4. Show resulting expression equals true parameter value

• Techniques

  • Slutsky's theorem combines convergence of multiple random variables
  • Continuous mapping theorem applies continuous functions to convergent sequences

Asymptotic normality via CLT

• Central Limit Theorem (CLT)

  • For i.i.d. random variables, sample mean approaches normal distribution
  • Requires finite mean and variance

• Demonstrating asymptotic normality

  1. Express estimator as function of sample means
  2. Apply CLT to sample means
  3. Use delta method for non-linear functions
  4. Determine asymptotic variance

• Asymptotic variance

  • Calculated using influence function or observed information matrix
  • Measures estimator precision, smaller values indicate higher efficiency

Applications in inference problems

• Confidence intervals
  • Asymptotic normality creates approximate intervals
  • Wald intervals use estimated standard errors
• Hypothesis testing
  • Wald test compares estimate to null hypothesis
  • Likelihood ratio test compares maximized likelihoods
  • Score test uses slope of log-likelihood at null value
• Estimator comparison
  • Asymptotic relative efficiency compares variances
  • Cramer-Rao lower bound sets minimum variance for unbiased estimators
• Large sample properties
  • Maximum likelihood estimators often consistent and asymptotically normal
  • Asymptotic efficiency achieved under regularity conditions
• Practical considerations
  • Sample size affects asymptotic property validity
  • Robustness varies with assumption violations (normality, independence)