13.1 Convergence Concepts: In Probability and Distribution

Convergence in probability and distribution are key concepts in statistical inference. They help us understand how random variables behave as sample sizes grow, allowing us to make predictions and draw conclusions from data.

These concepts form the foundation for important statistical tools like the Law of Large Numbers and Central Limit Theorem. Understanding convergence types and their applications is crucial for analyzing estimators, conducting hypothesis tests, and building statistical models.

Convergence in Probability and Distribution

Convergence types in probability

• Convergence in probability
  • $X_n \xrightarrow{P} X$ when $\lim_{n \to \infty} P(|X_n - X| > \epsilon) = 0$ for any $\epsilon > 0$
  • Measures how likely $X_n$ and $X$ are close as $n$ increases (coin flips approaching theoretical probability)
• Convergence in distribution
  • $X_n \xrightarrow{d} X$ when $\lim_{n \to \infty} F_{X_n}(x) = F_X(x)$ at $F_X(x)$ continuity points
  • Focuses on cumulative distribution functions' limiting behavior (sample means approaching normal distribution)
• Key differences
  • Convergence in probability stronger than convergence in distribution
  • Convergence in probability implies convergence in distribution, not vice versa
  • Convergence in distribution requires only limiting distribution function behavior similarity

Proofs of probabilistic convergence

• Proving convergence in probability
  • Markov's inequality bounds probability of large deviations
  • Chebyshev's inequality uses variance to bound probability
  • Demonstrate $\lim_{n \to \infty} P(|X_n - X| > \epsilon) = 0$ for any $\epsilon > 0$
• Proving convergence in distribution
  • Characteristic functions: $\lim_{n \to \infty} \phi_{X_n}(t) = \phi_X(t)$ for all $t$ implies $X_n \xrightarrow{d} X$
  • Lévy's continuity theorem links characteristic function convergence to distribution convergence
  • Portmanteau theorem provides equivalent conditions for convergence in distribution
• Probability measure properties
  • Countable additivity ensures probability of disjoint events sum
  • Non-negativity keeps probabilities positive
  • Normalization maintains total probability as 1

Applications of convergence concepts

• Law of Large Numbers
  • Weak LLN: Sample mean converges in probability to population mean
  • Strong LLN: Sample mean almost surely converges to population mean
• Central Limit Theorem
  • Standardized sample mean converges in distribution to standard normal
  • Applies to independent, identically distributed variables with finite variance
• Slutsky's theorem
  • Combines convergence in distribution and probability
  • Useful for analyzing functions of converging sequences (sample variance)
• Continuous Mapping Theorem
  • Preserves convergence in distribution under continuous transformations
  • Helpful in deriving limiting distributions of functions (log-likelihood ratios)

Convergence effects on statistical measures

• Consistency of estimators
  • Weak consistency: Estimator converges in probability to true parameter
  • Strong consistency: Estimator almost surely converges to true parameter
• Asymptotic normality
  • Standardized estimator converges in distribution to normal
  • Enables confidence interval construction and hypothesis testing (t-tests)
• Asymptotic efficiency
  • Estimator variance converges to Cramér-Rao lower bound
  • Measures estimator optimality in large samples (maximum likelihood estimators)
• Robustness
  • Convergence concepts assess estimator and test statistic behavior under assumption violations
  • Important for real-world applications with imperfect data (outlier effects)
• Large sample properties
  • Maximum likelihood estimators show consistency and asymptotic normality
  • Likelihood ratio test statistics have known asymptotic distributions (chi-square tests)