5.3 Efficiency and Mean Squared Error

Point estimation efficiency is all about finding the best way to guess a population's true value. It's like trying to hit a bullseye – the closer you get, the more efficient your estimate is.

Efficiency compares different estimation methods, looking at how spread out their guesses are. The method with less spread (lower variance) is more efficient. It's a balancing act between accuracy and precision in statistical guesswork.

Efficiency in Point Estimation

Concept of efficiency in estimation

• Efficiency in point estimation measures quality and precision of estimator
• Relative efficiency compares two estimators based on variances
• Fisher information quantifies parameter information inversely related to efficient estimator variance
• Cramér-Rao lower bound sets theoretical minimum variance for unbiased estimator used as efficiency benchmark

Efficiency comparison of unbiased estimators

• Lower variance indicates higher efficiency when comparing estimators
• Calculate and directly compare variances of different estimators
• Relative efficiency ratio computed as $\frac{\text{Var}(\hat{\theta}_1)}{\text{Var}(\hat{\theta}_2)}$
• Asymptotic efficiency evaluates as sample size approaches infinity
• Sufficient statistics contain all parameter information often yielding efficient estimators (sample mean for normal distribution)

Mean Squared Error (MSE)

Components of mean squared error

• MSE defined as expected squared difference between estimator and parameter $E[(\hat{\theta} - \theta)^2]$
• Bias component measures systematic deviation from true value $E[\hat{\theta}] - \theta$
• Variance component quantifies estimator spread around expected value $E[(\hat{\theta} - E[\hat{\theta}])^2]$
• Bias-variance decomposition expresses MSE as $\text{Var}(\hat{\theta}) + [\text{Bias}(\hat{\theta})]^2$
• Trade-off between bias and variance may lead to biased estimators with lower overall MSE (ridge regression)

Calculation and interpretation of MSE

• Calculate MSE by:
  1. Determine estimator's expected value
  2. Calculate bias
  3. Compute variance
  4. Sum squared bias and variance
• Lower MSE indicates better overall estimator performance (comparing OLS vs ridge regression)
• MSE for unbiased estimators equals variance $\text{Var}(\hat{\theta})$ when $E[\hat{\theta}] = \theta$
• Consistent estimators have MSE approaching zero as sample size increases (maximum likelihood estimators)
• Related measures include Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE)