🎲Intro to Probabilistic Methods Unit 9 – Estimation & Hypothesis Testing

Key Concepts and Definitions

• Estimation involves using sample data to make inferences about population parameters
• Point estimation provides a single value as an estimate of a population parameter (mean, proportion, variance)
• Interval estimation gives a range of values that likely contains the true population parameter with a certain level of confidence
• Hypothesis testing assesses whether sample data supports a claim about a population parameter
• Null hypothesis ($H_0$) represents the default or status quo, assuming no effect or difference
• Alternative hypothesis ($H_a$ or $H_1$) represents the claim or research question being tested
• Type I error (false positive) occurs when rejecting a true null hypothesis
  • Denoted by $\alpha$, the significance level of the test
• Type II error (false negative) occurs when failing to reject a false null hypothesis
  • Denoted by $\beta$, related to the power of the test ($1-\beta$)

Types of Estimation

• Point estimation
  • Provides a single value as an estimate of a population parameter
  • Examples include sample mean, sample proportion, and sample variance
• Interval estimation
  • Gives a range of values that likely contains the true population parameter
  • Confidence intervals are the most common form of interval estimation
• Bayesian estimation
  • Incorporates prior knowledge or beliefs about the parameter
  • Updates the estimate based on observed data using Bayes' theorem
• Nonparametric estimation
  • Makes fewer assumptions about the underlying population distribution
  • Useful when the population distribution is unknown or not normally distributed

Point Estimation Techniques

• Method of moments estimation
  • Equates sample moments (mean, variance) to population moments
  • Solves the resulting equations to estimate parameters
• Maximum likelihood estimation (MLE)
  • Finds the parameter values that maximize the likelihood of observing the sample data
  • Requires specifying the likelihood function based on the assumed distribution
• Least squares estimation
  • Minimizes the sum of squared differences between observed and predicted values
  • Commonly used in linear regression to estimate coefficients
• Bayesian estimation
  • Combines prior information with the likelihood of the data to obtain a posterior distribution
  • Estimates can be derived from the posterior, such as the mean or mode

Interval Estimation and Confidence Intervals

• Confidence intervals provide a range of plausible values for a population parameter
• The confidence level (e.g., 95%) represents the proportion of intervals that would contain the true parameter if the sampling process were repeated many times
• General form of a confidence interval: point estimate $\pm$ margin of error
  • Margin of error depends on the confidence level, sample size, and variability of the data
• Confidence intervals for means
  • For a large sample size or known population variance, use a z-interval
  • For a small sample size and unknown population variance, use a t-interval
• Confidence intervals for proportions
  • Use a z-interval with a continuity correction for small sample sizes
• Interpreting confidence intervals
  • A 95% confidence interval does not mean a 95% probability that the true parameter lies within the interval
  • Instead, it means that if the sampling process were repeated, 95% of the resulting intervals would contain the true parameter

Hypothesis Testing Fundamentals

• Hypothesis testing assesses the evidence in favor of a claim about a population parameter
• The null hypothesis ($H_0$) represents the default or status quo, while the alternative hypothesis ($H_a$ or $H_1$) represents the claim being tested
• The significance level ($\alpha$) is the probability of rejecting $H_0$ when it is true (Type I error)
  • Common choices for $\alpha$ are 0.01, 0.05, and 0.10
• The p-value is the probability of observing a test statistic as extreme as or more extreme than the one calculated from the sample data, assuming $H_0$ is true
• Rejecting $H_0$ when the p-value is less than $\alpha$ provides evidence in favor of $H_a$
• Failing to reject $H_0$ does not prove it is true, but suggests insufficient evidence against it
• One-tailed vs. two-tailed tests
  • One-tailed tests have a directional alternative hypothesis (greater than or less than)
  • Two-tailed tests have a non-directional alternative hypothesis (not equal to)

Common Statistical Tests

• Z-test for a single mean
  • Tests whether a population mean differs from a hypothesized value
  • Assumes a large sample size or known population variance
• T-test for a single mean
  • Similar to the z-test but used when the sample size is small and population variance is unknown
• Z-test for a single proportion
  • Tests whether a population proportion differs from a hypothesized value
• Two-sample t-test for comparing means
  • Independent samples: tests whether two population means are equal
  • Paired samples: tests whether the mean difference between paired observations is zero
• Chi-square test for independence
  • Tests whether two categorical variables are associated
• Chi-square goodness-of-fit test
  • Tests whether observed frequencies match expected frequencies based on a hypothesized distribution
• Analysis of Variance (ANOVA)
  • Tests for differences among three or more population means
  • One-way ANOVA for one factor, two-way ANOVA for two factors

Interpreting Results and Drawing Conclusions

• Rejecting the null hypothesis ($H_0$) suggests evidence in favor of the alternative hypothesis ($H_a$)
  • Conclude there is a significant effect or difference
  • The strength of evidence depends on the p-value and significance level
• Failing to reject $H_0$ does not prove it is true, but suggests insufficient evidence against it
  • Cannot conclude there is no effect or difference, only that there is not enough evidence to support $H_a$
• Statistical significance does not imply practical significance
  • Large sample sizes can lead to statistically significant results even for small effects
  • Consider the magnitude of the effect and its real-world implications
• Confidence intervals provide additional information about the precision and uncertainty of estimates
  • Narrower intervals suggest more precise estimates
  • Wider intervals suggest greater uncertainty
• Limitations and potential issues
  • Violations of assumptions (e.g., normality, independence) can affect the validity of results
  • Multiple testing and p-hacking can inflate Type I error rates
  • Confounding variables and other sources of bias can distort relationships

Real-world Applications and Examples

• Quality control
  • Hypothesis testing to determine if a manufacturing process is producing items within acceptable limits
  • Confidence intervals to estimate the proportion of defective items in a batch
• Clinical trials
  • Hypothesis testing to compare the effectiveness of a new drug to a placebo or standard treatment
  • Interval estimation to determine the range of likely treatment effects
• A/B testing in marketing
  • Hypothesis testing to compare the performance of two website designs or ad campaigns
  • Confidence intervals to estimate the difference in conversion rates
• Election polling
  • Interval estimation to construct a margin of error for the proportion of voters supporting a candidate
  • Hypothesis testing to determine if one candidate has a significant lead over another
• Psychology research
  • Hypothesis testing to assess the relationship between variables (e.g., stress and sleep quality)
  • ANOVA to compare the effectiveness of different therapy techniques
• Environmental studies
  • Hypothesis testing to determine if pollution levels exceed a critical threshold
  • Confidence intervals to estimate the average concentration of a contaminant in a water source