Intro to Econometrics Unit 5 Review: Hypothesis Tests & Confidence Intervals

Key Concepts

• Hypothesis testing involves making a claim about a population parameter and using sample data to assess the validity of the claim
• Null hypothesis ($H_0$) represents the default or status quo position, typically stating no effect or no difference
• Alternative hypothesis ($H_a$ or $H_1$) represents the claim being tested, often indicating an effect or difference
  • Can be one-sided (greater than or less than) or two-sided (not equal to)
• Type I error (false positive) occurs when rejecting a true null hypothesis, denoted by $\alpha$ (significance level)
• Type II error (false negative) occurs when failing to reject a false null hypothesis, denoted by $\beta$
  • Power of a test ($1-\beta$) measures the probability of correctly rejecting a false null hypothesis
• Critical value is the threshold used to determine whether to reject or fail to reject the null hypothesis based on the test statistic

Types of Hypothesis Tests

• One-sample tests compare a sample statistic to a hypothesized population parameter (mean, proportion)
  • Example: testing if the average income of a city differs from the national average
• Two-sample tests compare statistics between two independent samples (difference in means, proportions)
  • Example: comparing the effectiveness of two different marketing strategies on sales
• Paired tests compare two related samples or repeated measures (before-after, matched pairs)
  • Example: assessing the impact of a training program on employee performance
• One-way ANOVA tests for differences among three or more groups (means)
  • Example: comparing customer satisfaction ratings across multiple product categories
• Chi-square tests for independence or goodness-of-fit (categorical variables)
  • Example: testing if there is an association between education level and voting behavior
• F-tests compare the variances of two or more populations
• Non-parametric tests (Mann-Whitney U, Wilcoxon signed-rank, Kruskal-Wallis) for data that violates assumptions of parametric tests

Confidence Intervals Explained

• Confidence intervals provide a range of plausible values for a population parameter with a specified level of confidence
• Constructed using the sample statistic (point estimate) and the standard error
• Confidence level (e.g., 95%) represents the long-run probability that the interval contains the true population parameter
  • Higher confidence levels result in wider intervals, lower levels result in narrower intervals
• Interpretation: "We are 95% confident that the true population parameter lies within the calculated interval"
• Margin of error is half the width of the confidence interval, representing the maximum expected difference between the sample estimate and the population parameter
• Factors affecting the width of the confidence interval include sample size, variability, and confidence level
  • Larger sample sizes, lower variability, and lower confidence levels lead to narrower intervals

Statistical Significance and p-values

• Statistical significance indicates the likelihood that the observed results are due to chance rather than a real effect
• p-value measures the probability of obtaining the observed or more extreme results, assuming the null hypothesis is true
  • Smaller p-values provide stronger evidence against the null hypothesis
• Significance level ($\alpha$) is the threshold for determining statistical significance, commonly set at 0.05
  • If p-value < $\alpha$, reject the null hypothesis and conclude the result is statistically significant
  • If p-value ≥ $\alpha$, fail to reject the null hypothesis and conclude the result is not statistically significant
• Statistically significant results may not always be practically meaningful, importance of considering effect size and context
• Multiple testing problem: increased likelihood of Type I errors when conducting multiple hypothesis tests simultaneously
  • Bonferroni correction and false discovery rate (FDR) control are methods to adjust for multiple comparisons

Common Test Statistics

• z-statistic: standardized test statistic for comparing a sample statistic to a population parameter when the population standard deviation is known or the sample size is large
  • Example: testing if the proportion of defective products differs from a target value
• t-statistic: standardized test statistic for comparing a sample statistic to a population parameter when the population standard deviation is unknown and the sample size is small
  • Example: testing if the average weight loss of a diet program differs from zero
• $\chi^2$ (chi-square) statistic: test statistic for assessing the independence between categorical variables or the goodness-of-fit of a distribution
  • Example: testing if there is an association between gender and job satisfaction
• F-statistic: test statistic for comparing the variances of two or more populations or the overall significance of a regression model
  • Example: testing if the variances of exam scores differ between three schools

Assumptions and Limitations

• Assumptions underlying hypothesis tests include random sampling, independence of observations, and specific distribution requirements (e.g., normality)
  • Violations of assumptions can lead to inaccurate results and invalid conclusions
• Sample size and power considerations: larger samples generally provide more precise estimates and higher power to detect true effects
  • Underpowered studies may fail to detect important differences or relationships
• Outliers and influential observations can substantially impact the results of hypothesis tests
  • Robust methods (e.g., trimmed means, rank-based tests) can be used to mitigate the impact of outliers
• Hypothesis tests do not prove causality, only assess the strength of evidence against the null hypothesis
  • Observational studies are particularly prone to confounding factors that can bias results
• Publication bias: tendency for statistically significant results to be more likely published, leading to overestimation of effects

Applications in Econometrics

• Testing the significance of regression coefficients to determine if explanatory variables have a real impact on the dependent variable
  • Example: testing if education level significantly affects earnings
• Comparing the means or proportions of economic indicators across different groups or time periods
  • Example: testing if the average GDP growth rate differs between developed and developing countries
• Assessing the goodness-of-fit of economic models using chi-square tests or likelihood ratio tests
  • Example: testing if a proposed consumption function adequately explains consumer behavior
• Evaluating the effectiveness of economic policies or interventions using hypothesis tests
  • Example: testing if a minimum wage increase significantly impacts employment levels
• Testing for the presence of heteroscedasticity, autocorrelation, or multicollinearity in regression models
  • Example: using the Breusch-Pagan test to detect heteroscedasticity in a linear regression model

Tips and Tricks

• Always state the null and alternative hypotheses clearly and in terms of population parameters before conducting the test
• Choose the appropriate test based on the research question, data type, and assumptions
  • Consult flowcharts or decision trees to help select the correct test
• Double-check the assumptions of the test and consider alternative methods if assumptions are violated
  • Example: using a Welch's t-test instead of a standard t-test when variances are unequal
• Report the test statistic, p-value, and confidence interval (if applicable) when presenting results
  • Interpret the results in the context of the research question and consider practical significance
• Use visualizations (e.g., boxplots, scatterplots) to explore the data and communicate the results effectively
• Be cautious when interpreting results from multiple hypothesis tests and consider adjusting the significance level accordingly
  • Example: using the Bonferroni correction when conducting pairwise comparisons after an ANOVA
• Consider the limitations of hypothesis testing and use other approaches (e.g., confidence intervals, effect sizes) to provide a more comprehensive understanding of the data