Theoretical Statistics Unit 8 ReviewHypothesis testing

Key Concepts and Definitions

• Hypothesis testing assesses claims or conjectures about a population parameter based on sample data
• Null hypothesis ($H_0$) represents the default or status quo position, typically stating no effect or no difference
• Alternative hypothesis ($H_a$ or $H_1$) represents the claim or research hypothesis, suggesting an effect or difference
• Test statistic quantifies the difference between the observed data and what is expected under the null hypothesis
• Critical value determines the boundary for rejecting the null hypothesis based on the significance level
• p-value measures the probability of obtaining the observed data or more extreme results, assuming the null hypothesis is true
• Type I error (false positive) occurs when rejecting a true null hypothesis
• Type II error (false negative) occurs when failing to reject a false null hypothesis

Foundations of Hypothesis Testing

• Hypothesis testing allows researchers to make statistical inferences about population parameters based on sample data
• Relies on the concept of probability and sampling distributions to assess the likelihood of observed results
• Assumes random sampling and independence of observations to ensure validity of inferences
• Requires specifying the null and alternative hypotheses, which are mutually exclusive and exhaustive
• Involves calculating a test statistic and comparing it to a critical value or p-value to make a decision
• Balances the risks of Type I and Type II errors by setting an appropriate significance level
• Provides a framework for making evidence-based decisions in various fields (psychology, medicine, business)

Types of Hypotheses

• One-tailed (directional) hypotheses specify the direction of the difference or effect
  • Right-tailed: Alternative hypothesis states the parameter is greater than a specific value
  • Left-tailed: Alternative hypothesis states the parameter is less than a specific value
• Two-tailed (non-directional) hypotheses do not specify the direction of the difference or effect
• Simple hypotheses specify a single value for the population parameter
• Composite hypotheses specify a range of values for the population parameter
• Null hypothesis always contains an equality sign (=, ≤, or ≥), while the alternative hypothesis contains an inequality (<, >, or ≠)
• Choice of hypothesis type depends on the research question and prior knowledge or expectations

Steps in Hypothesis Testing

1. State the null and alternative hypotheses based on the research question
2. Choose the appropriate test statistic and distribution (z, t, F, or chi-square) based on the data and assumptions
3. Set the significance level ($\alpha$) to determine the risk of a Type I error
4. Calculate the test statistic using the sample data and the hypothesized parameter value
5. Determine the critical value(s) or p-value associated with the test statistic
6. Compare the test statistic to the critical value(s) or p-value to make a decision
   • If the test statistic falls in the rejection region or the p-value is less than $\alpha$, reject the null hypothesis
   • If the test statistic falls outside the rejection region or the p-value is greater than $\alpha$, fail to reject the null hypothesis
7. Interpret the results in the context of the research question and draw conclusions

Test Statistics and Distributions

• Test statistics are calculated from sample data and used to compare with critical values or determine p-values
• The choice of test statistic depends on the type of data, sample size, and assumptions about the population distribution
• Z-test statistic follows a standard normal distribution and is used for testing hypotheses about means with known population variance or large sample sizes
• T-test statistic follows a Student's t-distribution and is used for testing hypotheses about means with unknown population variance or small sample sizes
• F-test statistic follows an F-distribution and is used for testing hypotheses about variances or comparing multiple means (ANOVA)
• Chi-square test statistic follows a chi-square distribution and is used for testing hypotheses about categorical variables or goodness-of-fit
• Assumptions such as normality, homogeneity of variance, and independence must be checked before selecting the appropriate test statistic

Significance Levels and p-values

• Significance level ($\alpha$) is the probability of making a Type I error, typically set at 0.05 or 0.01
• Represents the maximum acceptable risk of rejecting a true null hypothesis
• Critical values are determined based on the significance level and the degrees of freedom
• p-value is the probability of obtaining the observed data or more extreme results, assuming the null hypothesis is true
• Smaller p-values provide stronger evidence against the null hypothesis
• If the p-value is less than the significance level, the null hypothesis is rejected; otherwise, it is not rejected
• p-values are often misinterpreted as the probability of the null hypothesis being true or the importance of the result

Errors in Hypothesis Testing

• Type I error (false positive) occurs when rejecting a true null hypothesis
  • Probability of a Type I error is equal to the significance level ($\alpha$)
  • Controlled by setting an appropriate significance level based on the consequences of the error
• Type II error (false negative) occurs when failing to reject a false null hypothesis
  • Probability of a Type II error is denoted by $\beta$ and is related to the power of the test
  • Influenced by factors such as sample size, effect size, and variability
• Power is the probability of correctly rejecting a false null hypothesis (1 - $\beta$)
  • Increasing sample size, using a larger significance level, or focusing on larger effect sizes can increase power
• Balancing the risks of Type I and Type II errors is crucial in designing and interpreting hypothesis tests
• Consequences of each type of error should be considered in the context of the research question

Applications and Examples

• Testing the effectiveness of a new drug compared to a placebo in a clinical trial
  • Null hypothesis: The drug has no effect on the outcome variable
  • Alternative hypothesis: The drug has a significant effect on the outcome variable
• Comparing the mean test scores of two teaching methods to determine if one is superior
  • Null hypothesis: The mean test scores are equal for both teaching methods
  • Alternative hypothesis: The mean test scores are different for the two teaching methods
• Investigating if there is a significant correlation between two variables (income and education level)
  • Null hypothesis: There is no significant correlation between income and education level
  • Alternative hypothesis: There is a significant correlation between income and education level
• Examining if a new manufacturing process produces items with a mean weight different from the current process
  • Null hypothesis: The mean weight of items produced by the new process is equal to the current process
  • Alternative hypothesis: The mean weight of items produced by the new process is different from the current process
• Determining if the proportion of defective items produced by a machine exceeds a specified threshold
  • Null hypothesis: The proportion of defective items is less than or equal to the threshold
  • Alternative hypothesis: The proportion of defective items is greater than the threshold