🎲Intro to Statistics Unit 10 – Two-Sample Hypothesis Testing

Key Concepts

• Two-sample hypothesis testing compares the means, proportions, or variances of two independent populations
• Null hypothesis ($H_0$) assumes no significant difference between the two population parameters
• Alternative hypothesis ($H_a$) suggests a significant difference between the two population parameters
• Test statistic measures the difference between the sample statistics and the null hypothesis
• P-value represents the probability of obtaining the observed results if the null hypothesis is true
• Significance level ($\alpha$) is the threshold for rejecting the null hypothesis (commonly 0.05)
• Type I error (false positive) occurs when rejecting a true null hypothesis
• Type II error (false negative) occurs when failing to reject a false null hypothesis

Types of Two-Sample Tests

• Two-sample t-test compares the means of two independent populations with normal distributions
  • Independent samples t-test assumes equal population variances
  • Welch's t-test assumes unequal population variances
• Two-proportion z-test compares the proportions of two independent populations with binary outcomes
• Two-sample F-test compares the variances of two independent populations with normal distributions
• Mann-Whitney U test (Wilcoxon rank-sum test) compares the medians of two independent populations with non-normal distributions
• Chi-square test compares the distributions of two independent populations with categorical data

Assumptions and Conditions

• Independence assumes that the samples are randomly selected and independent of each other
  • Randomization ensures that the samples are representative of their respective populations
  • Sampling without replacement maintains independence within each sample
• Normality assumes that the populations follow a normal distribution
  • Sample size of at least 30 is often considered sufficient for the Central Limit Theorem to apply
  • Shapiro-Wilk or Anderson-Darling tests can assess normality for smaller sample sizes
• Equal variances assumes that the population variances are approximately equal (for independent samples t-test)
  • Levene's test or F-test can assess the equality of variances
• Randomness assumes that the data is obtained through a random process without bias
• 10% condition ensures that the sample size is no more than 10% of the population size (for proportions)

Calculating Test Statistics

• Two-sample t-test statistic: $t = \frac{\bar{x}_1 - \bar{x}_2}{s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}$, where $s_p = \sqrt{\frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{n_1 + n_2 - 2}}$
• Two-proportion z-test statistic: $z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})(\frac{1}{n_1} + \frac{1}{n_2})}}$, where $\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$
• Two-sample F-test statistic: $F = \frac{s_1^2}{s_2^2}$, where $s_1^2$ and $s_2^2$ are the sample variances
• Degrees of freedom for t-test: $df = n_1 + n_2 - 2$ (independent samples) or $df = \frac{(s_1^2/n_1 + s_2^2/n_2)^2}{(s_1^2/n_1)^2/(n_1-1) + (s_2^2/n_2)^2/(n_2-1)}$ (Welch's)
• Degrees of freedom for F-test: $df_1 = n_1 - 1$ and $df_2 = n_2 - 1$

Interpreting P-values

• P-value represents the strength of evidence against the null hypothesis
• Smaller P-values indicate stronger evidence against the null hypothesis
• P-value < significance level ($\alpha$) suggests rejecting the null hypothesis
• P-value ≥ significance level ($\alpha$) suggests failing to reject the null hypothesis
• P-value does not measure the size of the effect or the importance of the result
• P-value is influenced by sample size, with larger samples more likely to yield smaller P-values

Making Decisions

• Compare the P-value to the chosen significance level ($\alpha$)
• If P-value < $\alpha$, reject the null hypothesis and conclude a significant difference between the populations
• If P-value ≥ $\alpha$, fail to reject the null hypothesis and conclude insufficient evidence of a significant difference
• Consider the practical significance of the results in addition to statistical significance
• Assess the potential consequences of Type I and Type II errors in the context of the problem
• Interpret the results in the context of the research question and domain knowledge

Common Mistakes

• Failing to check assumptions and conditions before conducting the test
• Using the wrong test for the given data and research question
• Interpreting a small P-value as evidence in favor of the null hypothesis
• Concluding a significant difference without considering the practical significance
• Confusing statistical significance with practical or clinical significance
• Overgeneralizing the results beyond the scope of the study
• Failing to account for multiple comparisons when conducting multiple tests simultaneously
• Misinterpreting the absence of evidence as evidence of absence

Real-World Applications

• Comparing the effectiveness of two different treatments or interventions (medical research)
• Evaluating the difference in customer satisfaction between two products or services (market research)
• Assessing the impact of two different teaching methods on student performance (education)
• Comparing the strength of two different materials or manufacturing processes (engineering)
• Investigating the difference in voter preferences between two demographic groups (political science)
• Analyzing the difference in financial performance between two investment strategies (finance)
• Comparing the environmental impact of two different energy sources (environmental science)
• Evaluating the difference in employee productivity between two management styles (organizational psychology)