📊Honors Statistics Unit 10 Review

Two-sample hypothesis tests let you compare two independent groups and decide whether the difference you observe is statistically significant or just due to random sampling variability.

Two-sample t-test (for means) compares the means of two independent populations (e.g., average heights of men vs. women).

Assumptions:

Both populations are approximately normally distributed (or sample sizes are large enough for the CLT to kick in)
The two samples are independent of each other

The test statistic is:

$t = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

where:

$\bar{x}_1, \bar{x}_2$ = sample means
$\mu_1, \mu_2$ = hypothesized population means (under $H_0$ , the difference $\mu_1 - \mu_2$ is usually 0)
$s_1^2, s_2^2$ = sample variances
$n_1, n_2$ = sample sizes

Degrees of freedom can be calculated with the Welch-Satterthwaite formula (your calculator handles this), or you can use the conservative approach: $df = \min(n_1, n_2) - 1$ .

Two-sample z-test (for proportions) compares proportions from two independent populations (e.g., defect rates at two factories).

Conditions: both samples need to be large enough that $n\hat{p} \geq 5$ and $n(1-\hat{p}) \geq 5$ for each group.

The test statistic is:

$z = \frac{(\hat{p}_1 - \hat{p}_2) - (p_1 - p_2)}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$

where:

$\hat{p}_1, \hat{p}_2$ = sample proportions
$p_1, p_2$ = hypothesized population proportions (under $H_0$ , typically $p_1 - p_2 = 0$ )
$\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$ = pooled sample proportion (the overall success rate across both samples combined)
$x_1, x_2$ = number of successes in each sample

You use the pooled proportion because the null hypothesis assumes the two populations have the same proportion, so your best estimate of that shared proportion comes from combining both samples.

Both tests can be one-tailed or two-tailed depending on the research question.

Two-sample tests for means vs proportions, Comparing two means – Learning Statistics with R

Confidence intervals for population differences

Confidence intervals give you a range of plausible values for the true difference between two populations. They complement hypothesis tests by showing the size of the difference, not just whether one exists.

CI for the difference between two means:

$(\bar{x}_1 - \bar{x}_2) \pm t_{\alpha/2} \cdot \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$

$t_{\alpha/2}$ is the critical value from the t-distribution at your chosen confidence level
Degrees of freedom follow the same rule as the t-test above

CI for the difference between two proportions:

$(\hat{p}_1 - \hat{p}_2) \pm z_{\alpha/2} \cdot \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}$

$z_{\alpha/2}$ is the critical value from the standard normal distribution (e.g., 1.96 for 95% confidence)
Notice that the CI formula does not use the pooled proportion. That's because you're no longer assuming the two proportions are equal (that was a null hypothesis assumption for the test).

Two-sample tests for means vs proportions, Distribution of Differences in Sample Proportions (5 of 5) | Concepts in Statistics

Additional considerations

Paired samples: When data points are naturally linked (e.g., before/after measurements on the same subjects), use a paired t-test instead. You compute the differences within each pair and run a one-sample t-test on those differences.
Effect size: Measures the magnitude of the difference between groups. A result can be statistically significant but have a tiny effect size, meaning it may not matter in practice.
Power analysis: Determines the sample size needed to detect a specific effect size at a given significance level. Larger samples give you more power to detect real differences.

Interpreting Results and Drawing Conclusions

Interpretation of statistical results

P-value interpretation tells you how strong the evidence is against the null hypothesis.

A small p-value (typically < 0.05) means the observed difference would be unlikely if $H_0$ were true. You reject $H_0$ and conclude there's a significant difference. Example: p = 0.003 for a height comparison suggests men and women really do differ in average height.
A large p-value (≥ 0.05) means you don't have enough evidence to reject $H_0$ . This does not prove the populations are the same; it just means you can't rule out that the difference is due to chance.

Confidence interval interpretation offers another way to assess significance.

If the CI for the difference does not contain 0, the difference is significant at that confidence level. Example: a 95% CI of (2.1 cm, 5.8 cm) for the difference in mean heights doesn't include 0, so the difference is significant.
If the CI does contain 0, you cannot conclude there's a significant difference.

The p-value approach and the confidence interval approach will always agree at the same significance/confidence level. If your 95% CI excludes 0, your two-tailed test at $\alpha = 0.05$ will reject $H_0$ , and vice versa.

Statistical significance vs. practical significance is a distinction you should always make when drawing conclusions.

Statistical significance tells you the difference is unlikely to be from chance alone.
Practical significance asks whether the difference actually matters in context. A study with thousands of participants might find a statistically significant difference of 0.3 cm in average height. That's real, but probably irrelevant for any practical purpose.

When writing your conclusion, tie it back to the original research question and note any limitations: Were the samples truly random? Could confounding variables explain the result? Is the sample representative of the population you care about?

📊Honors Statistics Unit 10 Review