13.5 Lab: One-Way ANOVA

13.5 Lab: One-Way ANOVA

One-Way ANOVA lets you test whether three or more group means are significantly different from each other. This lab walks you through performing the test step by step, from setting up your hypotheses to interpreting the output.

Setting Up the Problem

Before running any calculations, you need a clear framework:

1. Identify your groups. These are the categories of your independent variable (e.g., three different teaching methods, four brands of batteries).

2. Identify your response variable. This is the quantitative outcome you're measuring for each group (e.g., test scores, battery life in hours).

3. State your hypotheses.

   • $H_0$: All group means are equal ($\mu_1 = \mu_2 = \mu_3 = \dots$)
   • $H_a$: At least one group mean is different

4. Choose your significance level. Typically $\alpha = 0.05$ unless told otherwise.

Checking Assumptions

ANOVA results are only trustworthy if three conditions hold:

• Independence. Observations within and across groups should be independent of each other. Random sampling or random assignment helps satisfy this.
• Normality. The data in each group should be approximately normally distributed. With small samples, check histograms or normal probability plots for each group. With larger samples (roughly 30+ per group), the Central Limit Theorem makes this less of a concern.
• Equal variances (homoscedasticity). The spread of data should be similar across groups. A common rule of thumb: if the largest group standard deviation is no more than twice the smallest, you're generally fine.

Calculating the F Statistic

The F statistic compares variation between groups to variation within groups. Here's the process:

1. Find each group mean ($\bar{x}_1, \bar{x}_2, \bar{x}_3, \dots$) and the overall mean ($\bar{x}$) of all observations combined.
2. Calculate the between-group variation (MSG). This measures how far each group mean is from the overall mean, weighted by group size: $MSG = \frac{\sum n_i (\bar{x}_i - \bar{x})^2}{k - 1}$ where $k$ is the number of groups and $n_i$ is the size of group $i$.
3. Calculate the within-group variation (MSE). This measures the average spread of observations around their own group mean: $MSE = \frac{\sum (n_i - 1) s_i^2}{N - k}$ where $s_i^2$ is the variance of group $i$ and $N$ is the total number of observations.
4. Compute the F statistic: $F = \frac{MSG}{MSE}$

A large F value means the group means differ more than you'd expect from random variation alone.

Reading the ANOVA Table

Most software outputs an ANOVA summary table. Here's what each column means:

• df = degrees of freedom
• SS = sum of squares (total variation attributed to that source)
• MS = mean square (SS divided by its df)
• The p-value tells you the probability of getting an F statistic this large (or larger) if $H_0$ were true.

Making a Decision

Compare the p-value to your significance level:

• If $p \leq \alpha$: Reject $H_0$. There is statistically significant evidence that at least one group mean differs.
• If $p > \alpha$: Fail to reject $H_0$. You don't have enough evidence to conclude the means differ.

Keep in mind that rejecting $H_0$ does not tell you which specific groups differ. It only tells you that not all means are equal.

Example Walkthrough

Suppose you're comparing average exam scores across three study methods, with 10 students per group.

1. Hypotheses: $H_0: \mu_A = \mu_B = \mu_C$ vs. $H_a$: at least one mean differs.

2. Check assumptions: Groups are independent (randomly assigned). Sample sizes are small, so you'd check normality plots. Standard deviations are similar (largest is 6.4, smallest is 5.8; ratio well under 2).

3. Degrees of freedom: Between: $k - 1 = 2$. Within: $N - k = 27$.

4. Suppose your software gives: $F = 3.12$, $p = 0.048$.

5. Decision: Since $0.048 < 0.05$, reject $H_0$. There's significant evidence that at least one study method leads to a different mean exam score.

Post-Hoc Testing (What Comes After)

When you reject $H_0$, you'll often want to know which groups differ. Common follow-up procedures include:

• Tukey's HSD (Honest Significant Difference): Compares every pair of group means while controlling for the increased chance of a false positive from doing multiple comparisons.
• Bonferroni correction: Adjusts the significance level by dividing $\alpha$ by the number of pairwise comparisons. Simple but conservative.

These post-hoc tests are only appropriate after a significant ANOVA result.

Common Mistakes to Avoid

• Running multiple two-sample t-tests instead of ANOVA. Each t-test has its own chance of a Type I error. With many comparisons, those errors add up fast. ANOVA handles all groups in a single test.
• Claiming a specific group is different without post-hoc analysis. A significant F test only tells you something differs, not what.
• Ignoring the equal variance assumption. If one group's spread is much larger than the others, your F statistic and p-value may not be reliable.
• Confusing statistical significance with practical significance. A tiny difference in means can be "statistically significant" with a large enough sample. Always look at the actual size of the differences too.