Hypothesis Testing Steps

Why This Matters

Hypothesis testing is how statisticians move from sample data to conclusions about populations. On the AP Statistics exam, you're not just plugging numbers into formulas. You're being tested on the logic of inference: why we assume the null hypothesis is true, how we measure evidence against it, and what our conclusions actually mean in context. Every free-response question involving inference expects you to show this reasoning process.

The steps of hypothesis testing connect directly to core concepts like probability, sampling distributions, Type I and Type II errors, and the interpretation of p-values. Understanding each step will help you tackle everything from one-sample z-tests to chi-square tests for independence. Don't just memorize the sequence; know what each step accomplishes and how errors at any stage can invalidate your conclusions.

---

Setting Up the Test

Before any calculations happen, you need to establish what you're testing and what would count as convincing evidence. This foundation determines everything that follows.

State the Null and Alternative Hypotheses

• The null hypothesis ($H_0$) represents the "no effect" or "no difference" claim. It's what you assume is true until evidence suggests otherwise.
• The alternative hypothesis ($H_a$) captures what you're trying to find evidence for. This determines whether you run a one-sided or two-sided test.
• Write hypotheses using population parameters, not sample statistics. Use $\mu$, $p$, or $p_1 - p_2$, never $\bar{x}$ or $\hat{p}$. For example, write $H_0: p = 0.5$, not $H_0: \hat{p} = 0.5$.

Choose the Significance Level ($\alpha$)

• The significance level $\alpha$ is your threshold for "surprising enough to reject $H_0$." It's typically 0.05 unless the problem states otherwise.
• $\alpha$ equals the probability of a Type I error: rejecting $H_0$ when it's actually true (a false positive).
• Choosing $\alpha$ involves trade-offs. Lowering $\alpha$ reduces your Type I error risk but increases your Type II error risk and decreases power. You should set $\alpha$ before looking at the data.

---

Checking Conditions and Selecting Methods

The validity of your inference depends entirely on whether the conditions for the test are met. This is where many students lose points, either by skipping conditions or by stating them without actually verifying them with the given numbers.

Select the Appropriate Test Statistic

• Match your test to your data type and question. Use z-tests for proportions, t-tests for means, and chi-square for categorical distributions.
• The test statistic measures how far your sample result is from what $H_0$ predicts, standardized by the standard error.
• Common formulas:
  • For a one-proportion z-test: $z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$
  • For a one-sample t-test: $t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$

Notice that the proportion test uses $p_0$ (the null value) in the standard error, while the t-test uses $s$ (the sample standard deviation). That's because under $H_0$ we know $p_0$, but we never know the population standard deviation $\sigma$ for means.

Verify Conditions for Inference

You need to check three conditions. On FRQs, name each condition and show the math that confirms it.

1. Random condition: The data must come from a random sample or randomized experiment. This ensures the sample is representative of the population.
2. Independence condition (10% rule): When sampling without replacement, the sample size must be $\leq 10\%$ of the population. This keeps individual observations approximately independent.
3. Large Counts or Normality condition: For proportions, check that $np_0 \geq 10$ and $n(1-p_0) \geq 10$. For means, if $n \geq 30$, the Central Limit Theorem covers you; if $n < 30$, you need the population distribution to be roughly normal (check for strong skew or outliers in the sample).

---

Calculating and Evaluating Evidence

This is where the math happens. You're quantifying how surprising your sample result would be if $H_0$ were true.

Calculate the Test Statistic

1. Write out the formula for your chosen test.
2. Substitute in your sample values and the null parameter.
3. Compute the result.

Showing all three steps earns you process points on FRQs, even if you make an arithmetic mistake. The test statistic tells you how many standard errors your sample result falls from the hypothesized parameter.

Determine the P-Value

• The p-value is the probability of getting a result as extreme as (or more extreme than) yours, assuming $H_0$ is true.
• Use the correct distribution: standard normal (z) for proportions, t-distribution with $df = n - 1$ for means, chi-square for categorical tests.
• Direction matters. For a one-sided test, find the area in one tail. For a two-sided test, double the one-tail probability.

A common mistake: students sometimes describe the p-value as "the probability that $H_0$ is true." That's wrong. The p-value assumes $H_0$ is true and asks how likely your data would be under that assumption.

Compare P-Value to $\alpha$

• If p-value $\leq \alpha$: reject $H_0$. The evidence against the null is statistically significant.
• If p-value $> \alpha$: fail to reject $H_0$. There's insufficient evidence to support the alternative.
• Never say "accept $H_0$." You can only fail to reject it. Absence of evidence isn't evidence of absence.

---

Drawing Conclusions

The final steps connect your statistical results back to the real-world question. This is where you show understanding, not just calculation ability.

State Your Decision

• Use correct language: "reject $H_0$" or "fail to reject $H_0$." Never "accept" either hypothesis.
• Link the decision to your p-value and $\alpha$: "Since the p-value = 0.023 < $\alpha$ = 0.05, we reject $H_0$."
• Acknowledge uncertainty. Rejecting $H_0$ risks a Type I error; failing to reject risks a Type II error.

Interpret Results in Context

• Write a conclusion in plain language that someone without statistics training could understand.
• Reference the alternative hypothesis and the context. For example: "There is convincing evidence that the true proportion of defective items is greater than 0.02."
• Avoid causal language unless the study was a randomized experiment. Observational studies only support claims of association, not causation.

Assess Practical Significance

Statistical significance $\neq$ practical significance. A tiny difference can be "significant" with a large enough sample size.

Consider effect sizes and real-world implications. For example, a drug that lowers blood pressure by 0.5 mmHg might reach statistical significance with $n = 10{,}000$ patients, but a 0.5 mmHg drop is clinically meaningless. FRQs increasingly ask you to address whether results matter in context, not just whether they're "significant."

Also consider limitations: sample size, scope of inference, potential confounding variables, and how broadly the results can be generalized.

---

Quick Reference Table

---

Self-Check Questions

1. What is the difference between the null hypothesis and the alternative hypothesis, and why do we assume $H_0$ is true when calculating the p-value?

2. A student writes "We accept $H_0$ because the p-value is 0.12." Identify two errors in this statement and explain how to correct them.

3. Compare and contrast Type I and Type II errors: which one does $\alpha$ control, and how does increasing sample size affect each?

4. Why must you check conditions before performing a hypothesis test, and what happens to your conclusions if the conditions aren't met?

5. An FRQ presents a hypothesis test with p-value = 0.001 and asks whether the result is practically significant. What additional information would you need to answer this question, and why might statistical significance not imply practical importance?