Significance Level in AP Statistics

In AP Statistics, the significance level (α) is the cutoff you set before a test. If the p-value is at or below α, you reject the null hypothesis. α is also the probability of a Type I error (rejecting a true null).

What is Significance Level?

The significance level, written as α (alpha), is the line you draw in the sand before you look at your data. It answers one question: how unlikely does my result have to be, assuming the null hypothesis is true, before I stop believing the null? Common choices are α = 0.05, 0.01, and 0.10.

Once you have a p-value, the decision is a simple comparison. If p-value ≤ α, the result is statistically significant, so you reject H₀ and have convincing evidence for Hₐ. If p-value > α, you fail to reject H₀, which means you don't have convincing evidence for Hₐ. Think of α as the bar and the p-value as the jump. A small p-value clears a bar set at α. α also has a second meaning that the exam loves. It's the probability of making a Type I error, which is rejecting H₀ when H₀ is actually true. Pick α = 0.05 and you're accepting a 5% chance of a false alarm.

Why Significance Level matters in AP® Statistics

Significance level shows up anywhere you run a hypothesis test. It first appears in 3.6 p-Values, where you learn to compare a p-value to α. It returns in 3.7 Carrying Out a Test for a Population Proportion and 3.13 Carrying Out a Test for the Difference Between Two Population Proportions. In 3.8 Potential Errors When Performing Tests, it gets its second job as the Type I error probability. It's also the decision rule in 3.15 Carrying Out a Chi-Square Test for Homogeneity or Independence, 4.5 Carrying Out a Test for a Population Mean or Population Mean Difference, and 4.10 Carrying Out a Test for the Difference Between Two Population Means. The procedure changes from topic to topic, but α works the same way every time. That makes it one of the few ideas that ties every inference test in the course together. For the full mechanics of any one test, head to that topic's study guide.

How Significance Level connects across the course

p-Values (Unit 3)

The p-value comes from your data, while α comes from you before the data shows up. A test conclusion is just these two numbers going head to head. Small p beats α, so you reject H₀.

Type I Error and Power (Unit 3)

α is literally the probability of a Type I error, so lowering α makes false alarms less likely. The tradeoff is that a smaller α makes it harder to reject H₀, which raises the chance of a Type II error and lowers power. You can't shrink both error risks just by moving α, but a bigger sample helps.

Chi-Square Tests (Unit 3)

A chi-square test for homogeneity or independence uses a totally different statistic, but the ending is identical. You find the p-value from the chi-square distribution and compare it to α. Same decision rule, different distribution.

One-Sample and Two-Sample t-Tests (Unit 4)

When you move from proportions to means, you swap z for t, but α doesn't change its role at all. Whether you're testing one mean, a mean difference, or two means, you still reject H₀ only when the p-value is at or below α.

Is Significance Level on the AP® Statistics exam?

Multiple choice. Expect stems that hand you a p-value and ask what conclusion fits a given α. A p-value of 0.078 means you fail to reject at α = 0.05 but would reject at α = 0.10. A p-value of 0.12 isn't significant at any of the usual levels. Another common stem gives a test statistic, such as z = 1.96 in a right-tailed test. You turn that into a p-value (about 0.025) and compare it to α = 0.05. Watch for "what's wrong with this claim" questions too. A p-value of 0.078 on a two-sample t-test does not give "strong evidence" at α = 0.05.

Free response. Released FRQs that ask whether data give "convincing statistical evidence" all depend on α. Examples include 2017 Q5, 2018 Q6, 2021 Q4, and 2023 Q4. In your conclusion, you need to do three things. (1) State α, choosing one yourself if the problem doesn't give it. (2) Compare the p-value to α explicitly, like "since 0.03 < 0.05..." (3) Give a decision about H₀ and a conclusion in context about Hₐ. Skipping the explicit comparison is one of the most common ways to lose credit.

Significance Level vs p-value

α is a fixed threshold you choose before collecting data, and it doesn't change. The p-value is calculated from your sample, so it changes with every new sample. α asks "how rare is rare enough?" The p-value answers "how rare was my result, if H₀ were true?" You reject H₀ when the p-value is at or below α, never the other way around.

Key things to remember about Significance Level

• The significance level α is chosen before you collect data and is most often 0.05.

• If the p-value is less than or equal to α, you reject the null hypothesis and the result is statistically significant.

• If the p-value is greater than α, you fail to reject the null hypothesis, which is not the same as proving the null is true.

• α is the probability of a Type I error, which means rejecting a null hypothesis that is actually true.

• Lowering α reduces the chance of a Type I error but makes a Type II error more likely and decreases power.

• On FRQs, always state α and write the explicit comparison between the p-value and α before giving your conclusion in context.

Frequently asked questions about Significance Level

What is the significance level in AP Stats?

The significance level (α) is the cutoff you set before a hypothesis test for deciding whether to reject H₀. If the p-value is at or below α, usually 0.05, you reject H₀ and have convincing evidence for Hₐ.

Is the significance level the same as the p-value?

No. α is a threshold you pick ahead of time, while the p-value is calculated from your sample data. You compare the two. For example, a p-value of 0.078 is greater than α = 0.05, so you fail to reject H₀.

Does failing to reject H₀ at α = 0.05 mean the null hypothesis is true?

No. It only means the data didn't give convincing evidence against H₀ at that level. The null could still be false, and that missed detection is a Type II error.

What does α have to do with Type I error?

α is the probability of a Type I error, which is rejecting H₀ when it's actually true. With α = 0.05, you accept a 5% chance of that false alarm. With α = 0.01, the chance drops to 1%, but power also drops.

What significance level should I use if the FRQ doesn't give one?

State one yourself, typically α = 0.05, and use it in an explicit comparison like "because the p-value of 0.03 is less than α = 0.05, we reject H₀." Graders look for that comparison in the conclusion.