Statistical Significance in AP Psychology

How the p < .05 cutoff turns data into a conclusion

Two groups in a study rarely end up with exactly the same scores. Individual differences and chance variation can produce a gap between groups even when the independent variable has no effect. Significance testing asks whether the gap in this study is larger and more consistent than chance would usually produce.

Researchers answer that with inferential statistics, which produce a p-value. The p-value is the probability of getting results at least this extreme if the null hypothesis were true. If p is below .05, results like these would show up less than 5% of the time in a world with no real effect. That's rare enough for researchers to reject the null hypothesis and call the finding statistically significant.

Here's a small example. A researcher randomly assigns participants to sleep either 8 hours or 4 hours before a memory test. The 8-hour group's mean score is 82 and the 4-hour group's is 74, with p = .02. Because .02 is below .05, the difference is statistically significant, and the researcher can conclude that sleep amount likely affected memory scores. If p had been .30, the same 8-point gap would not be significant, and chance would remain a reasonable explanation.

This term doesn't belong to one unit. In the current AP Psychology course, research methods and statistics are built into the science practices that apply across all five units. Statistical significance falls under Science Practice 3, Data Interpretation, and it connects to Science Practice 2, Research Methods and Design, because whether a significant result supports a causal claim depends on how the study was designed. That's why it can show up attached to a study on sleep, conformity, therapy outcomes, or anything else.

What a significant result does not prove

Many wrong answer choices are built on stretching what significance means. A significant result says chance is an unlikely explanation. It does not say any of the following:

• The hypothesis is proven. Science deals in support, and a single significant study can still be a fluke or fail to replicate.
• The effect is large or important. With a big enough sample, even a tiny difference between means can reach p < .05. Effect size is a separate question.
• One variable caused the other. A significant correlation coefficient from a correlational study still shows only a relationship. Causal claims need an experiment with random assignment.
• A nonsignificant result means there's no effect. It means the study didn't find enough evidence to rule out chance, which can happen with a small sample.

Also keep the p-value and significance apart. The p-value is a number, and significance is the decision you make by comparing that number to .05. Report the p-value as your evidence and state significance as your conclusion.

Reading significance in an Article Analysis summary

The Article Analysis Question (Question 1) gives you a summarized study and asks you to interpret basic statistics and judge how well the findings support a concept. This is Science Practice 3 in action. When the summary reports a p-value or says results were significant, you're expected to say what that means about chance. You read the reported value and compare it to .05.

A strong response names the result and explains it in one move: "The difference between the sleep groups was statistically significant (p < .05), so it is unlikely to be due to chance, which supports the claim that sleep affects memory." Weak responses just write "it was significant" without saying what that rules out.

On the Evidence-Based Question (Question 2), use statistical significance when a source reports it, alongside the study's design and other findings, to judge how well that source supports your claim. This draws on Science Practice 4, Argumentation. Assess how directly each study supports your claim, considering its design and results. A well-designed experiment can support a causal claim in a way a correlational study cannot. Explaining how a source's design and results connect to your claim is what turns a quoted source into actual reasoning.

In multiple choice, set-based questions may show a table or graph of group means alongside a p-value and ask what researchers can conclude. The right answer usually limits itself to "unlikely due to chance." Choices that claim proof, a large effect, or causation from correlational data are the traps.