Post-hoc tests

Post-hoc tests are follow-up statistical comparisons done after a significant ANOVA to figure out which specific group means differ. In Cognitive Psychology, they help you tell where effects appear after comparing several conditions.

What are post-hoc tests?

Post-hoc tests are the follow-up comparisons you run in Cognitive Psychology after an ANOVA shows that not every group mean is the same. The ANOVA tells you there is some difference somewhere among your conditions, but it does not tell you which groups are different from each other. Post-hoc tests fill in that missing piece.

Think of a memory experiment with four study conditions, like words with no cue, sound cues, picture cues, and category cues. If the ANOVA gives you a significant F-value, you know at least one condition differs. A post-hoc test then checks the pairs, such as picture vs no cue or category vs sound, to find the specific comparisons that matter.

These tests exist because making lots of pairwise comparisons on your own can inflate Type I Error. That means you increase the chance of calling a difference significant when it is really just random noise. Post-hoc tests build in a correction or a more conservative decision rule so the final result is more trustworthy.

Different post-hoc tests make different tradeoffs. Bonferroni Correction is stricter, so it lowers false positives but can miss smaller real effects. Tukey's HSD is common when you want to compare all group pairs in a balanced design. Scheffé's test is even more flexible, but often more conservative.

In class, you usually see post-hoc tests after an experiment with more than two conditions, especially when the researcher wants to know exactly which manipulation changed attention, memory, or perception. They are not the first analysis step. They come after the overall ANOVA says, "There is something here worth looking at."

Why post-hoc tests matter in Cognitive Psychology

Post-hoc tests show you where the effect is hiding, which is a big deal in Cognitive Psychology because many experiments compare several conditions at once. A study on attention might compare reaction times across three task loads, or a memory study might compare recall after different study strategies. Without post-hoc tests, you only know that the groups are not all equal, not which condition actually produced the change.

This matters for interpreting results correctly. If one condition stands out, you can connect that pattern to a theory about encoding, interference, selective attention, or cognitive load. If the post-hoc results are not significant, you may need to say the overall ANOVA difference was driven by variation spread across groups rather than one clear pairwise effect.

It also trains you to think about error rates the way psychologists do. A lot of cognitive research depends on careful comparisons, so the choice of follow-up test changes how cautious your conclusions are. That makes post-hoc tests part of research literacy, not just statistics vocabulary.

How post-hoc tests connect across the course

ANOVA

ANOVA is the overall test that tells you whether there is any difference among three or more group means. Post-hoc tests come after a significant ANOVA when you need to locate the specific group pairs driving that difference. If the ANOVA is not significant, there is usually no reason to run post-hoc comparisons.

Type I Error

Type I Error is the false positive you get when you conclude a difference exists even though it does not. Post-hoc tests are designed to reduce this risk because multiple pairwise comparisons can quickly raise your chance of a mistake. The more comparisons you make, the more careful your analysis has to be.

Bonferroni Correction

Bonferroni Correction is one common way to make post-hoc comparisons more conservative. It lowers the cutoff for significance so you are less likely to claim a false difference between conditions. In Cognitive Psychology, that can be useful when you are comparing several attention or memory groups and want cleaner conclusions.

eta-squared

eta-squared tells you how much of the variation in your dependent variable is explained by the independent variable. Post-hoc tests tell you which specific groups differ, while eta-squared tells you how big the overall effect is. Together they give you both the pattern and the strength of the result.

Are post-hoc tests on the Cognitive Psychology exam?

A quiz or problem set will usually give you an ANOVA result and ask what to do next. Your move is to say that post-hoc tests are needed when the ANOVA is significant and the researcher wants to compare specific groups. You may also be asked to identify why the follow-up matters, which is to limit Type I Error while sorting out pairwise differences.

If you see a scenario with several conditions, look for the question "which groups differ?" That wording is the giveaway that post-hoc testing is the right next step. On essay questions, use the term to explain how psychologists move from a broad overall effect to a more precise interpretation of the data. In a lab report, you might mention the specific test used, such as Tukey's HSD or Bonferroni Correction, and explain what pairwise comparison came out significant.

Post-hoc tests vs ANOVA

ANOVA tests whether there is any overall difference among group means. Post-hoc tests do not replace ANOVA, they follow it and show which groups differ after the overall result is significant.

Key things to remember about post-hoc tests

• Post-hoc tests are follow-up comparisons used after a significant ANOVA to identify which group means differ.

• They matter because doing many pairwise comparisons raises the chance of Type I Error, or a false positive.

• In Cognitive Psychology, they often show which condition changed memory, attention, reaction time, or another mental process.

• Different post-hoc tests have different levels of strictness, so the test you choose can change which differences count as significant.

• If the overall ANOVA is not significant, post-hoc tests usually are not the next step.

Frequently asked questions about post-hoc tests

What is post-hoc tests in Cognitive Psychology?

Post-hoc tests are follow-up statistical tests used after a significant ANOVA to find out which specific groups differ. In Cognitive Psychology, that might mean checking which study condition led to better recall, faster reaction time, or lower error rates.

Why do you need post-hoc tests after ANOVA?

ANOVA only tells you whether there is an overall difference among group means. It does not tell you where that difference is, so post-hoc tests pinpoint the specific pair or pairs that are different while keeping false positives under control.

Are post-hoc tests the same as Bonferroni Correction?

Not exactly. Post-hoc tests are the broader category of follow-up comparisons, while Bonferroni Correction is one method that can be used to make those comparisons more conservative. You can think of Bonferroni as one tool inside the post-hoc toolbox.

How do post-hoc tests show up in Cognitive Psychology assignments?

You might see them in data analysis questions, lab reports, or experiment write-ups where a study compares three or more conditions. The task is usually to interpret which groups differ and explain what that pattern means for attention, memory, perception, or another cognitive process.