Post-Hoc Test

Definition

A post-hoc test, also known as a multiple comparison test, is a statistical analysis performed after a significant one-way ANOVA result to determine which specific groups or conditions differ from one another. These tests help identify where the differences lie within the overall significant ANOVA finding.

5 Must Know Facts For Your Next Test

1. Post-hoc tests are conducted after a significant one-way ANOVA result to determine which specific group means differ from one another.
2. These tests help identify the source of the significant difference found in the overall ANOVA, pinpointing where the differences lie between the groups.
3. Post-hoc tests adjust the p-values to control the familywise error rate, which is the probability of making at least one Type I error when conducting multiple comparisons.
4. Common post-hoc tests include Tukey's Honest Significant Difference (HSD), Dunnett's test, and Bonferroni correction.
5. The choice of post-hoc test depends on factors such as the number of groups, the assumption of equal variances, and the desired level of control over the familywise error rate.

Review Questions

• Explain the purpose of conducting a post-hoc test following a significant one-way ANOVA result.
  • The purpose of a post-hoc test is to determine which specific group means differ from one another after a significant one-way ANOVA result. The ANOVA indicates that at least one group mean is significantly different from the others, but it does not specify which groups are different. Post-hoc tests are used to identify the source of the significant difference by making pairwise comparisons between the group means, while controlling the familywise error rate.
• Describe how post-hoc tests adjust the p-values to control the familywise error rate.
  • Post-hoc tests, such as Tukey's HSD and Bonferroni correction, adjust the p-values to control the familywise error rate when making multiple comparisons. The familywise error rate is the probability of making at least one Type I error (false positive) when conducting multiple statistical tests. By adjusting the p-values, post-hoc tests ensure that the overall probability of making a Type I error across all the comparisons remains at the specified significance level (e.g., α = 0.05).
• Analyze the factors that influence the choice of post-hoc test in the context of a one-way ANOVA.
  • The choice of post-hoc test in a one-way ANOVA depends on several factors, including the number of groups, the assumption of equal variances, and the desired level of control over the familywise error rate. For example, Tukey's HSD is commonly used when the number of groups is large and the assumption of equal variances is met, as it provides a good balance between statistical power and control over the familywise error rate. Dunnett's test is often used when comparing each group to a control group, while the Bonferroni correction is a more conservative approach that can be used when the number of comparisons is small.

Related terms

One-Way ANOVA: A statistical test used to determine if there are any statistically significant differences between the means of three or more independent groups.

Multiple Comparison Procedure: A family of statistical tests used to control the overall Type I error rate when making multiple comparisons between groups.

Bonferroni Correction: A method of adjusting p-values to control the familywise error rate when conducting multiple statistical tests.