11.5 Comparison of the Chi-Square Tests

Chi-Square Tests

Chi-square tests let you analyze categorical data by comparing what you actually observe to what you'd expect under some assumption. All three types use the same core formula, but they answer fundamentally different questions. Knowing which test to use, and why, is the main challenge here.

Each test calculates a chi-square statistic from observed and expected frequencies, then uses degrees of freedom to find a p-value. The decision framework is the same across all three; what changes is the setup: how many populations, how many variables, and what the null hypothesis claims.

more resources to help you study

practice questions

Goodness-of-Fit vs. Independence vs. Homogeneity

These three tests look similar in mechanics but differ in structure and purpose.

Goodness-of-fit tests whether a single categorical variable's distribution matches a specific claimed distribution. You have one population and one variable. You're comparing observed frequencies against expected frequencies derived from the hypothesized proportions.

Independence tests whether two categorical variables are related within a single population. You have one population but two variables, and you're asking: does knowing one variable tell you anything about the other?

Data for independence tests gets organized into a contingency table (rows for one variable, columns for the other).

Homogeneity tests whether the distribution of a single categorical variable is the same across two or more populations. You have multiple populations but one variable.

A common point of confusion: independence and homogeneity tests both use contingency tables and the same formula for expected counts. The difference is in the study design. Independence starts with one sample and measures two variables on each subject. Homogeneity starts with separate samples from distinct populations and measures one variable.

Null and Alternative Hypotheses

Each test has its own hypothesis structure:

• Goodness-of-fit
  • $H_0$: The data follows the hypothesized distribution (observed proportions match expected proportions)
  • $H_a$: The data does not follow the hypothesized distribution
• Independence
  • $H_0$: The two categorical variables are independent
  • $H_a$: The two categorical variables are not independent (they are associated)
• Homogeneity
  • $H_0$: The distribution of the categorical variable is the same across all populations
  • $H_a$: The distribution differs for at least one population

Notice that all three alternative hypotheses are non-directional. Chi-square tests only detect that something deviates from the null; they don't tell you how it deviates. If you reject $H_0$, you'll need to look at individual cell contributions to understand where the differences are.

Populations and Variables at a Glance

Chi-Square Test Statistic and Interpretation

All three tests use the same test statistic formula:

$\chi^2 = \sum \frac{(O - E)^2}{E}$

where $O$ is each observed frequency and $E$ is the corresponding expected frequency. Each cell contributes one term to the sum, and larger differences between observed and expected values push $\chi^2$ higher.

Degrees of freedom differ by test type:

• Goodness-of-fit: $df = k - 1$, where $k$ is the number of categories
• Independence and homogeneity: $df = (r - 1)(c - 1)$, where $r$ is the number of rows and $c$ is the number of columns in the contingency table

The p-value comes from the right tail of the chi-square distribution. A larger $\chi^2$ means a smaller p-value, which means stronger evidence against $H_0$. You can also compare $\chi^2$ to a critical value from a chi-square table at your chosen significance level. If $\chi^2$ exceeds the critical value, you reject $H_0$.

One thing to watch: the chi-square approximation works well only when expected frequencies are large enough. The standard rule is that every expected cell count should be at least 5. If some cells fall below that threshold, you may need to combine categories or use a different method.