11.2 Goodness-of-Fit Test

Goodness-of-Fit Test

The goodness-of-fit test checks whether observed data matches a specific theoretical distribution. You collect frequency data across categories, then use a chi-square statistic to measure how far your observed counts fall from what the theory predicts. If the gap is large enough, you reject the hypothesized distribution.

more resources to help you study

practice questions

Goodness-of-Fit Test for Distributions

This test works with any distribution you can specify in advance: uniform, binomial, Poisson, normal, or any other multinomial setup. The key idea is that your hypothesized distribution tells you what the expected frequencies should be, and you compare those to what you actually observed.

• Null hypothesis ($H_0$): The data follows the specified distribution
• Alternative hypothesis ($H_a$): The data does not follow the specified distribution

Test procedure:

1. State your null and alternative hypotheses, identifying the theoretical distribution.
2. Calculate the expected frequency for each category based on that distribution. Each expected frequency equals the total sample size multiplied by the probability the distribution assigns to that category.
3. Verify the conditions: every expected frequency should be at least 5. If some cells fall below 5, you may need to combine adjacent categories.
4. Compute the chi-square test statistic (formula below).
5. Determine the critical value using your significance level ($\alpha$) and degrees of freedom, or find the p-value directly.
6. Compare and make your decision: reject or fail to reject $H_0$.

Test Statistic in the Chi-Square Distribution

The goodness-of-fit test statistic follows a chi-square distribution. The formula is:

$\chi^2 = \sum_{i=1}^{k} \frac{(O_i - E_i)^2}{E_i}$

• $O_i$: Observed frequency for category $i$
• $E_i$: Expected frequency for category $i$
• $k$: Number of categories

Each term in the sum measures how much one category deviates from expectation. Squaring the difference means both overcounts and undercounts contribute positively to the statistic. Dividing by $E_i$ scales each term so that a deviation of 5 matters more when you only expected 10 than when you expected 500.

Degrees of freedom: $df = k - 1$

You lose one degree of freedom because the expected frequencies must sum to the total sample size. If you also estimate parameters from the data (for example, estimating $p$ for a binomial or $\mu$ for a Poisson), you lose an additional degree of freedom for each estimated parameter. So if you estimate one parameter, $df = k - 2$.

Interpretation of Goodness-of-Fit Results

The goodness-of-fit test is always a right-tailed test. A small $\chi^2$ value means observed and expected frequencies are close, which supports the null. A large $\chi^2$ value means the data deviates substantially from the hypothesized distribution.

Two equivalent decision methods:

• Critical value approach: Reject $H_0$ if $\chi^2 >$ critical value; fail to reject if $\chi^2 \leq$ critical value.
• P-value approach: Reject $H_0$ if p-value $< \alpha$; fail to reject if p-value $\geq \alpha$.

When you reject $H_0$, you have sufficient evidence that the data does not follow the specified distribution. When you fail to reject $H_0$, you lack sufficient evidence to conclude the data deviates from the distribution. Failing to reject does not prove the distribution is correct; it only means the data is consistent with it.

Additional Considerations

• Sample size matters. Larger samples give the test more power to detect real departures from the hypothesized distribution. With very small samples, the chi-square approximation becomes unreliable, which is why the expected frequency of at least 5 per cell condition exists.
• Effect size measures the magnitude of the discrepancy between observed and expected frequencies, independent of sample size. A common measure for goodness-of-fit is Cramér's V or simply reporting the chi-square value relative to $n$.
• Frequency tables (not contingency tables, which involve two variables) are used to organize the observed and expected counts for a single categorical variable in a goodness-of-fit test.