Difference-in-differences

Difference-in-differences is a method for estimating the effect of a public health intervention by comparing how outcomes change over time in a group that got the intervention and a group that did not.

What is difference-in-differences?

Difference-in-differences is a public health evaluation method that compares change over time in two groups, one exposed to a policy or program and one not exposed. Instead of asking only whether the treated group improved, it asks whether the treated group improved more than a similar comparison group did.

The basic idea is simple. You measure both groups before the intervention, measure them again after, and then subtract the change in the control group from the change in the treatment group. That extra subtraction helps separate the program effect from general trends that were happening anyway, like seasonal illness patterns, a new statewide campaign, or changes in the local economy.

This method shows up a lot in Intro to Public Health because so many real interventions are not randomized. A city might add smoke-free rules, a county might expand a vaccination campaign, or a school district might change lunch policies. You cannot always assign people randomly, so difference-in-differences gives you a practical way to estimate whether the policy actually shifted health outcomes.

The biggest assumption is parallel trends. That means that if the program had never happened, the two groups would have moved in the same general direction over time. If the groups were already on very different paths before the intervention, the comparison gets weaker because the method cannot cleanly separate the policy effect from pre-existing differences.

A quick example makes it easier to see. Suppose one city starts a lead-testing outreach program and a similar city does not. If blood lead levels fall in both cities, but they fall much more in the city with the program, difference-in-differences treats that extra drop as evidence that the outreach likely helped. If both cities fall by the same amount, the program probably did not add much beyond the broader trend.

Researchers usually pair this method with careful choice of a control group, time points that clearly come before and after the intervention, and sometimes graphs that show whether the groups really were tracking together before the change. It is not magic, but it is one of the most useful ways public health workers estimate causal effects from real-world data.

Why difference-in-differences matters in Intro to Public Health

Difference-in-differences matters in Intro to Public Health because it is one of the main ways you judge whether a policy or intervention changed population health, not just whether outcomes changed at all. Public health is full of messy real-world settings where you cannot run a clean lab-style experiment, so you need tools that can still make a careful causal argument.

It also connects directly to program evaluation. If a county launches a nutrition program and obesity rates shift, you still have to ask whether the program caused the change, or whether something else happened at the same time. Difference-in-differences helps you separate the program’s contribution from background trends, which is exactly the kind of reasoning public health courses want you to practice.

The method also teaches you to think about comparison groups. A weak comparison group can make a policy look better or worse than it really is. A strong one, with similar starting trends, gives you a much more believable estimate and makes your interpretation of health data more careful and defensible.

You will also see this idea when the course covers health policy, environmental health, and disease prevention. Anytime an intervention is rolled out in one place before another, difference-in-differences is a natural way to ask whether the rollout changed outcomes like smoking rates, vaccination uptake, emergency visits, or test results.

How difference-in-differences connects across the course

Control Group

The control group is the comparison group that does not receive the intervention. In difference-in-differences, it matters because you are not just comparing one group before and after, you are comparing that change against what happened in the control group over the same time period. A good control group makes the estimate much more believable.

longitudinal designs

Difference-in-differences depends on data collected at more than one time point, which is why it fits under longitudinal designs. You need at least a before and after measurement, and often more time points make the trend clearer. In public health, that lets you track whether changes happened gradually, suddenly, or not at all after a policy began.

propensity score matching

Propensity score matching and difference-in-differences are both used when random assignment is not available, but they solve the problem differently. Matching tries to make groups look similar at baseline, while difference-in-differences focuses on whether the groups were changing in similar ways over time. They can also be used together in stronger observational studies.

formative evaluation

Formative evaluation happens early, when a program is being designed or adjusted, while difference-in-differences is more of an outcome evaluation tool. If formative evaluation asks, 'Is this program set up well?', difference-in-differences asks, 'Did the program change outcomes once it was running?' They often appear in different stages of the same public health project.

Is difference-in-differences on the Intro to Public Health exam?

A quiz question or short-answer prompt may give you two groups, two time periods, and a health outcome, then ask you to interpret the change. Your job is to identify the treatment group, the control group, the before and after measurements, and whether the extra change in the treatment group supports a program effect. If a graph is included, you may need to check whether the pre-intervention trends look parallel. In a case study, you might explain why the method is stronger than simply comparing post-program rates, especially when a city, school, or county rolled out a policy at a specific time.

Difference-in-differences vs propensity score matching

These are often confused because both are used in observational public health research, but they do different jobs. Propensity score matching tries to balance groups on observed characteristics before comparing outcomes, while difference-in-differences compares changes over time and leans on the parallel trends assumption. Matching is about who you compare, and difference-in-differences is about how those comparisons change across time.

Key things to remember about difference-in-differences

• Difference-in-differences estimates the effect of a public health intervention by comparing how outcomes change in a treatment group versus a control group.

• The method is useful when random assignment is not possible, which happens a lot in policy and program evaluation.

• Its main assumption is that the groups would have followed parallel trends if the intervention had never happened.

• A strong control group and clear before-and-after data make the result much easier to trust.

• In public health, this method is often used to evaluate policies, campaigns, and community interventions that affect population health.

Frequently asked questions about difference-in-differences

What is difference-in-differences in Intro to Public Health?

Difference-in-differences is a way to estimate the effect of a public health intervention by comparing outcome changes in a group that got the intervention with changes in a group that did not. It is especially useful for policies and programs that roll out in the real world, where random assignment is not practical. The method asks whether the treated group changed more than the untreated group over the same time period.

How does difference-in-differences work?

You measure both groups before the intervention and after it, then compare the size of each group’s change. The treatment effect is the extra change in the treated group beyond whatever change also happened in the control group. That helps separate the intervention from outside trends that affected both groups.

What is the parallel trends assumption?

Parallel trends means that, without the intervention, the treatment and control groups would have kept moving in the same general direction over time. This is the main assumption that makes the method useful. If the groups were already diverging before the intervention, the result is harder to interpret.

How is difference-in-differences different from a control group comparison?

A simple control group comparison looks at differences between groups at one point in time, often after the intervention. Difference-in-differences is stronger because it compares changes over time, not just end results. That makes it better for public health settings where the groups may already have been different before the policy started.