Difference-in-Differences
Parallel trends, event studies, staggered DiD, DiD failures
A city passes a minimum wage law in 2020. You want to know if it reduced employment. Treated group: businesses in the city. Control group: businesses in a neighboring city without the law. Pre-period: 2018–2019. Post-period: 2020–2021. Simple before-after comparison for the treated city would confound the policy effect with COVID-related employment drops that hit both cities.
DiD subtracts the control city's change from the treated city's change: (treated post − treated pre) − (control post − control pre). This removes the common time trend, leaving only the differential change that appeared when treatment was administered.
The parallel trends assumption is the key identifying assumption. In the absence of treatment, the treated group's outcomes would have followed the same trend as the control group. This is untestable in the post-period — the treated city's counterfactual employment trend under no policy is never observed. It can be tested in pre-periods: if treated and control trends were parallel in 2017 and 2018, they were likely to remain parallel in 2020 absent the intervention. Non-zero pre-period effects in an event study — a regression that estimates the treatment effect separately for each time period relative to treatment — are evidence against parallel trends.
Two-way fixed effects (TWFE) regression formalizes this: Y_it = α_i + λ_t + β D_it + ε_it. Unit fixed effects absorb permanent group differences; time fixed effects absorb common trends; β is the DiD estimate. This handles multiple periods and multiple treatment groups simultaneously.
Staggered treatment timing: when different units receive treatment at different times, TWFE produces biased estimates if treatment effects are heterogeneous across groups or time. Early-treated units act as implicit controls for later-treated units during periods when both are treated — but the early-treated units' outcomes already include treatment effects. The Callaway-Sant'Anna and Sun-Abraham estimators restrict the control group to not-yet-treated or never-treated units for each treatment cohort, producing unbiased estimates in staggered designs.
What parallel trends is not: a weak assumption that is always satisfied. Parallel trends fails when treatment was assigned based on pre-period trends (units selected for treatment because their outcomes were deteriorating), when confounding trends affect treated and control groups differently, when anticipation effects let units start reacting before the official treatment date (a foreseeable policy can shift behavior early, showing up as a pre-trend break right before adoption), or when the groups are fundamentally different in character. Always plot pre-period trends and test formally with an event study before reporting a DiD estimate — and if the pre-trend test fails, add unit-specific trends to the regression or switch to a synthetic control that reweights untreated units to match the treated group's pre-period path, rather than reporting the naive DiD anyway.
Key points
- Always plot an event study before reporting the DiD estimate — treatment effect estimated separately for each time period relative to treatment. The pre-period estimates should be near zero. This is the parallel trends diagnostic. Systematic non-zero pre-period estimates mean the treated and control groups were already diverging before the treatment, and the DiD estimate is confounded by that pre-existing divergence.
- Trap: using TWFE with staggered treatment timing when treatment effects vary over time or across cohorts. TWFE produces a weighted average of treatment effects that can be negative even when all individual effects are positive (Goodman-Bacon decomposition). Use Callaway-Sant'Anna for staggered designs — it constructs clean 2×2 DiDs for each treatment cohort using only not-yet-treated or never-treated units as controls.
- Diagnostic: if your DiD estimate changes substantially when you change the control group or the comparison period, the parallel trends assumption is fragile. Run placebo tests using outcome variables that should not be affected by the treatment. If there is a significant "effect" on placebo outcomes, something correlated with treatment is also driving the outcome — the DiD is picking up a confounded association, not the policy's causal effect.
DiD requires parallel trends — untestable post-treatment — and in staggered designs TWFE is biased even when parallel trends holds for every cohort, because early-treated units contaminate the control group for later-treated units.
Recap
- DiD = (treated post − treated pre) − (control post − control pre): subtracting the control group's change from the treated group's change removes the common time trend (e.g. a COVID drop hitting both cities), leaving only the differential change that appeared with treatment.
- Parallel trends is the key identifying assumption: absent treatment, the treated group's outcomes would have followed the same trend as the control's — the treated counterfactual is never observed, so this is untestable in the post-period.
- But you can test it in the pre-periods with an event study: estimate the effect for each period relative to treatment; pre-treatment coefficients should cluster near zero, and systematic non-zero ones are evidence the groups were already diverging (the estimate is confounded).
- TWFE formalises DiD: Y_it = α_i + λ_t + β D_it — unit fixed effects absorb permanent group differences, time fixed effects absorb common trends, and β is the DiD estimate across many periods and groups at once.
- Staggered treatment timing breaks TWFE when effects are heterogeneous: already-treated units act as *implicit controls* for later-treated ones while their own outcomes already include treatment effects, so TWFE can return the wrong sign even when every individual effect is positive (Goodman-Bacon decomposition).
- The fix — Callaway-Sant'Anna / Sun-Abraham: build clean 2×2 DiDs for each treatment cohort using *only* not-yet-treated or never-treated units as controls, then aggregate — unbiased in staggered designs.
- Placebo test: run the DiD on an outcome the treatment logically can't affect; a "significant" effect there means something correlated with treatment is driving the outcome — you're picking up a confounded association, not the causal effect.
Check your understanding
Q1. Your DiD estimate shows minimum wage increase reduced employment by 3%. A critic says the two groups had different pre-trends. How do you respond?
- A) Acknowledge the concern and simply vary the control group definition; if the -3% estimate stays roughly the same across a few different control choices, that alone is treated as confirmation parallel trends holds
- B) Add treated×period interactions to the TWFE regression and plot an event study; if pre-periods diverge, add unit-specific trends or synthetic control, and report the pre-trend test alongside the main estimate
- C) Re-run the DiD using a much shorter pre-period window, since pre-trend tests built on many pre-periods are inherently overpowered and will reject parallel trends even when it approximately holds in reality
- D) The -3% estimate is automatically valid as long as treatment assignment is as-good-as-random conditional on county and time fixed effects — pre-trends become entirely irrelevant once those effects are in the model
Q2. You are evaluating a product feature rolled out to user cohorts in January, March, and May. You plan TWFE DiD with January as treatment group and March/May as controls. Why is this problematic?
- A) TWFE strictly requires a never-treated control group by definition; because March and May cohorts are eventually treated, no valid DiD estimate can exist at all for any staggered-rollout design of this general kind
- B) The January cohort is simply too small to function as a treatment group, since TWFE requires roughly balanced treatment and control group sizes in order to produce a genuinely unbiased estimate here
- C) Using January as the treated cohort requires pre-period data before January to exist; without it, the parallel trends assumption can never be tested and the whole estimate becomes unreliable by default anyway
- D) TWFE uses already-treated March/May cohorts as implicit controls for January, and with heterogeneous effects this can flip the sign — fix with Callaway-Sant'Anna, using only not-yet-treated units as controls
Q3. A policy raising fuel efficiency standards was adopted by California in 2005 and no other state. Select the two genuine threats to parallel trends when using other US states as DiD controls.
- A) California is structurally different in geography, demographics, and political environment from most other states, so its emissions trend may have been diverging from the control states for reasons entirely unrelated to the 2005 policy
- B) Anticipation effects are plausible — automakers and consumers in California may have begun adjusting behavior before 2005 if the policy was foreseeable, which would show up as a pre-trend break right before the official adoption date
- C) The only real threat to parallel trends is that there are just 49 potential control states; simply adding more treated states elsewhere would resolve any parallel trends concern purely by increasing statistical power
- D) There is no genuine threat to parallel trends here at all, since California's adoption decision was clearly driven by state-level political factors that have nothing to do with national emissions trajectories
Try it interactively
ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.
Open ML Systems Lab →