When Difference-in-Differences Breaks: Parallel Trends Violations in Practice
DiD is the workhorse of causal inference in industry. It is also routinely applied incorrectly. The parallel trends assumption — that treatment and control groups would have followed the same trajectory absent the treatment — is untestable by definition. Here are the four ways it breaks in practice and what to do about it.
What the Parallel Trends Assumption Actually Means
Difference-in-differences estimates the causal effect of a treatment by comparing the change in outcomes for a treated group against the change in outcomes for a control group over the same time period. The identifying assumption — parallel trends — states that absent the treatment, the treated group would have followed the same trajectory as the control group.
This is not the same as "the two groups had similar trends before the treatment." That is a testable claim about pre-treatment data. Parallel trends is a claim about the counterfactual: what would have happened to the treated group if it had not been treated. That is, by definition, unobservable.
The pre-treatment trend test is evidence consistent with parallel trends, not proof of it. The parallel trends assumption can hold even when pre-trends differ (if the difference is predictable and adjustable) and can fail even when pre-trends look identical (if a confounding event was about to affect only the treatment group).
With that framing, here are the four ways the assumption breaks in practice.
Failure Mode 1: Compositional Shift
The treatment changes who is in the treatment group. A pricing change, a feature launch, or a policy change can attract a different type of user or customer to the treatment condition in the weeks around the treatment date. These new entrants have different baseline trends than the original treatment group members. The measured post-treatment trajectory is a mix of the original group's response and the new group's baseline behavior.
Sign to look for: track the demographic or behavioral composition of your treatment group in the 2–4 weeks before and after treatment. If age distribution, geographic distribution, acquisition channel mix, or engagement tier shifts meaningfully, compositional confounding is present. The DiD estimate conflates the treatment effect with the composition change.
Fix: restrict the analysis to users who were present in both pre- and post-treatment periods. This is a cohort-locked DiD — it eliminates compositional confounding at the cost of generalizability to newly acquired users.
Failure Mode 2: Anticipation Effects
Users or customers change behavior before the treatment takes effect because they know it is coming. A subscription price increase announced two weeks before it takes effect will cause early cancellations in the pre-treatment period. A new feature announced in a blog post will cause increased sign-ups before launch. These behavioral changes corrupt the pre-treatment baseline, making the pre-treatment trend for the treatment group look different from what it would have been without the announcement.
Sign to look for: pre-period trends diverge between treatment and control in the days or weeks immediately preceding the treatment date. This is distinct from a structural trend difference — it is a sudden divergence that begins at the announcement date.
Fix: extend the pre-period window far enough back to predate the announcement. Use a placebo test on an earlier period with no treatment and no announcement to verify that the groups were on parallel trajectories. If anticipation is unavoidable (e.g., regulatory changes require advance notice), adjust by modeling the announcement effect as a separate treatment.
Failure Mode 3: Concurrent Events
Something else happened to the treatment group around the treatment date that did not happen to the control group. A marketing campaign targeted the same user segment as the treatment. A competitor's outage drove traffic to a product used predominantly by the treatment group. A supplier disruption affected a geographic region that overlaps with the treatment condition.
This is the most common reason DiD estimates are wrong in industry settings. Experiments are never conducted in a vacuum. The treatment group is a real segment of a real business, and real businesses experience confounding events continuously.
Sign to look for: audit the event log for the treatment group in the ±4 week window around treatment. Marketing spend changes, sales campaigns, support incidents, product changes affecting the same segment — any of these can invalidate the parallel trends assumption.
Fix: there is no statistical fix for an unobserved concurrent event. The fix is operational: run a pre-analysis plan that requires an event audit before interpreting DiD results, and flag any correlated intervention as a confound. In post-hoc analyses, attempt to estimate the magnitude of the concurrent event separately.
Failure Mode 4: Different Cyclicality
Treatment and control groups have different weekly or monthly cycles. A B2B product used Monday through Friday by enterprise customers will have a very different day-of-week pattern than a B2C product used heavily on weekends. If treatment assignment correlates with product type or usage pattern, a DiD measured from Monday to Friday will have different seasonality baked in for the two groups. Averaging over these cycles without alignment introduces bias.
Sign to look for: plot the day-of-week profile for outcomes separately for treatment and control groups in the pre-period. If the shapes are different — one peaks on weekdays, one peaks on weekends — standard DiD will confound cyclicality differences with treatment effects.
Fix: align cohorts by day-of-week. Compute the DiD separately for each day of the week and aggregate. Or, aggregate both groups to weekly totals before taking differences, which averages out within-week cycles. Do not compare Tuesday-heavy treatment windows to Friday-heavy control windows.
How to Assess Parallel Trends Plausibility
Three practices make the parallel trends assumption more assessable:
Pre-trend test: regress the outcome on a time-trend variable, a group indicator, and their interaction, using only pre-treatment data. The coefficient on the interaction term should be statistically indistinguishable from zero. If it is not, there is a measurable pre-trend difference that the DiD assumption requires to extrapolate.
Placebo outcome test: apply the same DiD to an outcome that the treatment should not affect. If you are estimating the effect of a pricing change on revenue, also run the DiD on customer support ticket volume (which pricing should not change). A significant placebo effect suggests a confound is driving the result.
Event study plot: rather than a single pre/post comparison, estimate the treatment effect at each time period separately. Plot the coefficients. The pre-treatment coefficients should cluster around zero. A clean event study shows a flat pre-period and a step change at the treatment date. Divergence in the pre-period is a red flag.