Difference-in-Differences and Regression Discontinuity: When You Can't Randomise

Randomised experiments are the gold standard. But you cannot always randomise — policy changes happen company-wide, product launches happen by geography, and historical decisions were made without your consent. Difference-in-differences and regression discontinuity are the two most important quasi-experimental designs for estimating causal effects when randomisation is unavailable. Used correctly, they can provide identification as credible as an RCT.

When you cannot randomise, you need a research design that exploits natural variation in treatment assignment to identify causal effects. Two designs dominate applied causal inference in industry and economics: Difference-in-Differences (DiD) and Regression Discontinuity Design (RDD). Both rely on credible assumptions; knowing when those assumptions hold is the skill.

Difference-in-Differences

DiD estimates the causal effect of a treatment by comparing the change in outcomes for treated units (before and after treatment) to the change for untreated units over the same period. The treated-untreated difference in changes removes confounds that affect both groups equally.

Setup: you have panel data (multiple units observed over multiple time periods). Some units receive treatment starting at time T; others never do. DiD estimate: (Y_treated,after - Y_treated,before) - (Y_control,after - Y_control,before). In regression form: Y_it = α + β_1 Treated_i + β_2 Post_t + β_3 (Treated_i × Post_t) + ε_it. The coefficient β_3 is the DiD estimator — the causal effect of treatment.

The parallel trends assumption

DiD requires that in the absence of treatment, treated and control units would have evolved in parallel — the same trends. This is the parallel trends assumption. It is not directly testable (you cannot observe the counterfactual trend for treated units). The standard check: plot pre-treatment time trends for treated and control groups. Parallel pre-trends are necessary (though not sufficient) evidence for parallel trends post-treatment.

Violations: the treated and control groups differ systematically in ways that produce different trends. Example: you analyse the effect of a minimum wage increase in some states vs others. If high-wage-growth states disproportionately passed the minimum wage, their outcome trends would have diverged even without the policy.

Event study plots

An event study plots the estimated effect of treatment at each time relative to treatment onset (t=-k, ..., -1, 0, 1, ..., +k). Pre-treatment coefficients should be near zero (supporting parallel trends). Post-treatment coefficients show the dynamic effect over time. A spike exactly at t=0 that grows over time is the pattern consistent with a true treatment effect with ongoing adoption.

Regression Discontinuity Design

RDD exploits a threshold in a continuous "running variable" that determines treatment assignment. Units just above the threshold receive treatment; units just below do not. Near the threshold, units are nearly identical — the assignment is approximately random. Comparing outcomes just above and just below the threshold gives a causal estimate.

Example: a company gives a bonus to sales reps who exceeded 100% of quota in Q3. Reps at 99% of quota vs 101% of quota are nearly identical in ability and circumstances — but only the 101% group receives the bonus. Comparing their Q4 performance estimates the causal effect of the bonus.

Formal estimate: compare E[Y | running_var = c+ε] - E[Y | running_var = c-ε] as ε→0. In practice, fit a polynomial regression on each side of the cutoff separately and extrapolate to the cutoff. The discontinuity in fitted values is the RDD estimate.

Fuzzy RDD: imperfect compliance

Sharp RDD: everyone above the threshold is treated; everyone below is not. Fuzzy RDD: treatment probability jumps at the threshold but is not 0/1. The threshold is used as an instrument for treatment (see Post 81 on IV). The Fuzzy RDD estimate is a Local Average Treatment Effect (LATE) — the effect for the compliers (units whose treatment changes as a result of crossing the threshold).

Bandwidth selection and manipulation

The key tuning parameter in RDD is bandwidth: how far from the cutoff to include observations. Wider bandwidth = more data = lower variance, but greater risk of confounding (units far from the cutoff are less comparable). Narrower bandwidth = less data = higher variance, but stronger identification. The Imbens-Kalyanaraman bandwidth selector is the standard data-driven approach.

Manipulation check: if agents can precisely control the running variable, they may bunch just above the threshold (to receive treatment). This invalidates the RDD. McCrary density test detects discontinuities in the distribution of the running variable at the cutoff. If the density is discontinuous, there is manipulation.

Try on Colab: simulate a DiD study — generate two groups with parallel trends pre-treatment; add a treatment effect starting at t=5 for one group. Fit a two-way fixed effects regression and recover the treatment effect. Then add a pre-trend violation (different slopes before treatment) and observe the bias. For RDD: simulate a running variable with a cutoff; generate potential outcomes with a jump at the cutoff; estimate the effect using local linear regression on each side.