Mediation Analysis
Direct vs indirect effects, NDE/NIE decomposition, sequential ignorability
Job training increases earnings. How much of the effect is direct — training improves skills and directly raises wages — versus indirect — training leads to employment, which raises earnings? This is a mediation question: decomposing the total causal effect into the portion that flows through a mediator (employment) versus the direct path. The distinction matters for policy. If employment mediates the effect, job placement services might be as effective as skills training at lower cost. If the effect is direct, the content of the training matters and placement alone is insufficient.
Definitions. Total Effect (TE) = Direct Effect (DE) + Indirect Effect (IE). Natural Direct Effect (NDE) = Y(t, M(t′)) − Y(t′, M(t′)): the effect of treatment holding the mediator at the value it would take under control. Natural Indirect Effect (NIE) = Y(t, M(t)) − Y(t, M(t′)): the effect of the mediator shifting from its control-level to its treatment-level value while holding treatment fixed.
Baron-Kenny (classical approach):
- Regress outcome on treatment. - Regress mediator on treatment. - Regress outcome on both treatment and mediator.
The indirect effect equals the coefficient of treatment on mediator multiplied by the coefficient of mediator on outcome. Widely used and intuitive — but requires no unmeasured confounding of the mediator-outcome relationship, a strong assumption that an RCT does not guarantee. The RCT randomizes treatment, not the mediator.
Counterfactual approach (Imai, Keele, Tingley): estimates the Average Causal Mediation Effect (ACME) under sequential ignorability — treatment is as-good-as-randomized given pre-treatment confounders (as in Baron-Kenny), and the mediator is additionally as-good-as-randomized given treatment and those same confounders (the assumption Baron-Kenny simply asserts without naming). Because that second piece is rarely guaranteed even in a clean RCT, the approach pairs its ACME estimate with a sensitivity analysis to assess robustness to unmeasured mediator-outcome confounding. More rigorous than Baron-Kenny because it names and stress-tests the assumption Baron-Kenny leaves implicit.
What mediation analysis is not: controlling for the mediator in a regression. Controlling for M in a regression estimates the controlled direct effect, not the natural direct effect — and introduces collider bias if the mediator and outcome share an unmeasured common cause. M is a post-treatment variable. Any unmeasured variable that affects both M and Y creates a backdoor path through M that conditioning on M opens rather than closes. Mediation requires a structural model, not just adding an extra covariate to the regression.
Key points
- Draw the full DAG with treatment, mediator, outcome, and all confounders before choosing an identification strategy. Mediation analysis is only valid if you can block the backdoor paths into the mediator-outcome relationship — which requires specific covariates and assumptions about what was measured. A mediation result without a DAG is a regression coefficient with a mechanistic label, not a causal decomposition.
- Trap: interpreting mediation results without checking the no-unmeasured-mediator-outcome-confounding assumption. If there is any unmeasured variable that affects both the mediator and the outcome, Baron-Kenny gives biased direct and indirect effects. Run the Imai-Keele-Tingley sensitivity analysis — it tells you how strong the hidden confounding would need to be to overturn your conclusion. Without it, you are presenting a mechanism decomposition that cannot be defended.
- Diagnostic: if the sum of your direct and indirect effects does not equal the total effect from a simple treatment-on-outcome regression, check for specification error. This accounting identity — TE = NDE + NIE — is a basic consistency check on your mediation model. A violation means either the outcome model, the mediator model, or both are misspecified in a way that breaks the decomposition.
Mediation requires a stronger assumption than total effect estimation — an RCT guarantees T-ignorability but not M-ignorability, and every mediation result needs sensitivity analysis for unmeasured mediator-outcome confounding.
Recap
- Mediation decomposes the Total Effect into Direct + Indirect: how much of training's effect on earnings flows *through* a mediator (employment) versus *directly* (skills raising wages) — the split matters because it changes the policy (placement services vs training content).
- NDE = Y(t, M(t′)) − Y(t′, M(t′)): the natural direct effect — the effect of treatment while holding the mediator fixed at the value it would take under control.
- NIE = Y(t, M(t)) − Y(t, M(t′)): the natural indirect effect — the effect of shifting the mediator from its control-level to its treatment-level value while holding treatment fixed.
- Baron-Kenny (three regressions) is intuitive but leans on a strong assumption: it requires *no unmeasured confounding of the mediator–outcome relationship*, and an RCT does *not* provide it — indirect effect = (T→M coefficient) × (M→Y coefficient).
- An RCT randomises T, not M: so a mediator–outcome confounder survives randomisation — this is exactly why clean treatment randomisation still doesn't buy you a valid mediation decomposition.
- Controlling for M in a regression is *not* mediation: it gives the *controlled* direct effect, and because M is post-treatment, conditioning on it opens collider bias if M and Y share any unmeasured common cause. Mediation needs a structural model, not one extra covariate.
- Always run the Imai-Keele-Tingley sensitivity analysis: it reports how strong an unmeasured mediator–outcome confounder would have to be to overturn your NDE/NIE — without it, the decomposition can't be defended.
Check your understanding
Q1. A new recommendation algorithm (T) increases 7-day retention (Y) by 5 percentage points. You suspect mechanism is session length (M). Select the two true statements about a valid mediation analysis here.
- A) Estimate NDE and NIE (e.g. via the mediation package with bootstrap CIs) rather than three plain regressions, because sequential ignorability for M is a separate assumption from T's ignorability and can still fail even inside a clean RCT
- B) Before trusting the decomposition, check whether session length could be a collider or a descendant confounded with retention through an unmeasured path, and whether the T-M-Y relationship is genuinely linear or needs a VanderWeele-style interaction decomposition
- C) Simply run three regressions — Y on T, M on T, and Y on T and M — and report proportion mediated as the coefficient drop over the total effect; no additional assumptions are needed here since the treatment itself was randomized
- D) Mediation is not identifiable in this setting at all, because session length and retention are measured over the same window, and that simultaneity alone rules out any valid decomposition without extra instrumental-variable assumptions
Q2. You control for a variable M in a regression of Y on T, and the coefficient on T drops from 0.4 to 0.1. Your colleague concludes '80% of the effect of T on Y is mediated by M.' What questions would you ask?
- A) Ask only whether the sample size is large enough — a 0.3 point coefficient drop is meaningless without a confidence interval, and a large standard error alone is treated as enough to overturn the mediation claim entirely
- B) Ask whether M is a mediator or a confounder, whether sequential ignorability holds for M, whether M is instead a collider, and whether linear Baron-Kenny is appropriate given a possible T×M interaction
- C) Ask only whether M was measured before or after T — post-treatment timing is treated as sufficient on its own to confirm mediation, while pre-treatment timing is treated as sufficient on its own to rule it out entirely
- D) Ask only whether T causes M — a significant T-on-M coefficient in a separate regression is treated as sufficient by itself to validate the whole mediation interpretation, with no further questions about M's role ever needed
Q3. Your treatment T was assigned by a clean randomized experiment. Why can the mediation decomposition (NDE/NIE) through mediator M still be biased?
- A) Randomization guarantees ignorability of T but not of M — an unmeasured mediator-outcome confounder survives randomization entirely intact and can still bias both the NDE and the NIE
- B) It cannot actually be biased in this setup — randomizing T automatically forces the NDE and NIE to be unbiased too, so no separate assumption about M is ever required once T is randomized
- C) The mediator here is measured after treatment, and that post-treatment timing alone guarantees reverse causation between M and Y, which breaks the decomposition regardless of study design or measurement
- D) RCTs only ever recover average effects, and mediation instead requires individual-level counterfactuals, which are fundamentally unidentifiable in literally any experimental or observational study design
Try it interactively
ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.
Open ML Systems Lab →