Observational Causal Inference

Propensity score matching, IPW, doubly robust estimators, covariate balance

You want to know if a job training program increases earnings. You cannot randomize — people self-select into the program. People who join are more motivated, have higher baseline earnings, and are younger. A simple comparison of treated versus untreated overstates the program's effect because the treated group would have earned more anyway. Observational causal inference tries to recover the treatment effect without randomization, by making treated and control groups comparable using measured covariates.

Matching finds treated units and control units with identical or similar covariate profiles. Match on observed confounders — age, education, income, location — and the matched comparison removes their confounding. Propensity score matching compresses this into one dimension: the propensity score e(X) = P(T=1 | X=x) is a balancing score. Matching on e(X) balances all observed covariates simultaneously (Rosenbaum-Rubin theorem). Estimate e(X) with logistic regression, then match on the estimated scores and verify balance. Concretely: suppose the fitted model gives a 45-year-old non-participant and a 45-year-old participant with the same education and prior earnings the same estimated e(X) = 0.20 — that shared score is what makes the pair a valid match. Before matching, the treated group's mean age is 29 and the control group's is 34, with a pooled standard deviation of 8, so SMD = (29 − 34) / 8 = −0.63 — badly imbalanced. After matching on e(X), the treated mean age is 30.1 and the matched-control mean is 30.4, so SMD = (30.1 − 30.4) / 8 = −0.04, comfortably under the 0.1 threshold used later in this module.

Weighting is the continuous analog. Inverse Probability Weighting (IPW) weights each treated unit by 1/e(X) and each control unit by 1/(1−e(X)), creating a pseudo-population where treatment is uncorrelated with covariates. Take that same e(X) = 0.20: a control unit's weight is 1/(1 − 0.20) = 1.25, while a treated unit's weight is 1/0.20 = 5 — five times the pull of a typical unit. Push e(X) toward 0 or 1 and this ratio explodes, which is exactly why near-certain propensity scores produce the extreme weights (into the hundreds) that can dominate an IPW estimate. Doubly robust estimators (AIPW) combine regression adjustment with IPW and are consistent if either the outcome model or propensity model is correctly specified — not necessarily both. One wrong model is survivable; both wrong is not.

Common support is the region where both treated and control units exist with nonzero probability. Outside common support, inference requires extrapolation. Trim the sample to the region of common support before analysis and report what was trimmed.

What observational methods cannot do: remove confounding from unmeasured covariates. PSM, IPW, and AIPW are unbiased only if ignorability holds — only if all common causes of treatment and outcome are in X. An unmeasured confounder like motivation biases the estimate regardless of how sophisticated the estimator. No amount of covariate adjustment compensates for a variable you did not measure. The best observational studies acknowledge this explicitly and conduct sensitivity analysis for the residual unmeasured confounding — for example, computing the E-value: the minimum strength (on the risk-ratio scale) an unmeasured confounder like motivation would need, above and beyond the measured covariates, to fully explain away the observed effect. A large E-value means an unmeasured confounder that strong is implausible; a small one means the result could easily be an artifact of exactly the confounder the critic named.

Key points

• After propensity score matching, check covariate balance with standardized mean differences (SMD) — SMD < 0.1 for each covariate indicates good balance. Never report matching results without a balance table. If balance is poor for any covariate, re-specify the propensity score model — add polynomial terms, interactions, tighten the matching caliper (the maximum allowed propensity-score distance between a matched treated-control pair — tightening it discards poor matches at the cost of sample size), or switch to entropy balancing which directly optimizes balance rather than going through a propensity score. Skipping the balance check and assuming matching worked produces a biased estimate with no error message.
• Trap: matching on post-treatment variables. If the variable you are matching on was determined after treatment assignment, it can introduce collider bias — conditioning on a variable that is a common effect of both the treatment and the outcome creates a spurious statistical association between them where none exists causally, biasing the estimate — or it can block the causal path you want to measure. Match only on pre-treatment covariates. Always verify whether each covariate was determined before or after treatment began before including it in the propensity model.
• Diagnostic: if common support is very limited — less than 30% overlap between treated and control propensity score distributions — you can only estimate the treatment effect for a narrow subpopulation. Report this limitation explicitly. The estimate is not ATE for the full population; it is ATE for the overlap population, which may be quite different from the policy target. Extrapolating beyond common support is pure model assumption, not empirical comparison.

Recap

• Goal without randomisation: make treated and control groups comparable using *measured* covariates, so the remaining outcome difference reflects treatment rather than baseline differences (a naive treated-vs-untrained comparison overstates the effect because the treated would have done better anyway).
• The propensity score e(X) = P(T=1|X) is a balancing score: matching or weighting on this single number balances *all* observed covariates simultaneously (Rosenbaum-Rubin) — estimate it (e.g. logistic regression), match on it, then verify balance.
• IPW is the continuous analog: weight each treated unit by 1/e(X) and each control by 1/(1−e(X)) to build a pseudo-population where treatment is uncorrelated with the covariates.
• Doubly robust (AIPW) combines regression adjustment with IPW: it stays consistent if *either* the outcome model *or* the propensity model is correct — one wrong model is survivable, both wrong is not.
• Always check balance after matching: standardized mean difference (SMD) < 0.1 per covariate; never report a matching result without a balance table, or you're presenting a biased estimate with no error message.
• Common support / overlap: trim to the region where both treated and control units exist — outside it, any estimate is pure extrapolation, not empirical comparison, and the estimand becomes the overlap population.
• The hard ceiling: none of this removes bias from an *unmeasured* confounder (e.g. motivation) — no estimator, however sophisticated, fixes a variable you didn't measure. Report it and do a sensitivity analysis (E-value).

Check your understanding