Uplift Modeling: Did Your Intervention Actually Cause the Outcome?

Standard predictive models answer "who will convert?" Uplift models answer "who will convert because of our intervention?" The difference is enormous. Targeting your highest-conversion-probability users with a discount might mean you are paying people to do something they were going to do anyway. Uplift modeling identifies the incremental impact — the customers who would not have converted without the treatment.

Every retention campaign, promotion, and marketing intervention has a hidden inefficiency: some portion of the people you target would have converted anyway. Targeting these "sure things" wastes budget and can damage relationships (a loyal customer who receives a discount every time they're about to churn learns to delay purchases until a discount arrives). Uplift modeling targets the people for whom the intervention actually matters.

The four customer segments

The Rubin potential outcomes framework defines four segments for any binary treatment (offer, email, intervention): Persuadables — would convert with treatment, would not without. Sleeping dogs — would convert without treatment, less likely to convert with (your email annoyed them). Lost causes — would not convert with or without. Sure things — would convert with or without. Standard ML models predict total conversion probability, which conflates all four groups. You want to identify and target only persuadables.

Individual Treatment Effect and the Fundamental Problem of Causal Inference

The Individual Treatment Effect (ITE) for person i is: ITE_i = Y_i(1) - Y_i(0) — the difference in outcome under treatment vs no treatment. The fundamental problem: you can only ever observe one of these potential outcomes. The same person cannot be both treated and untreated simultaneously.

Uplift modeling estimates ITE from observational or experimental data by modelling P(Y=1 | T=1, X) - P(Y=1 | T=0, X) — the difference in conversion probability between treated and untreated groups, conditional on features X.

Two-model uplift (T-learner)

Fit two separate models: one on the treated group (model_1: P(Y=1 | X, T=1)) and one on the control group (model_0: P(Y=1 | X, T=0)). Uplift score for each individual = model_1(X) - model_0(X). Simple to implement; estimates are biased when the treated and control groups are unbalanced (which they always are in observational data).

Meta-learners: S-learner, X-learner

S-learner: include the treatment indicator T as a feature in a single model. Uplift = f(X, T=1) - f(X, T=0). Can be biased when T is correlated with confounders. X-learner (Künzel et al., 2019): stage 1, fit outcome models on each group. Stage 2, compute imputed treatment effects D̃_i = Y_i - μ̂_0(X_i) for treated units and D̃_i = μ̂_1(X_i) - Y_i for control units. Stage 3, regress D̃_i on X to get a single uplift model. X-learner is more efficient with unbalanced treatment assignment.

Propensity score matching: controlling for selection bias

In observational data (no randomised experiment), treatment assignment is not random — it correlates with outcomes. Propensity score e(X) = P(T=1 | X) — the probability of receiving treatment given observed covariates. Matching: for each treated unit, find the control unit with the most similar propensity score. Compare their outcomes. The matched comparison approximates what you would observe if treatment had been randomly assigned. Inverse propensity weighting (IPW): weight each observation by 1/e(X_i) for treated, 1/(1-e(X_i)) for controls. This reweights the sample to look like a randomised experiment.

Both methods rely on the unconfoundedness assumption: all confounders are observed and included in X. If there are unobserved confounders, neither propensity matching nor IPW is valid.

Evaluating uplift models

Standard accuracy metrics (AUC, precision) do not apply — you never observe both Y(1) and Y(0). The Qini curve: sort users by predicted uplift descending, incrementally compute the actual uplift in each decile (comparing treatment and control outcomes within each decile). A good uplift model concentrates the true persuadables in the top deciles. Qini coefficient = area under the Qini curve, similar to AUC. An uninformative model has Qini = 0; a perfect model has Qini = 1.

Try on Colab: use the Criteo Uplift dataset (25M users, binary treatment/control, binary conversion). Train a T-learner uplift model (XGBoost for each arm). Compute per-user uplift scores. Plot the Qini curve — compare against a random targeting baseline and a naive conversion-probability baseline. Observe that the naive model (targeting high converters) has lower Qini than the uplift model.