Fairness in ML: Disparate Impact, Equalized Odds, and What You Actually Measure in Production

Fairness is not one thing. Demographic parity, equalized odds, calibration, and individual fairness are mathematically incompatible — you cannot satisfy all of them simultaneously when base rates differ across groups. Understanding which definition to use and why is one of the hardest judgment calls in applied ML, and interviewers at FAANG companies ask about it constantly.

Fairness in machine learning is contested not because practitioners are careless, but because the word "fair" means different things to different people — and those differences are irreconcilable under certain data conditions. Every practitioner needs to know the main definitions, their relationships, and when each is appropriate.

Why fairness is hard: differing base rates

If the true positive rate (prevalence of the outcome) differs across demographic groups, many fairness definitions become mathematically incompatible. Chouldechova (2017) proved that when base rates differ: a classifier cannot simultaneously achieve equal false positive rates, equal false negative rates, AND be perfectly calibrated across groups. This is not a limitation of current algorithms — it is a mathematical impossibility. You must choose which fairness criterion to prioritise based on the domain and the costs of different error types.

Demographic parity (Statistical parity)

P(ŷ = 1 | A = 0) = P(ŷ = 1 | A = 1). The probability of a positive prediction is equal across groups A = 0 and A = 1. The 80% rule (EEOC in the US): if the selection rate for any group is less than 80% of the highest group's rate, disparate impact is presumed. Problem: demographic parity ignores whether the groups actually have different base rates for the outcome. If 30% of Group A and 10% of Group B qualify for a loan based on creditworthiness, demographic parity forces equal approval rates, which means approving unqualified Group B applicants or rejecting qualified Group A applicants.

Equalized odds

P(ŷ = 1 | Y = y, A = a) is equal across groups for both y = 0 and y = 1. The true positive rate AND the false positive rate must be equal across groups. This is a stronger condition: not just equal outcomes, but equal accuracy across groups. Weaker version — Equal Opportunity (Hardt et al., 2016): only require equal true positive rates (don't penalise equal false positives). The intuition: if you receive a positive prediction, your true label distribution should be the same regardless of group membership.

Calibration across groups

P(Y = 1 | ŷ = p, A = a) = p for all groups a. Among individuals assigned score p, approximately fraction p should be positive outcomes, in every subgroup. COMPAS (the recidivism prediction tool criticised in ProPublica's 2016 investigation) was well-calibrated — among defendants scored 7/10 for recidivism risk, approximately 70% did re-offend in both Black and white defendants. But the false positive rate was much higher for Black defendants. Both can be true simultaneously when base rates differ.

Individual fairness

Similar individuals should be treated similarly: if d(x_i, x_j) is small (individual-level distance), then |f(x_i) - f(x_j)| should be small. This avoids group-level aggregation but requires defining a meaningful similarity metric, which is non-trivial and domain-dependent.

Counterfactual fairness

A decision is counterfactually fair if, in a world where the protected attribute were different (holding everything else fixed causally), the outcome would be the same. This requires a causal model (DAG) of the data-generating process. Computationally expensive but captures the intuition that protected attributes should not causally influence decisions.

Proxy variables: the main practical problem

Removing protected attributes from the model does not achieve fairness. Other features (ZIP code, name, credit history, school) may be highly correlated with protected attributes and serve as proxies. Removing all proxies may destroy model performance. The right approach: measure disparate impact on outcomes, not just on inputs.

What to do in practice

Step 1: define the harm. Who can be harmed, in what direction, and at what cost? A false negative in loan approval (rejecting a creditworthy applicant) has different costs from a false positive in recidivism prediction (incarcerating an innocent person). Step 2: pick the fairness criterion that minimises the most serious harm. Step 3: measure. Use a fairness auditing library (Fairlearn, AI Fairness 360) to compute all criteria across demographic slices. Step 4: intervene at the right level. Pre-processing: reweight training data. In-processing: add fairness constraints to the loss function (Lagrangian relaxation). Post-processing: threshold adjustment per group (Hardt et al. equalised odds post-processor).

Intersectionality

Single-attribute fairness analysis misses intersectional groups. A model may be fair for women and fair for people over 40 separately, but unfair for women over 40. Test on intersections, especially for high-stakes decisions.

The interview angle

Interviewers at FAANG companies ask about fairness to test whether you understand that it is a value choice, not a technical optimisation problem. Good answers: name the competing definitions, explain the impossibility result, and describe how you would structure the trade-off given the specific domain's cost structure. Bad answers: "we use demographic parity" or "we just remove the sensitive feature."

Try on Colab: use the Adult Income dataset (UCI). Train a logistic regression model predicting income. Compute demographic parity difference, equalized odds difference, and false positive rate disparity across gender and race using Fairlearn. Apply Fairlearn's ThresholdOptimizer with equalized odds constraint. Report the trade-off between demographic parity and overall accuracy.