Model Calibration

Reliability diagrams, ECE, Platt scaling, isotonic regression

When a weather forecaster says "70% chance of rain tomorrow," something quietly impressive is going on. Look back over all the days they said 70%, and it really did rain on about 70% of them. Their stated confidence matches reality. That property has a name — calibration — and it is exactly what most machine-learning models do *not* have, even good ones.

Here is the gap. A model can be excellent at *ranking* — putting the sick patients above the healthy ones, high-risk loans above low-risk ones — and still be hopeless at *probabilities*. Suppose it stamps "90% chance of disease" on a group of patients, but only 60% of them actually turn out sick. The ranking is fine (those patients really are higher-risk than the ones it scored 50%), but the number 0.9 is a lie. And the moment you *use* that number — to price insurance, to decide a treatment, to feed another model — the lie costs you.

Two different questions

It helps to see that ranking and calibration answer different questions. Ranking (measured by AUC) asks: does the model put riskier cases above safer ones? Calibration asks: when the model says 0.7, does the thing happen 70% of the time? A model can ace one and flunk the other. A credit model might rank ten thousand applicants perfectly by risk yet lowball every probability — great for deciding who to approve, useless for estimating how much money you will lose. If your decision only needs the *order*, calibration may not matter. The instant it needs the actual *number*, it does.

How to see it: the reliability diagram

There is a simple picture that reveals miscalibration at a glance. Take all the model's predictions, sort them into buckets (everything it called about 0.1, about 0.2, and so on), and for each bucket plot the predicted probability against the *actual* fraction that turned out positive. If the model is calibrated, every point lands on the diagonal line where "predicted = actual." If the curve sags *below* the diagonal, the model is overconfident — it says 0.8 for things that happen only 0.6 of the time. If it rides *above*, the model is underconfident.

Miscalibration is the rule, not the exception — but the *shape* differs by model. Modern neural networks are famously overconfident: their reliability curves sag below the diagonal, saying 0.95 for things that happen 0.80 of the time. Random forests bend the *other* way: averaging many trees pushes probabilities *away* from 0 and 1 (a truly-positive case rarely gets every tree to vote yes, so the forest hesitates to say 0.99), giving a characteristic sigmoid-shaped curve — under-confident at the extremes, over-confident in the middle. Knowing your model's typical distortion tells you which correction to reach for.

The fix, and the one rule you cannot break

You usually do not retrain to fix calibration — you patch it afterward. Hold out a separate slice of data (a calibration set), see how the model's scores line up with reality on it, and fit a small correcting function that bends the scores back onto the diagonal. Two common choices: Platt scaling fits a simple sigmoid — fast, needs little data, and works when the miscalibration is a smooth one-directional bend. Isotonic regression fits a more flexible staircase that can straighten out any shape, but it needs more data (roughly a thousand-plus points) or it just memorises the calibration set.

And the one rule you cannot break: the calibration set must be separate from both training and test. Calibrate on the training data and you are correcting against numbers the model already memorised — the fix looks perfect and fails in the wild. Calibrate on the test data and you have spoiled your only honest measure of how good the model really is. Train, calibrate, and test on three different slices. To put a single number on how calibrated you are, people use the expected calibration error (ECE) — the average gap between the buckets and the diagonal, where zero is perfect.

ECE's blind spots

ECE is convenient but genuinely fragile, and interviewers probe this. It depends heavily on your binning: change the number of bins or use equal-width versus equal-count bins and the ECE number moves, sometimes a lot. It's biased by sample size (few points per bin makes the estimate noisy). Worst, it can hide local miscalibration — a model badly overconfident in one region and underconfident in another can post a small overall ECE because the errors average out. So don't reduce calibration to a single ECE number; always look at the reliability diagram, and consider class-conditional views.

The Brier score, and what it decomposes into

A more complete single number is the Brier score — just the mean squared error between predicted probabilities and outcomes ($\frac{1}{N}\sum(\hat{p}_i - y_i)^2$). Its value is that it splits into three meaningful parts (the Murphy decomposition): reliability (calibration — are the probabilities honest?), resolution (discrimination — do the predictions actually separate outcomes?), and uncertainty (the irreducible base-rate difficulty). This is why Brier is richer than ECE: a model can be perfectly calibrated (great reliability) but useless (zero resolution, it always predicts the base rate), and Brier catches that where ECE alone would look fine.

Temperature scaling — the neural-network default

For neural networks, the standard fix (from Guo et al., 2017) is temperature scaling: divide the logits by a single learned scalar T before the softmax. T > 1 softens overconfident probabilities toward the middle; T < 1 sharpens them. It's the simplest possible calibrator — *one* parameter fit on a validation set — and because it only rescales logits it leaves the ranking (and accuracy) completely unchanged while fixing the confidence. That single-parameter simplicity is exactly why it rarely overfits and became the go-to for deep models.

Calibrating more than two classes

Multiclass calibration is trickier and worth flagging. You can calibrate one-vs-rest (one calibrator per class) but then the per-class probabilities no longer sum to 1 and need renormalising. You also have to decide *what* you're calibrating: top-label calibration (is the model's confidence in its top prediction honest?) versus classwise calibration (is every class's probability honest?). Multiclass ECE has to pick one of these, which is why a single multiclass calibration number is even easier to misread than the binary one. Temperature scaling sidesteps some of this by scaling all logits together.

Calibration is not thresholding

Keep these two separate — they're often confused. Calibration fixes the *truthfulness* of the probability (0.7 should mean 70%). Thresholding picks the *decision cutoff* that turns a probability into an action, chosen from business costs. They're related — a well-calibrated probability makes threshold selection meaningful and transferable across contexts — but they're different steps. You calibrate so the number is honest, *then* threshold so the decision is optimal.

Calibration decays under drift

Finally, calibration is not permanent. A model calibrated on last year's data can drift out of calibration as the world changes — covariate shift (the input mix moves) or concept drift (the relationship changes) both break it, even though your original test-set calibration looked perfect. So calibration is something to *monitor* in production (track ECE or reliability over time), not a one-time fix at training. When a deployed model's probabilities start lying, drift is the usual cause.

Key points

• Calibration is whether the model's probabilities are literally true: when it says 0.7, does the thing happen 70% of the time? It is separate from accuracy and from ranking (AUC) — a model can rank cases perfectly yet report probabilities that are badly off. Calibration only matters when you actually *use* the probability: pricing, risk scores, medical decisions, or feeding another model in a stack. If you only need the ranking (who is riskier than whom), you can often ignore it. The moment a real number matters, check it.
• The trap: assuming a model's probabilities are trustworthy straight out of the box. Most are not. random forests are pushed *away* from 0 and 1 by tree-averaging (a sigmoid-shaped curve, under-confident at the extremes); modern neural networks are famously overconfident; SVMs do not really output probabilities at all. So do not read a raw score as a probability without checking. The check is a reliability diagram: bucket the predictions and compare each bucket's predicted probability to the actual rate. If the curve sags below the diagonal, the model is overconfident and needs a fix before its numbers can be trusted.
• The fix is a post-hoc patch on a separate calibration set — never the training or test set. Hold out a slice of data, see how the scores line up with reality on it, and fit a small correcting function: Platt scaling (a simple sigmoid, good when data is scarce and the bend is smooth) or isotonic regression (a flexible staircase, needs a thousand-plus points). Fit it on the calibration slice only. Calibrate on training data and the fix is fooled by memorised outputs; calibrate on test data and you have spoiled your honest score. Train, calibrate, and test on three different slices.
• Don't trust ECE alone — use the reliability diagram and the Brier score. ECE depends on binning choice and bin count, is noisy with few samples, and can hide local miscalibration (overconfident in one region, underconfident in another, averaging to a small number). The Brier score (mean squared error of probabilities) is richer because it decomposes into reliability (calibration), resolution (discrimination), and uncertainty — so it catches a perfectly-calibrated-but-useless model that always predicts the base rate. Always read the reliability diagram, not just a single scalar.
• Know temperature scaling, the multiclass subtleties, and that calibration ≠ thresholding — and decays under drift. Temperature scaling (Guo et al.) divides logits by one learned scalar T — the neural-network default, since it fixes confidence without changing ranking or accuracy. Multiclass calibration must choose top-label vs classwise and renormalise one-vs-rest outputs. Keep calibration (making the probability truthful) separate from thresholding (choosing the decision cutoff from costs). And calibration isn't permanent — covariate shift and concept drift break it even when the original test calibration looked perfect, so monitor ECE/reliability in production rather than treating it as a one-time fix.

Recap

• AUC = does it rank correctly. Calibration = are the probabilities literally true? Separate properties.
• Calibrated means: when it says 0.7, the thing happens ~70% of the time.
• Most models come out overconfident (random forests, neural nets).
• Whenever a decision uses the probability itself, plot the reliability diagram.
• Fix with Platt scaling or isotonic regression on a separate calibration set — never train or test.
• Don't trust ECE alone — pair it with the reliability diagram and the Brier score.
• Calibration ≠ thresholding, and it decays under drift.

Check your understanding