Error Analysis

Confusion matrix drill-down, error slicing, systematic failures

An NLP intent classifier scores 94% accuracy. The product team is ready to ship. Before you sign off, you pull 200 of its mistakes and actually read them. The breakdown is stark: 67% are short 3-word commands ("turn on light"), 18% are code-switching (Spanish phrases mixed into English), 12% are negations ("don't turn on"), and the remaining 3% is a long tail of one-off causes (typos, out-of-vocabulary slang, garbled audio) too scattered to bucket individually. The model looks great on paper and fails on exactly the inputs your core users send most. That single accuracy number hid all of it.

That is what error analysis is for: turning "the model has 6% error" into "the model fails on 3-word commands, and here is why." The aggregate metric tells you a problem *exists*; error analysis tells you *which* group has it, *what kind* it is, and *what will fix it*.

Before you even sample individual errors, look at the confusion matrix — the table of predicted class vs. actual class counts. It's the fastest way to see *which* classes get confused with which: a spike in the "negation → positive" cell tells you exactly where to start sampling, before you've read a single example.

The five steps

1. Sample the errors — but not uniformly. Sort by confidence and look hardest at the *high-confidence* mistakes (the model said 0.95, the truth was the opposite). Those are not noise; they are the model confidently doubling down on a wrong pattern, and it will keep doing it.

2. Tag each error by cause. A bad training label? Not enough signal for this input type? A rare pattern with too few examples? Or genuine ambiguity that humans disagree on too? Drop every error into a bucket.

3. Count each bucket — both its error rate and how many of the total errors it accounts for.

4. Prioritise by impact × feasibility. A bucket that is 30% of your errors with a known fix beats one that is 5% with a mysterious cause. And weight by *cost*, not just frequency: those 12% negation errors, where the assistant does the exact opposite of what a user asked, may hurt far more than a mangled 3-word command.

5. Trace to root cause. Is it a *data* gap (no examples of this type), a *feature* gap (the model cannot even represent the pattern), or a *distribution* mismatch (production looks different from training)?

Why it pays off

Back to the classifier: two-thirds of the errors are short commands, because the model saw too few of them in training and its vocabulary came from longer queries. The fix is not a bigger model or a new architecture — it is collecting 200 labelled 3-word commands, retraining, and re-measuring that bucket's error rate. One cheap data effort clears the majority of the errors. That is the recurring lesson: systematic errors cluster by group or pattern and have targeted fixes, and collecting data for the worst bucket usually moves the metric more than any architecture change. (Errors that cluster by nothing are just irreducible noise — more data will not help those.)

And never let a high headline number end the conversation. A model that is 96% accurate but fails 100% of the time on one demographic, or a 98% spam filter that misses every email in one language, is not acceptable. Break errors down by subgroup, confidence, input length, and any business-critical slice. The aggregate metric is the *last* thing you report, not the first.

That "one demographic" case is worth naming precisely: error slicing asks *where* the model is wrong, on any slice, so you can fix it. Subgroup fairness analysis is the narrower, harm-focused case of slicing — it asks whether the model is wrong *more often* for a protected group (race, gender, age, disability status, and similar) in a way that causes real-world harm, even when that slice's raw error rate looks unremarkable next to a noisier one. Slicing becomes a fairness analysis the moment the slice in question is a protected group and the disparity carries real cost.

Small slices lie — check the support before you react

The most common error-analysis mistake is over-reacting to a tiny slice. "Group X has 40% error!" means nothing if group X has 5 examples — that's 2 errors, pure noise. Always report the slice size alongside its error rate, put a confidence interval on the rate (a 40% error on n=5 might really be anywhere from 5% to 85%), and set a minimum-support threshold (say, ignore slices under 30–50 examples) before drawing conclusions. A large slice with a modestly elevated error rate usually matters more than a tiny slice with a scary-looking one.

The sharper root-cause taxonomy

"Data / feature / distribution" is a good start, but name the fuller set so you can match a fix to each: label noise (the ground truth is wrong), data scarcity (too few examples of this pattern), distribution shift (production differs from training), feature blindness (the inputs can't even represent the distinction), annotation ambiguity (humans genuinely disagree, so no model can be "right"), a preprocessing bug (a pipeline error corrupts this slice), or a threshold/calibration issue (the model ranks fine but the cutoff is wrong for this group). Each points to a different fix — collecting more data won't cure a calibration bug or annotation ambiguity.

Slice false positives and false negatives separately

Don't lump all errors together — FPs and FNs usually have different causes and different costs. In fraud, the FNs (missed fraud) might cluster in a new merchant category while the FPs (false alarms) cluster in high-velocity legitimate users — two unrelated problems needing two different fixes. And their business costs differ (a missed fraud vs an annoyed customer). Always build the error breakdown twice, once for each error type.

Comparing two models with error analysis, not just their scores

When two models tie on the aggregate metric, error analysis is how you pick between them: compare their confusion matrices to see which classes each one fails on, compare their critical-segment slices, check whether their errors are high-confidence (confidently wrong) or genuinely uncertain, and check the correlation between their error sets. If the two models are wrong on different examples, their errors are uncorrelated and they are strong candidates to ensemble — combining them can cancel out each other's mistakes. If they are wrong on the same examples, ensembling buys you little.

From notebook to monitor

Error analysis shouldn't be a one-time notebook exercise you do before launch and forget. The slices you discover — 3-word commands, code-switching, new merchant categories — should become monitored production slices, tracked continuously so you catch when a slice's error rate creeps up after deploy. A finding that lives only in a notebook decays; a finding wired into monitoring keeps paying off.

The human side: labels and adjudication

When errors trace to *annotation ambiguity*, the fix is a labeling process, not a model change. Measure inter-annotator agreement — if two humans disagree on a slice, the model can't be blamed for missing it. Set up an adjudication step for disputed cases, update the labeling guidelines to resolve the ambiguity, and accept that some errors are "ambiguous but acceptable" — genuinely reasonable answers the rigid label just didn't credit. Not every error is a bug.

Prove the fix with a counterfactual check, and prioritise with the real formula

Before claiming a fix works, run an ablation: remove the suspicious feature, or add the targeted data, or apply the augmentation — then compare the *slice* metric before and after, not just the aggregate. And prioritise slices with more than a raw error count: impact = slice volume × error rate × cost per error × fix feasibility. A big, expensive, easily-fixed slice beats a small, cheap, mysterious one — the count of total errors alone will point you at the wrong work.

Key points

• Always stratify errors by confidence level first — high-confidence wrong predictions (model says 0.95 positive, it is negative) indicate systematic bias, not random noise. These are your priority. For the intent classifier: sample 200 errors. Group them into confidence buckets: errors where model confidence was 0.5–0.7, 0.7–0.9, and 0.9–1.0. High-confidence errors in the 0.9–1.0 bucket are the model doubling down on a wrong pattern — a systematic feature or data problem. Low-confidence errors near 0.5 are boundary cases where the model is appropriately uncertain. Fix the high-confidence bucket first. In sklearn: sort errors by abs(predicted_prob - 0.5) descending, inspect the top 30. They will cluster by a specific pattern almost every time.
• Trap: sampling errors uniformly and concluding that common categories matter most. A rare error category that happens to affect high-value users or safety-critical decisions matters more than a common category that does not. Weight by business impact, not frequency. For the intent classifier: 12% of errors are negation patterns ("don't turn on"). This seems small. But those are the errors where the assistant does the opposite of what was asked — a user explicitly said not to do something and the system did it anyway. The business impact of acting on a negation error is catastrophically higher than misclassifying a benign 3-word utterance. Prioritize by cost(error type) × frequency(error type), not frequency alone. The cost matrix is a business decision, not a modeling decision.
• Diagnostic: if error categories do not have obvious fixes, you have a data gap — collect 200 examples from the hardest category and retrain. This typically moves more metric than any architectural change. For the intent classifier: code-switching errors (Spanish phrases in English queries) represent 18% of all errors. The model has almost no Spanish-English mixed examples in training. The fix is not a larger model or a better architecture — it is 200 labeled code-switching examples added to the training set. Retrain. Measure the category error rate on a held-out slice of code-switching examples. If it drops from 60% to 20%, the data gap was the problem. If it stays high, there is a feature representation problem. Data collection is cheaper than architecture search and should come first.
• Check slice support, name the real root cause, and slice FP/FN separately. Don't react to a 40%-error slice of 5 examples — report slice size, a confidence interval on the rate, and a minimum-support threshold before concluding. Match the fix to the real cause: label noise, data scarcity, distribution shift, feature blindness, annotation ambiguity, preprocessing bug, or threshold/calibration issue — more data won't cure a calibration bug or genuine ambiguity. And build the breakdown twice, once for false positives and once for false negatives, since they usually have different causes and costs.
• Wire findings into monitoring, handle the human side, and prove fixes counterfactually. Turn discovered slices into continuously-monitored production slices rather than one-off notebook findings. When errors trace to annotation ambiguity, fix the labeling process — measure inter-annotator agreement, adjudicate disputes, update guidelines, accept "ambiguous but acceptable" errors. Before claiming a fix, ablate (remove feature / add data / augment) and compare the slice metric before vs after. Prioritise by impact = slice volume × error rate × cost per error × fix feasibility, not raw error count.

Recap

• The aggregate metric says a problem *exists*; error analysis says *which* group has it, *what kind*, and *what fixes it*. A 94%-accurate classifier can fail on exactly the inputs core users send most (3-word commands, code-switching, negations) — the single number hides all of it.
• Five steps: (1) sample errors non-uniformly, favouring *high-confidence* mistakes; (2) tag each by cause (bad label, no signal, rare pattern, genuine ambiguity); (3) count each bucket's error rate and share of total; (4) prioritise by impact × feasibility; (5) trace to root cause (data gap, feature gap, or distribution mismatch).
• High-confidence errors are systematic bias, not noise: when the model says 0.95 and the truth is the opposite, it's confidently doubling down on a wrong pattern and will keep doing it. Sort errors by |predicted − 0.5| descending, inspect the top ~30 — they cluster by a specific pattern almost every time. Fix these first.
• Weight by cost, not frequency: 12% negation errors ("don't turn on" → the assistant does the opposite) can hurt far more than 67% of mangled benign commands. Prioritise by cost(error type) × frequency, and the cost is a business decision, not a modeling one.
• A data gap usually beats an architecture change: if two-thirds of errors are short commands the model rarely saw in training, collecting 200 labelled 3-word commands and retraining clears most of the errors — cheaper and higher-impact than a bigger model. (Errors that cluster by nothing are irreducible noise; more data won't help those.)
• Small slices lie — check support before reacting: "group X has 40% error!" is meaningless if X has 5 examples (2 errors, a CI from ~5% to 85%). Report slice size, put a confidence interval on the rate, and set a minimum-support threshold (~30–50) before drawing conclusions. A large slice with a modestly elevated rate usually matters more than a tiny scary one.
• Slice FP and FN separately, wire findings into monitoring, prioritise with the real formula: FPs and FNs usually have different causes and costs (missed fraud clusters in a new merchant category; false alarms in high-velocity legit users) so build the breakdown twice. Turn discovered slices into monitored production slices, and prioritise by impact = slice volume × error rate × cost per error × fix feasibility — not raw error count.

Check your understanding