Anomaly Detection

Isolation Forest, one-class SVM, LOF, autoencoder-based, evaluation without labels

Every day your e-commerce site processes a flood of transactions. 99.9% are ordinary. 0.1% are fraud — and the tricky part is that the newest fraud looks like nothing anyone has labelled before. You cannot just train a fraud classifier, because you have almost no fraud examples and the next attack will be one you have never seen. So you flip the problem: instead of learning what fraud looks like, learn what *normal* looks like, and flag anything that does not fit. That is anomaly detection.

There are a few different ways to define "does not fit," and each one is a different algorithm.

Statistical: live in the fat part of the distribution

Fit a distribution to your normal data (a bell curve, say) and flag any point that lands far out in the thin tail. Simple, and great when "normal" really does form a neat blob — but it struggles the moment normal has a lumpy, complicated shape.

Isolation Forest: anomalies are easy to fence off

This one has a genuinely clever insight. Pick a feature at random, pick a random value to split on, and cut the data in two. Keep cutting. A *normal* point is buried in a dense crowd, so it takes many random cuts to fence it off by itself. An *anomaly* sits out on its own, so just a couple of cuts isolate it. So the anomaly score is simply *how few cuts it took to isolate this point* — fewer cuts, more anomalous.

It is fast (scales to millions of points), makes no assumption about the shape of normal, and is the sensible default for tabular data.

Reconstruction and density: two more lenses

A reconstruction method (an autoencoder — the same architecture used for dimensionality reduction) learns to rebuild normal data, then flags anything it rebuilds badly. A density method like LOF (Local Outlier Factor) compares how crowded a point's neighbourhood is versus its neighbours' neighbourhoods — a point sitting in a sparse patch surrounded by dense ones stands out. The clever bit about LOF is that it judges density *locally*, so it can flag the odd point at the edge of a loose cluster without wrongly flagging an entire tight-but-small cluster.

Turning a score into a decision (and checking it works)

All of these produce a continuous *score*, not a yes/no. To act on it you pick a threshold — flag the top 1%, or flag as many as your review team can actually investigate, or tune it against whatever few labels you have. And do not fool yourself into thinking you need labelled anomalies to *train* — you do not, and that is the whole point. But grab even a tiny labelled set to *check* the detector: if the precision at your threshold is not at least 2× the base rate, the method is barely beating random and you should try another. A common practical check is Precision@50 — have an expert eyeball the top 50 flagged items and count how many are real.

One-class SVM: draw a boundary around normal

The method in the title deserves its own paragraph. A one-class SVM learns a boundary (in a kernel-lifted space) that *tightly encircles* the normal data; anything falling outside is an anomaly. With an RBF kernel it can wrap a nonlinear normal region, which makes it a good fit for small-to-medium, clean datasets where normal has a complex shape. The costs: it's very sensitive to scaling and to its hyperparameters — `nu` (roughly the expected fraction of outliers / margin softness) and `gamma` (kernel width) — and it scales poorly (roughly O(n²)–O(n³)), so it's the wrong tool for millions of points. Reach for it on modest, well-cleaned data; reach for Isolation Forest when volume or dimensionality is high.

Novelty detection vs outlier detection

A distinction that trips people up. Novelty detection assumes your training data is *clean normal* and you want to flag *new, unseen* anomalies at inference (one-class SVM and autoencoders are natural here). Outlier detection assumes the training data is *already contaminated* with anomalies and you want to find them *within* it (Isolation Forest and LOF are typically used this way). The difference decides both which method fits and how you must treat the training set — a novelty method fed contaminated "normal" data learns the contaminants and misses them later.

Time-series anomalies come in three kinds

Point-in-time methods miss a whole class of anomalies in sequences. Point anomalies are single wildly-off values (a sensor spikes to 1000). Contextual anomalies are values that are normal in general but abnormal *in context* (30°C is fine in summer, anomalous in December). Collective / sequence anomalies are a run of individually-normal values that together form an abnormal *pattern* (a heartbeat rhythm that's wrong as a shape). Isolation Forest and other per-point methods catch point anomalies but miss contextual and collective ones — those need sequence models (LSTM/Transformer autoencoders) that see the temporal context.

Scaling matters — mostly

Feature scale changes what "far" means. Distance-, kernel-, and reconstruction-based methods (LOF, one-class SVM, autoencoders) are scale-sensitive — an unscaled large-range feature dominates the distance and drowns the others, so standardise first. Isolation Forest is *less* scale-sensitive (it splits on random thresholds per feature), but it's still affected by feature quality and irrelevant noise features. Standardising is the safe default across the board; it's only truly optional for tree-based isolation.

The method-selection map

Match the tool to the situation. Isolation Forest — the scalable default for tabular data, no distributional assumption. LOF — when anomalies are *local* (sparse relative to a nearby dense cluster), but avoid it on very large datasets (O(n²)). One-class SVM — a clean, nonlinear boundary around normal on smaller data. Autoencoders — images, sequences, and nonlinear structure where reconstruction error is meaningful. Pick by data size, structure, and the *kind* of anomaly you expect.

Evaluating without labels, expanded

Since you usually can't label anomalies, evaluation is a craft. Synthetic anomaly injection (add known-weird points and check they're caught) is useful but caveated — synthetic anomalies may not resemble real ones, so a detector that aces synthetic can still miss real fraud. Precision@K expert audit (have a human review the top-K flags) gives a real precision estimate. Track the alert acceptance rate (what fraction of alerts investigators confirm), the incident-confirmation lag (truth arrives weeks later), and set up post-deployment label collection so confirmed incidents become a labeled set for retraining. No single label-free metric is trustworthy alone — triangulate.

The operational layer: alerts, not just scores

A production detector isn't a score, it's an alerting system, and that layer is where most of them fail. Deduplicate so one incident doesn't fire fifty alerts. Manage alert fatigue — too many false positives and investigators stop trusting the system, so a technically-good detector becomes useless. Add severity scoring and investigation queues so the worst cases surface first, wire a false-positive feedback loop back into tuning, and monitor for drift — "normal" shifts over time, so a detector calibrated last quarter silently degrades. The model produces scores; the operational layer decides whether anyone acts on them.

Key points

• Use Isolation Forest as your default anomaly detector for tabular data — it is fast (O(n log n)), handles high dimensions, requires no distributional assumption, and its key tunable parameter is the contamination rate (expected fraction of anomalies) — though n_estimators (number of trees) and max_samples (subsample size per tree) are also commonly adjusted. For time-series data with contextual anomalies, switch to an LSTM or transformer autoencoder that reconstructs sequences — Isolation Forest treats each point independently and misses anomalies that only appear anomalous in context.
• Trap: using per-feature Z-scores to flag anomalies. Multivariate anomalies — combinations that are individually normal but jointly unusual — are invisible to per-feature analysis. A transaction of 100 dollars at 3pm in New York is normal on each dimension; together they might be anomalous for a user who has never used the card outside California. Always use multivariate methods.
• Diagnostic: always test your anomaly detector on a small labeled holdout set, even if you cannot label everything. If precision at threshold is less than 2× the base rate, the method is barely better than random — try a different method or transform the feature space. Precision@50 (expert review of the top 50 flagged samples) is a practical operating metric when full labeled sets are unavailable.
• Know one-class SVM, the novelty/outlier split, and the time-series anomaly taxonomy. One-class SVM wraps a nonlinear boundary around clean normal data — good on small/medium sets, but scaling- and (nu, gamma)-sensitive and O(n²)–O(n³), so not for millions of points. Novelty detection assumes clean training data and flags new anomalies (one-class SVM, autoencoders); outlier detection assumes contaminated training data and finds anomalies within it (Isolation Forest, LOF). For time series, distinguish point, contextual (abnormal-in-context), and collective/sequence anomalies — per-point methods miss the latter two, which need sequence models. Standardise for distance/kernel/reconstruction methods (Isolation Forest is less scale-sensitive but still affected).
• Evaluate by triangulation and build the operational alerting layer. Without labels, combine synthetic anomaly injection (caveat: synthetics may not match real anomalies), Precision@K expert audits, alert acceptance rate, and post-deployment label collection for retraining — no single label-free metric is trustworthy alone. And a detector is an alerting system, not just a score: deduplicate alerts, manage alert fatigue (too many false positives and investigators stop trusting it), add severity scoring and investigation queues, feed false positives back into tuning, and monitor for drift since "normal" shifts over time.

Recap

• Learn normal, flag what doesn't fit — no fraud labels needed, catches unseen attacks.
• Isolation Forest = scalable tabular default: anomalies isolate in few random cuts, no distributional assumption.
• LOF flags locally sparse points; autoencoder flags what rebuilds badly; one-class SVM wraps a nonlinear boundary (small/clean data, $O(n^2)$–$O(n^3)$).
• Per-feature Z-scores miss multivariate anomalies — individually normal, jointly unusual.
• Score → threshold is a business choice; check Precision@K against base rate on a tiny labeled set.
• Novelty (clean training) vs outlier (contaminated training) decides method and data handling.
• Time-series anomalies: point, contextual, collective — per-point methods miss the last two; a detector is an alerting system, not a score.

Check your understanding