Autoencoders for Dimensionality Reduction

Encoder-decoder mechanics, bottleneck, VAE, anomaly detection via reconstruction error

Think about describing a friend's face to a sketch artist. You cannot list every pixel — you compress it into a handful of essentials ("round face, thick eyebrows, crooked smile"), and from those few words the artist rebuilds something recognisable. An autoencoder is a neural network that learns to do exactly this on its own: squeeze data through a narrow middle, then rebuild it from the squeezed version.

Take 28×28 MNIST digit images — that is 784 numbers each. An autoencoder has two halves. The encoder funnels those 784 numbers down through shrinking layers (784 → 256 → 64 → 32) into a tiny 32-number code. The decoder takes those 32 numbers and expands them back out (32 → 64 → 256 → 784), trying to reproduce the original image. You train the whole thing on a single goal: make the rebuilt image match the original as closely as possible.

The magic is in the squeeze. Because everything has to pass through that 32-number bottleneck, the network cannot just copy the input across — it is forced to keep only what matters for rebuilding, and throw the rest away. Those 32 numbers become a compact summary of the digit. And unlike PCA, which can only compress along straight lines, an autoencoder is a neural net, so it can learn *curved* structure — which is why its codes separate MNIST digits far better than PCA's do.

Two flavours worth knowing

A denoising autoencoder makes the job harder on purpose: it feeds in a *corrupted* input (add noise, or blank out some pixels) and asks the network to reconstruct the *clean* original. To pull that off, the encoder has to learn the real underlying structure rather than memorise surface detail — which makes it a popular way to pre-train representations when you have lots of unlabelled data.

A variational autoencoder (VAE) changes what the encoder outputs. Instead of one fixed code per input, it encodes to a little *cloud* of possible codes (a distribution), and adds a term to the loss that keeps those clouds tidy and continuous. That continuity is what makes a VAE *generative*: you can sample a fresh point from the latent space and the decoder turns it into a brand-new, plausible image. A plain autoencoder cannot do this — sample a random point from its latent space and you usually get garbage, because it never learned to fill the gaps between training examples. That is the key difference: VAEs are generative, plain autoencoders are not.

A bonus use: catching anomalies

Here is a neat trick that falls out of the design. Train an autoencoder only on *normal* data — normal transactions, healthy sensor readings. It becomes very good at rebuilding normal things. Now feed it something weird: because it never learned to compress that pattern, the rebuild comes out badly and the reconstruction error spikes. So a high reconstruction error flags an anomaly, for free.

But it only works if the bottleneck is sized right, and this is the whole game. Too *narrow* and the network cannot even rebuild normal data well, so everything looks anomalous. Too *wide* and the network has enough room to memorise *everything* — including the weird stuff — so nothing looks anomalous. The bottleneck has to be tight enough to force real compression, yet loose enough to reconstruct genuine normal data. Get that balance wrong and the anomaly detector fails silently.

The reconstruction loss must match the data

"Make the rebuild match the original" needs a *specific* loss, and the choice depends on the data. MSE (squared error) fits continuous real-valued data — sensor readings, standardised features. Binary cross-entropy (BCE) fits data in [0,1] like normalised pixel intensities or binary features. For images where pixel-perfect error misses perceptual quality, a perceptual loss (distance in a pretrained network's feature space) matches human judgement better. Use MSE on a [0,1] pixel target and it under-penalises blur; use BCE on unbounded sensor values and it's meaningless. Match the loss to the data type first.

The reparameterisation trick, properly

The VAE encodes to a distribution and then *samples* a code z ~ N(μ, σ²) — but sampling is random, and you can't backpropagate gradients through a random draw. The reparameterisation trick rewrites the sample as z = μ + σ·ε, with ε ~ N(0,1) drawn *outside* the computation graph. Now the randomness sits in ε (a constant for that step), while μ and σ are ordinary differentiable outputs, so gradients flow back to the encoder. This one rewrite is what makes VAEs trainable by gradient descent — worth being able to state, not just recognise.

The VAE loss, and where it breaks

A VAE's loss has two terms: a reconstruction loss (rebuild the input) plus a KL-divergence term that pulls each input's latent cloud toward a standard normal, keeping the latent space continuous and sampleable. The failure mode to name is posterior collapse: if the decoder is powerful enough to reconstruct without using the latent code, the KL term wins and the encoder outputs the prior for everything — the latent variables carry no information. β-VAE exposes a knob β on the KL term: β > 1 pushes toward more disentangled (but blurrier) codes, β < 1 toward sharper reconstruction with a messier latent space. Tuning β trades reconstruction against latent structure.

Setting the anomaly threshold

A reconstruction-error detector is useless without a cutoff, and you don't get one for free. The standard approach: after training on normal data, compute the reconstruction-error distribution on a held-out normal set and set the threshold at a high percentile (say the 95th or 99th) — accepting that percentage as your expected false-positive rate. If you have even a few labeled anomalies, tune the threshold on the precision/recall trade-off they give you instead. Either way, the cutoff is a deliberate business choice about false-positive rate versus miss rate, not a default.

The silent killer: contaminated training data

The whole method assumes training data is *pure normal*. If real anomalies hide in your "normal" training set, the autoencoder learns to reconstruct them too — so at inference they produce low error and slip through, and the detector fails without any warning. This is the most common reason a reconstruction detector misses known defects. Guard against it: clean the training set as best you can, or use robust training that down-weights high-error examples during training.

Low error isn't proof of normal (and high error isn't proof of anomaly)

Reconstruction error is a noisy signal in both directions. Low error can occur for a genuine anomaly the model happened to memorise, or one that's small relative to MSE dominated by other dimensions. High error can come from plain input noise, a scaling/preprocessing mismatch, a sensor dropout, or a rare-but-perfectly-valid sample — none of which are true anomalies. So treat reconstruction error as evidence to investigate, not a verdict, and always sanity-check flagged cases.

Architecture choices

The encoder/decoder shape should match the data. Dense (fully-connected) autoencoders suit tabular data; convolutional autoencoders suit images (they respect spatial locality); sequence autoencoders (LSTM or Transformer encoder-decoder) suit time series and text. Beyond the bottleneck size (the main knob), you regularise with dropout, weight decay, or an explicit sparsity penalty on the code (a sparse autoencoder). Reaching for a dense AE on images, or an oversized bottleneck with no regularisation, is a common way to get a detector that quietly memorises everything.

Key points

• A denoising autoencoder can pretrain representations when unlabeled data is abundant and labels are scarce — but it's not automatically the winner. The denoising objective forces more robust representations, and fine-tuning on the labeled subset *can* beat training from scratch when unlabeled data is plentiful — but "consistently outperforms" is too strong: modern contrastive/self-supervised methods (SimCLR-style, masked modeling) often produce better representations than a vanilla autoencoder. Treat AE pretraining as one option to benchmark, not a guaranteed win. The corruption level is a hyperparameter — start with 20–30% masking and tune on the labeled validation set.
• Trap: the reconstruction loss can be minimized by memorizing training examples rather than learning compact representations. Check that the latent space clusters by meaningful categories (not just by individual examples) using visualization before using the representations downstream. A latent space that looks like a random cloud when colored by class label has not learned useful structure.
• Diagnostic: if reconstruction loss is low but downstream task performance is poor, the autoencoder is encoding reconstruction-irrelevant information. Add a classification loss or explicit invariance (contrastive learning) to align the representation with the downstream task. Low reconstruction loss is a necessary but not sufficient condition for useful representations.
• Match the loss and architecture to the data, and know the VAE's two terms and its failure mode. Use MSE for continuous data, BCE for [0,1] data, perceptual loss for image quality; use dense AEs for tabular, convolutional for images, sequence (LSTM/Transformer) for time series/text, and regularise with dropout, weight decay, or a sparsity penalty. A VAE's loss is reconstruction + KL-to-prior, trained via the reparameterisation trick z = μ + σ·ε; watch for posterior collapse (decoder ignores the code) and use β-VAE's β to trade disentanglement against reconstruction.
• For anomaly detection, set the threshold deliberately and distrust contaminated data and raw error. Set the cutoff from the held-out normal error distribution (a high percentile = your false-positive budget), or tune on precision/recall if you have labeled anomalies. The method assumes pure-normal training data — real anomalies hiding in it get learned and reconstructed well, so they slip through silently; clean the data or down-weight high-error examples. And low error isn't proof of normal (memorised or MSE-swamped anomalies) nor is high error proof of anomaly (noise, scaling bugs, rare-but-valid samples) — investigate flagged cases, don't trust the score blindly.

Recap

• Encoder squeezes to a bottleneck, decoder rebuilds; trained to minimize reconstruction error.
• Bottleneck forces keeping only what matters — nonlinear compression, beats PCA on curved structure.
• Denoising AE: reconstruct clean from corrupted input → learns real structure.
• VAE is generative: encodes to a distribution + KL term; sample new points; plain AE cannot.
• Reparameterisation trick: $z=\mu+\sigma\cdot\varepsilon$ makes sampling differentiable; posterior collapse is the failure mode.
• Anomaly detection via reconstruction error — bottleneck sizing is the whole game (too wide memorizes, too narrow rejects normal).
• Match loss to data: MSE continuous, BCE [0,1], perceptual for images; contaminated "normal" training fails silently.

Check your understanding