Advanced 40 min read t-SNEUMAPvisualisationmanifold

t-SNE & UMAP

Perplexity, why t-SNE clusters mislead, UMAP vs t-SNE tradeoffs

You have 50,000 single-cell RNA sequences, each described by 20,000 gene expression values. You want a 2D visualization to see whether cell types cluster naturally. PCA gives a blurry projection where all cells overlap. t-SNE produces a crisp 2D visualization where distinct cell types form separated islands — and that visualization guided the discovery of a previously unknown rare cell population, 0.1% of cells with a distinct gene expression profile invisible in PCA.

t-SNE: computes pairwise similarities in high dimensions (Gaussian distribution of distances, perplexity parameter controls effective neighborhood size). Computes pairwise similarities in low dimensions (Student t-distribution — heavier tails prevent crowding). Minimizes KL divergence between the two similarity distributions via gradient descent. The heavy tails in low dimensions mean that moderate distances in high dimensions map to large distances in low dimensions — this creates the well-separated clusters that make t-SNE visualizations visually striking.

UMAP (Uniform Manifold Approximation and Projection): grounded in Riemannian geometry and topological data analysis. Preserves more global structure than t-SNE, faster (O(n log n) vs O(n²)), supports out-of-sample transformation (new points can be projected without refitting). Default choice over t-SNE for most use cases.

NOT-this: "t-SNE cluster distances are interpretable." The distances between clusters in a t-SNE plot are meaningless — a cell type that appears far from another in t-SNE might be close in the actual high-dimensional space. t-SNE preserves local neighborhoods but distorts global distances. Never interpret inter-cluster distances in a t-SNE plot. UMAP preserves more global structure but still warps distances.

Key points

Use UMAP over t-SNE for nearly all visualization tasks — it is faster, supports out-of-sample projection, and preserves more global structure while producing equally clear local cluster separation. The only reason to use t-SNE over UMAP is when existing analyses were done with t-SNE for direct comparison. For new work, UMAP is the default.
Trap: interpreting cluster sizes in t-SNE or UMAP as meaningful. Point density in the 2D projection does not correspond to point density in the original space — a small dense cluster in the plot might represent a small or large group in high dimensions. Report cluster sizes from the original space, not the projection. Visual size carries no quantitative meaning.
Diagnostic: if UMAP shows one undifferentiated blob, try increasing n_neighbors (too small → disconnected graph, no global structure) or decreasing min_dist (too large → all points crowded to center). If UMAP shows many tiny isolated clusters, n_neighbors is too small. Structures that persist across multiple n_neighbors values are more trustworthy than those that appear only at one setting.

Takeaway

t-SNE and UMAP are for looking at data, not for generating features — inter-cluster distances in t-SNE are deliberately distorted, UMAP's are approximate, and clustering on either's 2D output will mislead you at exactly the boundaries that matter most.

Recap

For looking, not for features — 2D visualization of nonlinear structure.
t-SNE: match high-D and low-D similarities, minimize KL; Student-t tails prevent crowding.
Inter-cluster distances are meaningless — deliberately distorted; never interpret them.
UMAP default over t-SNE: faster ($O(n\log n)$), preserves more global structure, out-of-sample transform.
Cluster sizes in the plot carry no quantitative meaning — report from original space.
Never run K-means on the 2D output — cluster on PCA-50, use UMAP for display only.
Perplexity / n_neighbors are knobs: trust structure that persists across settings.

Check your understanding

Q1. You run t-SNE with perplexity=5 and see 50 tiny, tight clusters. You run with perplexity=100 and see 3 blobs. Which two of the following are correct ways to figure out the "true" structure?

A) Run at several intermediate perplexities (15, 30, 50) and compare against a UMAP embedding of the same dataset
B) Cluster in the original high-dimensional space with k-means and check which structures persist across perplexities
C) Perplexity=5 is always more accurate, since lower perplexity reveals true local structure that high perplexity obscures
D) Perplexity=100 is always more accurate, since t-SNE requires high perplexity to capture meaningful global structure

Q2. A team visualises 50,000 single-cell RNA-seq measurements with t-SNE, sees 12 distinct clusters, and runs k-means on the 2D t-SNE embedding. Explain two problems with this approach.

A) t-SNE is far too slow for 50,000 cells to process, and k-means simply cannot handle raw RNA-seq data directly at all
B) The number of clusters should be set by BIC, not visual inspection; and k-means should use cosine distance for RNA-seq data
C) k-means on t-SNE uses distorted distances, wrong boundaries; the 12 clusters may be perplexity-dependent artefacts here
D) t-SNE does not work on high-dimensional data at all — PCA must be applied first, then k-means on the 2D output is valid

Q3. Your t-SNE plot shows two large clusters that are completely separated. Does this mean these two groups are very different from each other in the original space?

A) Yes — complete visual separation in a t-SNE plot always directly corresponds to large distance in the original high-D space
B) Yes — the heavy-tailed t-distribution t-SNE uses in low dimensions preserves inter-cluster distances proportionally too
C) No — t-SNE repulsive forces push separated groups apart regardless of true distance; verify with centroid distance or UMAP
D) Only if the exact same separation appears consistently at both low and high perplexity settings across multiple runs

Q4. You need to compress 768-dimensional BERT embeddings to 2D for visualisation and downstream k-means clustering. What is your pipeline and why?

A) Apply UMAP directly from 768D to 2D, then run k-means on the 2D output — UMAP preserves distances well enough for clustering here
B) Apply t-SNE for visualisation and k-means simultaneously — t-SNE is preferred since it handles dense manifold structure better
C) Apply PCA to 50D and cluster there, then apply UMAP to 2D separately for display only, colouring by the k-means labels
D) Run k-means directly in 768D — dimensionality reduction before clustering discards information and reduces clustering accuracy

Try it interactively

ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.

Open ML Systems Lab →