The Recommendation System Stack: Retrieval → Ranking → Re-Ranking

No production recommendation system is a single model. It is a funnel: retrieval narrows 100M items to 1,000; ranking scores those 1,000 with a feature-rich model; re-ranking applies business rules, diversity, and freshness constraints. Each stage trades off recall and precision differently. This is the full stack, with the engineering decisions that made YouTube, Netflix, and TikTok work at scale.

A recommendation system has one job: show the right item to the right user at the right time. At the scale of YouTube (800M videos, 200M daily users), doing this naively is physically impossible — you cannot score every item for every user in a latency budget of 100ms. The solution is a staged funnel that progressively narrows the candidate set while increasing prediction quality at each stage.

Stage 1: Retrieval — 100M → 1,000

Retrieval must be fast (< 20ms) and have high recall (the truly relevant items should be in the retrieved set). It does not need to rank items precisely — it just needs to not miss things. The two-tower architecture (see Post 70) is the standard: precompute item embeddings offline, at query time compute a user embedding and do approximate nearest-neighbour search.

Multiple retrieval sources are combined: collaborative filtering retrieval (users with similar histories liked this), content-based retrieval (items similar to the user's recent engagements), trending/fresh content (recent items with high early engagement), social graph retrieval (items engaged by people you follow). Each source contributes hundreds of candidates; the combined set is deduplicated to ~1,000.

Stage 2: Ranking — 1,000 → 50

The ranking model scores all retrieved candidates with a single unified model. Because it only processes ~1,000 items (not millions), it can use expensive features: dense text and image embeddings, user-item interaction features, contextual features, long user history, and cross-features (user demographic × item category interactions). The model is typically a deep neural network (DCN, DeepFM, or a Transformer-based interaction model) or gradient-boosted trees.

Training objective: predict engagement signals (click, watch duration, like, share, save) from features. Multi-task learning is common: one model head predicts click probability, another predicts watch duration, another predicts like probability. The final ranking score is a weighted combination: score = w1 * P(click) + w2 * E(watch_minutes) + w3 * P(like). The weights encode business priorities — a 10-minute watch is worth more than a click.

Stage 3: Re-ranking — 50 → final feed

The ranked list is modified before serving to enforce constraints that the ranking model optimised away: diversity (no more than 2 consecutive items from the same channel), freshness (inject a recent item even if the model scores it lower than older ones), policy compliance (remove items flagged by safety classifiers), serendipity (occasionally surface items outside the user's usual pattern to avoid filter bubbles), and business rules (sponsored content slots, promoted items).

Re-ranking is where the ML pipeline meets business logic. It is often heuristic or rules-based rather than learned, which makes it faster to iterate on but harder to optimise holistically.

The feedback loop: closing the system

User interactions with the surfaced items become training data for the next iteration of all three models. This feedback loop is what makes recommendations improve over time — and also what creates filter bubbles. If the ranking model learns primarily from clicks, it optimises for click-through rate rather than user satisfaction or content quality. YouTube's 2019 re-design explicitly added a "satisfaction" signal (post-watch survey scores) alongside engagement signals to address this.

Feature stores: the infrastructure that makes real-time features possible

Both the ranking model and the re-ranker need features computed in real time (user's last 5 actions, current trending items) and in batch (user's 30-day engagement history, item quality scores). Feature stores (see Post 77) provide low-latency access to precomputed features during serving, while ensuring consistency between training-time and serving-time feature values.

Cold start: the hardest problem

New users have no history. New items have no engagement statistics. Cold-start recommendations must rely entirely on content features, demographic priors, and early weak signals. Common approaches: user onboarding flow (ask preferences explicitly), content-based bootstrapping (recommend items similar to explicitly stated interests), and exploration policies (serve diverse content early to rapidly learn the new user's preferences).

TikTok's architecture: lightweight but effective

TikTok's architecture (as described in leaked documents) uses a comparatively lightweight candidate generation phase (~10,000 candidates from a pool including user-specific and global trending pools) followed by a very capable ranking model that uses video content features (sound, visual style, text) alongside collaborative filtering signals. The short video format reduces cold start: even a brand new video can go viral within hours based on early engagement signals, allowing the system to learn quickly.

Try on Colab: implement the full funnel on MovieLens-1M. Stage 1: train a two-tower model, retrieve top-500 candidates per user via FAISS. Stage 2: train a gradient-boosted ranker on (user, movie, context) features, score the 500 candidates. Stage 3: apply a diversity rule (max 2 movies per director in final 10). Evaluate recall@500 from retrieval and NDCG@10 from ranking separately, then end-to-end.