Recommender System Overview

Retrieval vs ranking vs diversity, the staged funnel, the data flywheel

A single ranking model cannot solve the scale problem and the quality problem at once. Score 10 million items with a good model in under 50ms? At even 1ms per item that's 10,000 seconds — you're off by five orders of magnitude.

So every large recommender is a funnel. Retrieval (candidate generation) narrows millions to thousands with fast approximate methods; ranking orders the survivors with an expensive precise model; re-ranking layers diversity, freshness, and business rules on top. Each stage is a different engineering tradeoff, and the ordering is forced: the cheap stage must run over everything, the expensive stage only over what the cheap stage kept.

Recall at retrieval is a ceiling you can't raise later. An item retrieval drops is gone — no downstream stage can rank an item it never received. That's why retrieval optimizes recall and ranking optimizes precision: they're different objectives because they sit at different points in the funnel.

Re-ranking exists because "most relevant" and "best final list" aren't the same thing. A ranker sorted purely by predicted engagement will happily fill the whole list with near-duplicates of one dominant interest — each individually well-scored, but repetitive as a set. Re-ranking corrects for that after ranking, not instead of it: it caps how many near-duplicate items can sit together, injects freshness the ranker alone would never favor, and applies business rules (e.g., no two ads back-to-back) on top of the ranker's precision-ordered list. Skip re-ranking and the ranker's raw output ships as-is — accurate item-by-item, but monotonous overall.

The data flywheel is why incumbents are so hard to displace. More users → more interaction data → better models → more engagement → more users. Much of that model quality comes from collaborative filtering — inferring what a user will like from patterns across *other* users' interactions (people who watched X also watched Y), not from anything intrinsic to the item itself. A new entrant with no interaction history can't run collaborative filtering at all and must limp along on content features until it accumulates a base.

Cold start is the flywheel's edge case, and it has a specific playbook. A user with zero watch history can't be served by collaborative filtering, so the system leans on what it does have: context (device, time of day, location), content features (the item's own attributes, not who else liked it), and a popularity fallback (globally or regionally trending items) to make the first few recommendations reasonable. From there, every watch-time signal in the session — a 2-second skip vs. a 30-second watch — updates a real-time embedding for that user, so personalization sharpens within the same session rather than waiting for a next login.

Key points

• The funnel exists because accuracy and scale can't be one model. Retrieval must be fast and high-recall (a missed item is unrecoverable); ranking can be expensive and precise because it only sees hundreds of candidates. Different objectives → different architectures → different stages.
• Retrieval recall caps final quality. If retrieval's recall@1000 is 0.7, then 30% of the items a user would have loved are already gone before ranking starts — and no amount of ranking sophistication recovers them. Diagnose a "great ranker, mediocre results" system by auditing retrieval recall first.
• Re-ranking curates the ranked list, it doesn't re-score it. A ranker optimizing pure predicted engagement can fill an entire list with near-duplicates of one dominant interest — 10 videos from the same creator, each well-scored individually but monotonous as a set. Re-ranking caps near-duplicates, injects freshness, and layers business rules on top of the ranker's output after the fact.
• The data flywheel compounds the incumbent advantage. Collaborative signal (patterns across other users' interactions) requires interaction history; a cold platform has none, so it underperforms an incumbent even with identical architecture until it accrues data. Exploration is the deliberate cost that keeps the flywheel fed with signal on new items.
• Cold start leans on context + content + popularity, then adapts fast. With no watch history to run collaborative filtering on, a first session opens on context, content features, and a trending fallback; early watch-time signals (a skip vs. a long watch) then update a real-time embedding within that same session, so personalization sharpens without waiting for a next login.
• Pure exploitation collapses the catalog, not just short-term diversity. Always serving the highest-predicted-engagement item narrows the system onto a shrinking set of popular items and starves the long tail of the signal it would need to ever be scored well, so coverage falls over time even as short-term clicks look fine — see *cold_start_system_design* and *recsys_feedback_loops* for the full collapse mechanism and the exploration-based fix.

Recap

• One model can't do scale AND quality at once: scoring 10M items with a good model at ~1ms each is ~10,000s — five orders of magnitude past a 50ms budget. That impossibility is why every large recommender is a funnel, not a single model.
• The funnel is three stages, each a different tradeoff: retrieval (candidate generation) cheaply narrows millions → thousands optimising *recall*; ranking runs an expensive precise model over the survivors optimising *precision*; re-ranking layers diversity, freshness, and business rules on top. The cheap stage must run over everything; the expensive stage only over what the cheap stage kept.
• Retrieval recall is a ceiling you can never raise downstream: an item retrieval drops is gone — no ranker can score an item it never received, so if recall@1000 is 0.7, 30% of items the user would have loved are already lost. Tell: "great ranker, mediocre results" → audit retrieval recall first, before touching the ranker.
• The data flywheel is why incumbents are hard to displace: more users → more interaction data → better models → more engagement → more users. A cold platform has no interaction history, so it can't run collaborative filtering at all and must limp along on content features until it accrues a base.
• Exploration is a deliberate, non-optional cost: pure exploitation (always serve the highest predicted-engagement item) collapses onto a shrinking set of popular items, starves the model of signal on new/long-tail items, and narrows users — coverage falls even as short-term clicks look fine. Exploration trades a little engagement now to keep the flywheel fed.

Check your understanding