ML System Design · ML Systems Lab

How to Design a Recommendation System (The MLE Interview Framework)

Every senior MLE interview at Spotify, Netflix, Meta, or Airbnb eventually lands on a recommendation system design question. The surface area is enormous: candidate generation, ranking, serving, monitoring, cold start, exploration. Here's the framework that works.

There are six components to any recommendation system design. Miss one and you'll look junior. Cover all six and you look like someone who has shipped this before.

1. Problem framing (5 minutes)

This is where most candidates lose points immediately. They jump to architecture before establishing what the system is optimising for. Ask:

What is the target action? (play, purchase, watch, click, follow)

What is the serving context? (homepage, search results, email, push notification)

What are the latency constraints? (<100ms for real-time, <1s for near-real-time, minutes for batch)

What is "good"? (engagement? revenue? diversity? user satisfaction?)

These answers fundamentally change everything downstream.

2. Candidate generation (retrieval)

At scale (10M+ items), you cannot rank all items for every user. Retrieval narrows the candidate set to ~100–1000 items that are plausibly relevant.

Methods: Collaborative filtering (matrix factorisation), content-based (item embeddings), two-tower model (user embedding + item embedding → ANN lookup), popularity + filters.

Two-tower is the industry standard for large-scale retrieval. User tower: encodes user history, demographics, context. Item tower: encodes item metadata, engagement signals. Train with in-batch negatives. Serve by indexing item embeddings in FAISS or ScaNN.

3. Ranking

Given ~500 candidates, rank them. This is where you can afford more expensive models.

Options: GBDT (XGBoost/LightGBM) for tabular features — interpretable, fast, doesn't need GPUs. Deep neural networks for feature interaction learning. LTR (Learning to Rank) if you have explicit relevance labels.

Features for ranking: user-item interaction history, item quality signals (CTR, completion rate, rating), freshness, context (time of day, device, location), social signals (friend interactions).

4. Post-processing

Raw ranking scores are not what you serve. Apply:

Business rules (filter out-of-stock, NSFW, legal restrictions)

Diversity (maximum X% from same creator/category in top-20)

Freshness boost (promote new items to combat popularity bias)

Exploration injection (ε-greedy: 10% random items for data collection)

5. Serving infrastructure

Online serving: user embedding lookup from feature store → ANN retrieval → candidate features from feature store → ranker inference. Total budget: typically <100ms P99.

Key latency sources: feature lookup (~5ms Redis P50), ANN retrieval (~10ms with FAISS), ranker inference (~20ms GPU batch). Sum to 35ms with headroom.

6. Monitoring

Engagement metrics (CTR, completion rate, time spent) — leading indicators. Satisfaction metrics (ratings, explicit feedback, return visits) — lagging but reliable. Diversity metrics (ILD, coverage, long-tail ratio) — prevents filter bubbles. Data quality: feature freshness, null rates, embedding staleness. Model performance: prediction score distribution drift, AUC on logged feedback.

```python import numpy as np

class TwoTowerScorer: """Minimal two-tower scoring at inference time."""

def __init__(self, user_tower, item_tower): self.user_tower = user_tower # returns 128-dim embedding self.item_tower = item_tower # returns 128-dim embedding

def get_user_embedding(self, user_features: dict) -> np.ndarray: return self.user_tower.predict([user_features])[0] # (128,)

def score_candidates(self, user_emb: np.ndarray, item_ids: list) -> list[tuple]: """Dot-product scores for a candidate set. O(k * d).""" item_embs = self.item_tower.predict(item_ids) # (k, 128) scores = item_embs @ user_emb # (k,) ranked = sorted(zip(item_ids, scores), key=lambda x: x[1], reverse=True) return ranked # [(item_id, score), ...]

# Production note: item embeddings are pre-computed and stored in # an ANN index (HNSW/ScaNN). You never score the full catalog — # you retrieve top-1000 candidates, then rerank with a heavier model. ```

Continue interactively

Read this post inside ML Systems Lab — with Simplify toggle, interview Q&As, inline glossary, and the MLE Path forward pointer.

Open in MSL →