Interview Prep · ML Systems Lab

The 6-Step Framework That Answers Any ML System Design Question

Most ML system design interviews fail at step zero: the candidate jumps to model architecture before clarifying what success looks like, what latency the system can tolerate, or whether it's even an ML problem worth solving. The 6-step framework exists to prevent exactly this. Here's how it works, with a full worked example.

An ML system design interview tests whether you think like a senior engineer — someone who understands that model selection is step 5, not step 1. The framework below is a checklist that forces the right order of reasoning.

Step 1: Clarify the objective

Before anything else: what is the system trying to optimise for? And is that the same as the business metric?

The classic trap: "optimise click-through rate." CTR is easy to maximise — show clickbait. The real objective is something like "increase long-term user satisfaction and retention." These require different models, different labels, different evaluation metrics. State this explicitly.

Questions to ask: What is the north star metric? What does failure look like? What is the traffic volume? What is the acceptable latency? What is the cost of a false positive vs a false negative?

Step 2: Define the ML task

Given the objective, what type of ML problem is this? This sounds obvious but gets surprisingly complex:

Recommendation: retrieval (ANN search over candidates) + ranking (point-wise vs pair-wise vs list-wise scoring)

Fraud detection: binary classification, but with extreme class imbalance and adversarial dynamics

Search ranking: learning-to-rank with query-document pairs

ETA prediction: regression with right-skewed distribution and temporal features

Content moderation: multi-class classification with a "human review" third class

State the task type, the output type, and whether you need a single model or a pipeline of models.

Step 3: Define labels

How do you know what "correct" looks like? This is where most designs fail.

For recommendation: explicit (5-star rating) vs implicit (click, dwell time, share). Implicit labels are noisier but available at scale. Define your positive/negative events carefully — a 2-second dwell is not a positive signal.

For fraud: the label is available (fraud confirmed / not), but delayed (chargebacks arrive 30 days after the transaction). Your training data is 30 days stale by definition. State this.

For content moderation: you need human annotators. What is your labelling agreement threshold? How do you handle ambiguous cases? What is your label latency (time from content publication to label availability)?

Step 4: Feature design

Now you can talk about features. By this point you know the task, the labels, and the constraints — so feature design is guided by actual requirements, not intuition.

Structure your feature discussion around: user features (demographics, historical behaviour, preferences), item features (content embeddings, metadata, popularity signals), context features (time of day, device, session context), and interaction features (user-item affinity, historical clicks on similar items).

For each feature, state: availability at serving time? Point-in-time correct? Any leakage risk? This is where many candidates reveal whether they've actually worked with ML data pipelines.

Step 5: Model choice

Only now do you choose a model. Given everything above: the task type, the latency requirement, the label quality, the feature space, and the training data volume.

Common mapping:

Low latency + tabular features: GBM (LightGBM, XGBoost) — fast inference, interpretable

Retrieval stage: two-tower neural network — handles dense embedding spaces

Ranking stage: transformer over user-item sequence — captures sequential patterns

Small data + interpretability needed: logistic regression with handcrafted features

Large-scale NLP: fine-tuned BERT or task-specific instruction-tuned LLM

State your training infrastructure assumptions: how often does the model retrain? Online learning or batch? What is the training compute requirement?

Step 6: Evaluation and serving

Offline evaluation: what is your test set? Is it a time-ordered hold-out? What metric? Why that metric and not the alternatives?

Online evaluation: shadow mode first, then canary (5–10% traffic), then champion-challenger framework for promotion. What are your rollback triggers?

Serving: latency budget, caching strategy (can you pre-compute scores for a subset of users?), failover behaviour (what does the system serve if the model is unavailable?), monitoring.

Worked example: news feed ranking

Step 1: Objective — maximise long-term user engagement, not pure CTR. North star: weekly active days. Step 2: Task — list-wise ranking over candidate posts for a user. Retrieval (ANN) + ranking (transformer). Step 3: Labels — implicit: 10s+ dwell = positive, scroll past = weak negative, share = strong positive. Labels available in realtime. Step 4: Features — user embedding (historical engagement), post embedding (BERT on text), user-post affinity, recency decay, source reliability signal. Step 5: Model — two-tower for retrieval, 6-layer transformer ranker for top-100 candidates. Retrain daily. Step 6: Evaluation — NDCG@10 offline; A/B on weekly active days online with 2-week exposure minimum.

The interview test: can you do this in 45 minutes, and does your answer reveal that you've thought about what happens when the model is wrong?

```python from dataclasses import dataclass from typing import Literal

@dataclass class Feature: name: str available_at_serving: bool # exists at inference time (not post-hoc)? point_in_time_correct: bool # no future data bleeds into training rows? source: Literal['realtime', 'batch_precomputed', 'derived']

def step4_audit(features: list) -> None: """Call this before training, not after debugging a production incident.""" for f in features: flags = [] if not f.available_at_serving: flags.append("SERVING SKEW — not available at inference") if not f.point_in_time_correct: flags.append("LEAKAGE — future data contaminates training rows") print(f"{'✓' if not flags else '✗'} {f.name}: {flags or 'OK'}")

# News feed ranking — step 4 audit step4_audit([ Feature('user_click_history_7d', True, True, 'batch_precomputed'), Feature('post_engagement_rate', True, False, 'derived'), # leakage! Feature('avg_session_duration', False, True, 'batch_precomputed'), # serving skew Feature('realtime_trending_score', True, True, 'realtime'), ]) ```

Continue interactively

Read this post inside ML Systems Lab — with Simplify toggle, interview Q&As, inline glossary, and the MLE Path forward pointer.

Open in MSL →