Multi-Task & Multi-Objective Ranking
Value models, shared-bottom vs MMoE, combining CTR + dwell + shares
Real feeds don't rank by one thing. A staff-level ranker predicts several outcomes at once — probability of click, of a long dwell, of a share, of a report — and then a value model combines them into the single score that decides the order. "Rank by engagement" is not a design; "rank by 1.0·p(click) + 1.2·p(dwell) + 0.5·p(share) − 3.0·p(report)" is.
One model, many heads. A shared-bottom network learns common representations, then splits into task-specific heads (click head, dwell head, share head). It's cheap but forces all tasks to share one trunk — when tasks conflict (clickbait maximizes clicks but minimizes dwell), the shared trunk is pulled in opposite directions and every task suffers (negative transfer). MMoE (Multi-gate Mixture-of-Experts) fixes this: several expert sub-networks, and each task has its own gate that softly picks which experts to use — so conflicting tasks can route to different experts.
The value-model weights are a product decision, not a learned parameter. They encode what the business values: how much a share is worth relative to a click, how hard to penalize a report. There's no weight vector that maximizes every objective — pushing CTR up promotes clickbait and raises the report rate. The weights are usually tuned by online A/B tests against a north-star metric (long-term retention), not by offline loss.
Every head must be calibrated, not just correctly ranked, or the sum lies. The value model adds the heads together as if each pᵢ were a real probability on the same 0–1 scale. Say the true click probability for an item is p(click) = 0.1, but the click head is uncalibrated and outputs 0.2 — that head's contribution to the value score is now double what it should be, silently outweighing a correctly-calibrated dwell head standing right next to it in the same sum. A head can have perfect ranking accuracy (it sorts items in the right order) and still wreck the value model this way, because ranking accuracy only cares about order, not the actual magnitude of the probability.
Guardrails ride in the same score. Harm signals (report, "see fewer", hide) enter the value model as *negative* weights, so harmful-but-clicky content is demoted at ranking time rather than filtered after the fact.
Key points
- Multi-task ≠ multi-objective. The model predicts multiple heads; the value model combines them. Keep them separate conceptually: the heads are learned (p(click), p(dwell), …); the combination weights are chosen to encode business value and tuned online.
- Shared-bottom is cheap but suffers negative transfer when tasks conflict; MMoE's per-task gates route conflicting tasks to different experts. Symptom of negative transfer: adding a task *lowers* another task's metric versus training it alone. MMoE (or PLE — Progressive Layered Extraction, a variant that stacks shared and task-specific expert layers instead of MMoE's single shared layer) is the standard fix at scale.
- Value-model weights are tuned online, not offline. Offline loss can't see long-term retention or harm. Weights are calibrated by A/B tests against a north-star metric — which is why every prediction head must be *calibrated* (a probability, not just a rank score) for the weighted sum to be meaningful.
Staff-level ranking predicts several calibrated outcomes with a multi-task model (MMoE routes conflicting tasks to separate experts) and fuses them with a value model whose weights are a business decision tuned by online A/B tests — with harm signals entering as negative weights so guardrails live inside the ranking score.
Recap
- Real feeds rank by a value model, not one metric: "rank by engagement" is not a design; "rank by 1.0·p(click) + 1.2·p(dwell) + 0.5·p(share) − 3.0·p(report)" is. A staff-level ranker predicts several outcomes, then the value model combines them into the single score that decides order.
- Multi-task ≠ multi-objective — keep them separate: the *multi-task model* learns multiple heads (p(click), p(dwell), p(share), p(report)); the *value model* is the weighted combination of those heads. Heads are learned; combination weights are chosen.
- Shared-bottom is cheap but negative-transfers when tasks conflict: one trunk feeding all heads gets pulled in opposite directions (clickbait maximizes clicks but minimizes dwell), so every task suffers. Tell of negative transfer: adding a task *lowers* another's metric vs training it alone. MMoE/PLE (PLE = Progressive Layered Extraction, a variant that stacks shared and task-specific expert layers on top of MMoE's single shared layer) fixes it — several expert sub-networks with per-task gates that route conflicting tasks to different experts.
- Value-model weights are a business decision, tuned online, not a learned parameter: they encode what a share is worth vs a click, how hard to penalize a report — and no weight vector maxes every objective (pushing CTR up promotes clickbait). Tune them by online A/B against a north-star (long-term retention), not offline loss.
- Heads must be calibrated or the weights lie: the weighted sum treats each pᵢ as a real probability with comparable scale — an uncalibrated head that outputs 2× true probability has its effective weight doubled. Guardrails ride the same score as negative weights on harm signals (report, "see fewer", hide), so harmful-but-clicky content is demoted at ranking time, not filtered after.
Check your understanding
Q1. A shared-bottom model jointly trains click and dwell heads. Adding the click task *lowers* dwell-head accuracy versus training dwell alone. What's happening and what's the standard fix?
- A) Overfitting — the added click task brings extra parameters; adding dropout specifically to the dwell head fixes it.
- B) Negative transfer — click and dwell conflict, pulling the shared trunk in opposing directions. MMoE routes conflicting tasks to separate experts.
- C) Label leakage from the click signal seeping directly into dwell labels; removing all click-derived features from the dwell head resolves it completely.
- D) A learning-rate mismatch between heads; giving each head its own optimizer makes the conflict disappear entirely.
Q2. Why must each prediction head be *calibrated* before the value model combines them as w₁·p₁ + w₂·p₂ + …?
- A) Calibration only matters for the primary click head; every other head is free to output uncalibrated raw rank scores.
- B) The weighted sum treats each pᵢ as a real probability; an inflated head (say, 2× true probability) silently doubles its effective weight.
- C) Uncalibrated heads cause numerical NaNs to silently appear inside the weighted dot product computation during standard GPU inference passes.
- D) Calibration isn't required at all — sorting by the weighted sum is mathematically invariant to any per-head scaling.
Q3. Select the two correct statements about how value-model weights (click, dwell, share, report) should be set.
- A) They should be treated as a product decision, tuned via online A/B tests against a north-star metric like 30-day retention.
- B) Offline click loss alone can't see long-term value or harm, so grid-searching a ranking metric like NDCG (Normalized Discounted Cumulative Gain, an offline ranking-quality score) on the click label alone isn't sufficient.
- C) Fitting them by minimizing offline cross-entropy on historical logs is sufficient, since the logs already encode the true optimum.
- D) Setting the report weight to the exact negative of the click weight and leaving the rest at 1.0 gives the optimum by symmetry.
Try it interactively
ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.
Open ML Systems Lab →