The Cold Start Problem: Beyond Popularity Heuristics

Every recommendation system has a cold start problem. Most teams solve it by falling back to popularity rankings. This works for items. It doesn't work for users. And it leaves a lot of personalisation quality on the table from the very first session.

Cold start manifests in three forms, each requiring a different solution.

User cold start (new user, no history):

The wrong approach: show popular items. This creates a feedback loop — popular items get more exposure, accumulate more clicks, become more popular. You end up serving the same 50 items to every new user.

Better approaches:

1. Onboarding signals. Ask for explicit preferences (genre, interest, category). Even 3–5 data points enable personalisation. Spotify's onboarding artist selection reduces cold start significantly. The friction cost is worth the quality gain.

2. Contextual features. Browser/OS, time of day, location, UTM source, device type — all available before the first interaction. A new user arriving via a "Python tutorial" referral is not the same as one arriving from "deep learning paper."

3. Item-based fallback. Instead of "most popular overall," serve "most popular in your apparent context." New user, evening, mobile → entertainment; new user, weekday morning, desktop → professional content.

4. Meta-learning (few-shot). Train a model to quickly adapt to a user with few examples. MAML and its variants. Expensive to train but powerful for high-value users. Used at some scale by Netflix and Spotify for new premium subscribers.

Item cold start (new item, no interactions):

Content-based embeddings. Embed items using their metadata (title, description, category, author) using a pretrained encoder. New items immediately have a representation in the same space as established items. Quality improves as interactions accumulate.

Exploration budget. Allocate a fixed fraction of impressions to new items across all users, regardless of predicted CTR. Treat it as a multi-armed bandit problem — Thompson sampling or UCB will naturally allocate more to items that are accruing positive signals.

System cold start (new recommendation system):

This is the hardest form. You have no historical interactions at all.

Sequence of phases: (1) Popularity-based serving. Log everything. (2) Content-based model. Build from item metadata + contextual features. No interaction data needed. (3) Collaborative signals. Once you have enough interactions (typically 100k+ user-item events), matrix factorisation or two-tower starts outperforming content-based. (4) Hybrid model. Blend content-based scores with collaborative scores, weighting toward collaborative as interaction data grows.

The mistake is trying to jump to phase 3 too early. A collaborative model trained on 1000 interactions is worse than a well-designed content-based model.

```python def route_recommendation_request(user_id: str, interaction_count: int, catalog_metadata: dict) -> dict: """ Route cold vs warm vs hot users to the right model. Cold = < 5 interactions → content-based + UCB exploration Warm = 5–50 interactions → hybrid collaborative + content Hot = 50+ interactions → full collaborative filtering """ if interaction_count < 5: # Cold start: use item content features + explore broadly candidates = content_based_retrieval(catalog_metadata) # UCB score = estimated_reward + sqrt(log(t) / n_shown) candidates = ucb_explore(candidates, user_id) model_used = "content_ucb"

elif interaction_count < 50: # Transitional: blend collaborative signal with content fallback collab_score = collaborative_model.score(user_id, candidates) content_score = content_model.score(catalog_metadata, candidates) blended = 0.6 * collab_score + 0.4 * content_score model_used = "hybrid"

else: # Full collaborative: user embedding has enough signal candidates = two_tower_retrieve(user_id, top_k=500) model_used = "collaborative"

return {"candidates": candidates, "model": model_used, "interaction_count": interaction_count} ```