Feature Importance Drift: When Your Top Features Become Noise
Your model's top 3 features — the ones that dominate predictions and deliver 60% of the model's lift — silently stopped working in production. They're still in the model. They're still computing. But their relationship with the target has decayed. This is feature importance drift, and it's invisible to PSI monitoring because the feature distributions haven't changed — only their predictive power has.
Feature importance drift is different from covariate shift. Covariate shift means your input features X are moving away from the training distribution. Feature importance drift means the features are stable in distribution, but their predictive relationship with the target has shifted. P(X) is unchanged. P(Y|X) is not.
The concrete scenario:
Your model predicts loan default. The top feature by SHAP importance is `debt_to_income_ratio`. During training, users with debt-to-income > 0.5 had a 42% default rate. Users with ratio < 0.3 had a 2% default rate. The feature is powerful.
Six months post-launch, the feature's SHAP importance has fallen 30%. The distribution of `debt_to_income_ratio` is identical to training (PSI = 0.08, well below the alert threshold). But the predictive relationship has decayed: users with ratio > 0.5 now have a 15% default rate. The ratio still matters — but half as much.
Why? A regulatory change, a macro event, or a shift in the user population (new marketing channel attracting a different segment) has altered the causal relationship between the feature and the outcome. The feature didn't go stale — it went uncorrelated.
Why this is invisible to standard monitoring:
PSI (Population Stability Index) checks if P(feature_value_today) matches P(feature_value_at_training). It does. Your alert doesn't fire.
KS (Kolmogorov-Smirnov) test checks if the distributions differ. They don't. No alert.
Prediction distribution monitoring checks if P(score_today) differs from P(score_at_training). It does slightly, but you attribute this to base rate changes.
The signal that matters — P(Y|X) drift — requires labeled data. And labels arrive with delay (30 days for loan default, weeks for churn, days for fraud).
The three detection patterns:
1. SHAP importance divergence: compute SHAP values on recent data and compare to training-era SHAP values. If your top features have dropped > 20% in importance while maintaining stable distribution, concept drift has likely occurred.
2. Calibration loss: train a calibration curve (reliability diagram) on training data. Apply it to recent predictions. If the actual event rate for a given score bucket diverges from the calibration curve, the model's P(Y|X) mapping has shifted.
3. Residual analysis by feature segment: stratify recent data (with available labels) by your top 5 features. For each segment, compute mean residual (actual - predicted). If residuals diverge across segments in a way that differs from training, P(Y|X) has shifted.
The production failure mode:
A model with decayed feature importance keeps shipping predictions, keeps making decisions, keeps affecting customers — but with reduced signal. A default model that should catch 85% of defaults now catches 60%. A churn model that should identify 70% of churners now identifies 40%. The degradation is silent because the model's plumbing is intact.
How to prevent it:
1. Log feature values alongside predictions. Ship a feature importance sidecar that recomputes SHAP importance weekly on logged data.
2. Implement a feature importance regression test: if any of your top-10 features drops more than 15% in importance, trigger a retraining review (not automatic retrain — review).
3. Implement calibration monitoring: for high-confidence predictions (score > 0.9), track the actual positive rate. If it's < 0.7, your model is overconfident — recalibration or retraining needed.
4. Use a holdout set with labels: label a sample of recent predictions (even if labels are delayed) and compare residuals against training baseline. This is the only direct way to detect P(Y|X) drift.
The business signal:
The earliest indication of feature importance drift is often a business metric diverging from the model's own metrics. Model precision looks stable. Fraud ops reports that the flagged transactions are increasingly hard to review. The signal-to-noise ratio on the model's output has degraded. This is feature importance drift.