Ensemble Methods: Bagging vs Boosting vs Stacking — Mechanics, Trade-offs, and When Each Wins

Every winning tabular ML system in the last decade is an ensemble. XGBoost, LightGBM, CatBoost, the Random Forest in your favourite sklearn tutorial — all ensembles. But "ensemble" is not one thing. Bagging, boosting, and stacking are three fundamentally different recipes that combine weak models into strong ones, and they win on different problems. Knowing which to reach for is what separates a senior MLE from someone who just types xgboost.fit().

Ensembling — combining multiple models to outperform any single one — is the most reliably successful technique in applied ML. Almost every Kaggle winner and almost every production tabular system uses one. But three different recipes get called "ensembling," and they work for different reasons. Understanding the mechanics is what lets you reach for the right one and debug it when it fails.

Why ensembles work: the variance-bias decomposition

The expected squared error of any model decomposes as: error = bias² + variance + irreducible_noise (from post 73). Different ensemble methods attack different terms. Bagging reduces variance while leaving bias unchanged. Boosting reduces bias while leaving variance roughly unchanged (or sometimes increasing it slightly). Stacking does both in a different way — by learning which model to trust in which region of the feature space. The error term you have most of is the one you need to attack.

Bagging — variance reduction via averaging

Bagging (Bootstrap Aggregation, Breiman 1994): train B models on B independently bootstrapped datasets (each sampled n-with-replacement from the original training set), then average their predictions. The mathematical guarantee: if each model has variance σ² and the models are pairwise uncorrelated, the variance of the average is σ²/B. If they have pairwise correlation ρ, variance becomes σ²[(1-ρ)/B + ρ]. The first term shrinks to zero as B → ∞; the second term is a floor you cannot beat by adding more models. So the entire engineering challenge in bagging is reducing ρ — making the base models as decorrelated as possible. Bootstrap sampling alone gets you partway. Random Forests add feature-subset randomness at every split, which decorrelates the trees further. Extra Trees take this further with random thresholds. Bagging works best when the base model is low-bias and high-variance — deep decision trees, deep neural nets that overfit. It does almost nothing for a linear regression on top of a small feature set, because there's no variance to reduce.

Boosting — sequential bias reduction

Boosting trains weak models sequentially, each one focused on the errors of its predecessors. AdaBoost (Freund and Schapire, 1995): each training sample carries a weight. Train a weak classifier h₁. Misclassified samples get their weights increased — h₂ pays more attention to them. The final prediction is a weighted vote: sign(Σ αₜ hₜ(x)) where αₜ depends on the weak learner's accuracy. Gradient Boosting (Friedman, 2001) generalises this: each new model fits the negative gradient of the loss with respect to the current ensemble's predictions. For squared loss, that's the residual y - F(x). For log loss, it's a weighted residual. XGBoost, LightGBM, CatBoost are all gradient boosting machines with engineering refinements (second-order Newton step in XGBoost, leaf-wise growth in LightGBM, ordered boosting in CatBoost for target leakage protection). Boosting works best with high-bias low-variance base learners: shallow trees, stumps (depth 1). The base learner deliberately underfits; the sequence of corrections is where the model capacity comes from.

Stacking — meta-learning over base models

Stacking (Wolpert, 1992) trains a meta-learner whose inputs are the outputs of several base models. The meta-learner learns when to trust which model. Layer 1: train several diverse base models (e.g., logistic regression, random forest, XGBoost, neural net). Layer 2: train a meta-learner (often a simple linear model or shallow tree) whose features are the predictions of the layer-1 models. Stacking shines when the base models have genuinely different inductive biases and make different kinds of errors. A linear model and an XGBoost model fail on different examples; the meta-learner can learn which to trust.

WARNING — Production tell for stacking: out-of-fold predictions or you have already shipped a leakage bug. The trap: train base models on the full training set, predict on the full training set, feed those predictions to the meta-learner. The base models have already seen the labels they're predicting — their training-set predictions are unrealistically good. The meta-learner overfits to those optimistic predictions and collapses on the test set. The fix is k-fold cross-validation: for each fold, train the base models on the other k-1 folds, predict on the held-out fold, and concatenate the held-out predictions to form the meta-learner's training data. Every stacking failure I have seen in production traces back to this. Senior interviews ask this exact question to filter for people who have actually built stacked models.

When each one wins

Bagging: when your base model overfits and you have compute to spare. Random Forests are the default — fast to train, robust, almost no hyperparameter tuning, OOB error gives you a free validation estimate. Production teams reach for Random Forests when they need a working model fast and don't want to tune anything.

Boosting: when you need maximum tabular accuracy and you're willing to spend tuning time. Gradient boosted trees are the SOTA for tabular ML and will outperform Random Forests on almost every benchmark — but they require careful hyperparameter tuning (learning rate, max depth, regularisation, early stopping) and they are sensitive to noisy labels in a way RF isn't. XGBoost and LightGBM win nearly every Kaggle tabular competition.

Stacking: when you have several already-trained models and ensembling them gives a real boost. Almost never the right first move — you should tune a single XGBoost or RF first. Stacking is what you do when you already have a strong baseline and need to squeeze out the last 1-2% for a competition or a benchmark.

Diversity is the constraint, not size

Three almost-identical XGBoost models with different seeds and a meta-learner on top: marginal improvement, because the base models are correlated. One XGBoost, one Random Forest, one neural net, one linear model, and a meta-learner on top: real improvement, because the base models make different errors. The lesson: ensembles need diverse base learners. You can't ensemble your way out of a single-architecture bottleneck.

Interview questions on this topic

"Why does bagging reduce variance but not bias? Why does boosting reduce bias but not variance?" — Bagging averages predictions from models trained on similar data. The expected value of the average equals the expected value of one model (bias is unchanged), but the variance of the average is lower (1/B of the original variance if uncorrelated). Boosting fits each new model to the residuals of the current ensemble. Each model corrects a piece of the bias; the ensemble's expected prediction moves closer to the true function. But because boosting is sequential and depends on the noise in the residuals, it doesn't reduce variance — and aggressive boosting can amplify it (which is why early stopping matters).

"A junior engineer is stacking XGBoost, Random Forest, and a neural net. Their cross-validated stacking ensemble beats every base model by 3%. On the test set, the stacking ensemble does worse than XGBoost alone. What's the most likely bug?" — They almost certainly trained the meta-learner on the base models' training-set predictions instead of out-of-fold predictions. The base models had seen the training labels, so their training-set predictions were unrealistically accurate, the meta-learner overfit to that pattern, and the gap closes on the test set. Fix: regenerate the meta-learner's training data by k-fold cross-validation where each fold's predictions come from base models trained on the other folds.

"You're training XGBoost with 1000 trees and learning rate 0.1. The training loss keeps decreasing but validation loss starts increasing after iteration 300. What are your two main levers and how do they trade off?" — Two levers: lower the learning rate (each tree corrects less, more trees needed to fit the data, less risk of overshooting the validation optimum) or strengthen regularisation (lower max_depth, raise min_child_weight, add L1/L2 on leaf weights). Lower learning rate gives you smoother convergence but costs training time. Stronger regularisation gives you a worse fit at low tree counts but a better fit at high tree counts. Use early stopping with a validation set as the safety net.

"In a Random Forest with 500 trees, you measure pairwise correlation between tree predictions at 0.6 on a holdout set. Should you be concerned?" — Yes — high correlation between trees means the variance reduction from averaging is limited. The variance of the ensemble is σ²[(1-0.6)/500 + 0.6] ≈ 0.6 σ². You're getting only 40% of the variance reduction you'd get from fully decorrelated trees. Likely cause: too many features used per split (try lowering max_features), or one dominant feature that every tree splits on early. Adding more trees won't help past this point — you've hit the correlation floor.

Try on Colab: generate a synthetic regression dataset with 1000 samples, 20 features. Train (a) a single deep decision tree, (b) a bagged ensemble of 100 deep trees, (c) a Random Forest with √20 features per split, (d) a gradient boosted ensemble of 100 stumps with learning rate 0.05. Compare training MSE, test MSE, and prediction variance across 10 different random seeds. Then build a stacking ensemble of (c) and (d) with a linear meta-learner. Train it BOTH ways — once with in-fold predictions, once with 5-fold out-of-fold predictions. Show on the test set how the in-fold version overfits.