Gradient Boosted Trees: What XGBoost Is Actually Doing

XGBoost wins Kaggle competitions not by magic but by iteratively fitting residuals with regularised trees. Each tree corrects the errors of the previous ensemble. The second-order Taylor expansion of the loss is the key ingredient that makes tree-finding efficient. Understanding this makes XGBoost's hyperparameters principled rather than arbitrary knobs.

Gradient boosting is an ensemble method that builds trees sequentially: each new tree corrects the errors of the current ensemble. XGBoost (Chen & Guestrin, 2016) is the most widely used implementation, with LightGBM and CatBoost as close competitors. All three use the same core idea with different engineering optimisations.

Additive tree ensembles

A boosted model is a sum of T trees: F(x) = Σ_{t=1}^{T} f_t(x), where each f_t is a regression tree. Training is additive: at step t, the model is F_{t-1}(x) + f_t(x). The question is: what should f_t(x) look like?

Gradient boosting: fit the residuals

In ordinary gradient descent, you update parameters to minimise the loss by moving in the negative gradient direction. Gradient boosting does the same thing, but the "parameters" are the predictions F(x), and "moving in the negative gradient direction" means fitting a tree to the negative gradient of the loss at the current predictions.

For MSE loss L = (y - F(x))^2, the negative gradient is y - F(x) — the residual. So fitting a tree to the residual and adding it to the ensemble is exactly gradient descent in function space. For other losses (log-loss, Huber), the negative gradient is different, and gradient boosting handles all of them uniformly by fitting trees to the pseudo-residuals.

XGBoost's key innovation: second-order Taylor expansion

Vanilla gradient boosting uses only the first-order gradient (the pseudo-residual). XGBoost uses a second-order Taylor expansion of the loss: L ≈ L(F_{t-1}) + g_i * f_t(x_i) + (1/2) * h_i * f_t(x_i)^2, where g_i = ∂L/∂F(x_i) is the gradient and h_i = ∂^2L/∂F(x_i)^2 is the Hessian. For each leaf in tree t, the optimal leaf weight (given the tree structure) is: w* = -Σ_i g_i / (Σ_i h_i + λ), where λ is L2 regularisation on leaf weights. This closed-form optimal leaf value means XGBoost can evaluate candidate tree structures more accurately and efficiently than first-order methods.

Regularisation in trees

XGBoost adds regularisation terms to the objective: Ω(f_t) = γT + (λ/2) Σ_j w_j^2, where T is the number of leaves and w_j are leaf weights. γ penalises the number of leaves (minimum gain per split), λ penalises large leaf weights (L2 regularisation). These terms are tunable hyperparameters. Larger γ = fewer splits = simpler trees. Larger λ = smaller leaf weights = more conservative predictions.

Tree construction: exact and approximate splits

For each candidate split (feature, threshold), XGBoost computes the gain: Gain = (1/2)[G_L^2/(H_L+λ) + G_R^2/(H_R+λ) - (G_L+G_R)^2/(H_L+H_R+λ)] - γ. The split is made if gain > 0. Exact algorithm: evaluate all possible splits over all features. Approximate algorithm: bucket continuous features into quantiles, evaluate only split points at quantile boundaries. LightGBM uses gradient-based one-side sampling (GOSS) — only sample the data points with large gradients for split finding — which makes it faster than XGBoost on large datasets.

Key hyperparameters and their effects

n_estimators: number of trees. More trees → lower training loss; potential overfitting without regularisation. learning_rate (η): shrinks each tree's contribution. Lower η + more trees typically beats higher η + fewer trees. max_depth: maximum depth per tree. Shallow trees (3-6) are faster and regularise well; deep trees capture more interactions. subsample: fraction of training data used per tree. Reduces variance, speeds training. colsample_bytree: fraction of features considered per tree. Reduces correlation between trees.

When to use gradient boosting vs neural networks

Gradient boosted trees win on: tabular data with mixed feature types, small-to-medium datasets (< 10M examples), when training time matters, when interpretability via feature importance is needed. Neural networks win on: images, text, audio, sequences, large datasets where representation learning is the bottleneck, multi-task settings.

Production tells — how XGBoost actually fails

The four most common production failures look identical in dashboards but trace to different causes. (1) Gain-based feature importance lies on high-cardinality columns. "user_id" or "session_id" as a top feature is almost never real signal — it is the tree exploiting any residual variance the regularisation did not squeeze out. Drop it, retrain, see if anything changes; usually nothing does. (2) The quantile binning in the approximate algorithm assumes train-serve feature distributions match. When they do not (e.g., feature scaling drift, currency change, schema migration), the same numeric value falls into a different bin and the tree routes it down a wrong branch. Silent. (3) Categorical encoding inconsistency. XGBoost does not handle raw categoricals natively (unlike CatBoost). If your training pipeline uses pandas categorical codes and your serving pipeline uses a different encoder (one-hot, target encoding, hashing), the model receives a feature value that has nothing to do with the training-time meaning. Test this before every deploy. (4) Number-of-trees mismatch between training (with early stopping) and serving (default predict uses all trees). If you train with early_stopping_rounds and never call predict with ntree_limit set to best_iteration, you serve the overfit late trees. Half the production XGBoost regressions traced to this one line.

Try on Colab: train XGBoost on the Adult Income dataset. Plot the training loss vs validation loss as a function of n_estimators — identify the early stopping point. Then vary max_depth (2, 4, 6, 8) and learning_rate (0.01, 0.1, 0.3) independently. Visualise feature importances. Compare against a random forest baseline and a logistic regression baseline — the gradient boosting gain over random forests is typically 2-5% accuracy.