Time Series Forecasting: ARIMA, Prophet, and When Neural Models Win

Time series forecasting is not regression with a date column. Serial correlation, seasonality, non-stationarity, and distribution shift over time require specific modelling decisions. ARIMA handles autocorrelation. Prophet handles seasonality and holidays. Neural models (N-BEATS, Temporal Fusion Transformer) win when you have many related series and enough data. The wrong model for the wrong data produces confident wrong answers.

Time series forecasting appears in almost every company: demand forecasting for inventory, revenue forecasting for planning, metric forecasting for anomaly detection, user growth projections. The standard ML instinct — "throw gradient boosting at it" — works poorly when temporal structure is ignored.

The core structure: trend, seasonality, noise

Most time series decompose into: trend (long-term direction — growing, shrinking, flat), seasonality (periodic patterns — daily, weekly, yearly), and noise (random variation). STL decomposition (Seasonal and Trend decomposition using Loess) splits a series into these three components non-parametrically. Plotting the decomposition is the first diagnostic step: it tells you how strong each component is and whether your model needs to capture them.

ARIMA: modelling autocorrelation

ARIMA(p, d, q) is the classical approach for univariate time series. d is the differencing order — applying d differences to make the series stationary (constant mean and variance). A series is integrated of order d if d-differencing makes it stationary. p is the autoregressive order — include the last p values as predictors. AR(p): y_t = c + Σ_{i=1}^{p} φ_i y_{t-i} + ε_t. q is the moving average order — include the last q error terms. MA(q): y_t = c + ε_t + Σ_{i=1}^{q} θ_i ε_{t-i}.

ARIMA combines these. The autocorrelation function (ACF) and partial autocorrelation function (PACF) diagnose appropriate p and q: ACF plots correlation of y_t with y_{t-k}; PACF plots correlation after removing the effect of intermediate lags. For AR(p) processes, PACF cuts off at lag p; for MA(q) processes, ACF cuts off at lag q. SARIMA adds seasonal terms.

Prophet: designed for business time series

Prophet (Taylor & Letham, Facebook, 2018) is designed for the time series characteristics most common in business settings: strong weekly and yearly seasonality, holiday effects, trend changepoints. It decomposes: y(t) = g(t) + s(t) + h(t) + ε(t), where g(t) is trend (linear or logistic), s(t) is seasonality (Fourier series), h(t) is holiday effects (dummy variables). Changepoints — where the trend slope changes — are detected automatically using a sparse prior on changepoint magnitudes.

Prophet is fast to fit, interpretable, handles missing data gracefully, and produces uncertainty intervals. It outperforms ARIMA on many business metrics because it explicitly models seasonality structure that ARIMA captures only through seasonal differencing.

Cross-validation for time series: no data leakage

Standard k-fold cross-validation shuffles examples randomly — invalid for time series because future values cannot predict past values. Time series cross-validation uses expanding windows: train on [1,t], validate on t+1,...,t+h. Repeat for many values of t. This simulates the actual forecasting setting. scikit-learn's TimeSeriesSplit implements this. Common mistake: using a random train/test split on a time series. This creates data leakage (test examples are in the middle of training examples) and produces optimistic estimates.

Feature engineering for tabular time series models

Tree-based models (XGBoost, LightGBM) outperform ARIMA on many time series when features are engineered correctly: lag features (y_{t-1}, y_{t-7}, y_{t-365}), rolling statistics (rolling mean and std over 7d, 30d windows), date features (hour, day of week, month, is_holiday, is_month_end), and external regressors (weather, promotions, competitor prices). The critical rule: only use lag features that would be available at the time of prediction to avoid leakage.

Neural models: when they win

N-BEATS (Oreshkin et al., 2020) and Temporal Fusion Transformer (Lim et al., 2021) win over classical methods when: you have many related time series with shared structure (e.g., thousands of product sales series), long-range dependencies beyond ARIMA's typical p=1-3, or multivariate dependencies across many input series. TFT uses self-attention across time steps, gating to select relevant features, and quantile regression for uncertainty. On M4 and M5 competitions (business forecasting benchmarks), TFT-based models achieve state-of-the-art.

Anomaly detection in time series

Identify time points where y_t deviates from what the model expected. Residuals e_t = y_t - ŷ_t. Points where |e_t| > k * σ_e are flagged (sigma-clipping). More sophisticated: fit a distribution to the residuals (normal, t-distribution for heavy tails), compute the probability of each observation, flag low-probability events. ARIMA residuals should be white noise — plotting their ACF diagnoses whether residual structure remains unexplained.

Try on Colab: download the M5 competition dataset (Walmart store-level daily sales, 30,490 series). Forecast 28 days ahead for 100 series using: (1) ARIMA with auto-selection, (2) Prophet, (3) XGBoost with lag features. Evaluate with WRMSSE (competition metric). Build the cross-validation setup correctly — ensure no leakage. Compare the three approaches on accuracy and training time.