ARIMA Family
AR/MA intuition, ACF/PACF identification, SARIMA, Box-Jenkins, structural break failure
Monthly airline passenger counts tell a story that breaks every naive forecasting approach. The Box-Jenkins 1976 dataset shows a clear upward trend plus strong yearly seasonality — summer peaks, winter troughs — with amplitude that grows as the level grows. A straight regression on time misses the seasonality. A seasonal average ignores the trend. Feed the raw series into a model without preprocessing and the non-stationary mean will invalidate every coefficient estimate.
ARIMA is built to handle exactly this. The "I" — integrated — is the trend fix: difference the series once (y_t − y_{t-1}) and the linear trend disappears, leaving a stationary series the AR and MA components can model. AR(p) captures autocorrelation through the series' own past: the current value is a weighted sum of the last p observations. MA(q) captures dependence on past shock terms. Together they describe how today's value relates to yesterday's observations and yesterday's surprises.
For the airline data, the seasonal structure at period s=12 requires SARIMA: seasonal AR(P) at lags 12, 24, 36; seasonal differencing at lag 12 to remove the repeating yearly cycle; seasonal MA(Q) for dependence on past seasonal shocks. The SARIMA(p,d,q)(P,D,Q)[12] notation stacks non-seasonal and seasonal layers into one model.
NOT this. "ARIMA is outdated and should always be replaced by ML methods." ARIMA is interpretable, requires no GPU, handles small datasets (n < 200) well, and its parameters have direct statistical interpretations: φ₁ is the momentum coefficient, θ₁ is the shock decay rate. For monthly aggregate forecasting with fewer than 5 years of data, ARIMA is often competitive with or better than neural methods. The real limitation is the Box-Jenkins identification workflow: ACF/PACF cutoffs appear cleanly only in simulated data. Real series mix AR and MA contributions, making the cutoffs ambiguous. The workflow is iterative — tentatively identify orders, fit, diagnose residuals, revise — and it breaks at scale. At 50,000 SKUs, per-series ARIMA identification is impractical regardless of accuracy.
Key points
- AR(p) captures momentum: Y_t = φ₁Y_{t-1} + ... + φ_pY_{t-p} + ε_t. The current value is a weighted sum of p lagged values plus noise. Positive φ₁ captures persistence — values tend to continue in the same direction. Negative φ₁ captures oscillation. The "I" in ARIMA handles non-stationarity by differencing d times before fitting: one difference removes a linear trend, so ARIMA(p,1,q) is the appropriate model for a trending series. Over-differencing (d=2) is a common error — if d=1 residuals still show a unit root, suspect a structural break, not more differencing.
- ACF/PACF give a starting point, not a recipe: AR(p) shows PACF cutoff at lag p; MA(q) shows ACF cutoff at lag q; ARMA shows both decaying. In practice these clean cutoffs appear only in simulated data. Real series mix AR and MA contributions, so textbook cutoffs are rarely visible. Treat ACF/PACF as a prior for candidate orders, then compare ARIMA(1,1,0), ARIMA(0,1,1), ARIMA(1,1,1) by AIC. The Ljung-Box test on residuals must pass at all lags including seasonal ones — a p-value of 0.42 overall can mask a lag-12 spike that signals missing seasonality.
- ARIMA's structural break failure is its most consequential limitation in production. The model assumes p, d, q are constant over time. A regime change — competitor entry, regulatory shift, macroeconomic shock — violates this. After a break, the model absorbs the shift as a long-lag autocorrelation effect, producing biased forecasts indefinitely. CUSUM detects gradual drift; Zivot-Andrews ADF tests for an unknown break date. The correct response is to re-identify ARIMA orders on post-break data, not to add more lags to the existing model.
ARIMA's "I" solves non-stationarity by differencing — one difference removes a linear trend, enabling the AR and MA components to model the stationary residuals. ACF/PACF cutoffs are a starting point, not a recipe: textbook-clean patterns only appear in simulated data, so Box-Jenkins is always iterative via residual diagnostics. The most consequential failure mode is the structural break: after a regime change the model absorbs the shift as a spurious long-lag effect, producing biased forecasts indefinitely — the fix is re-identifying orders on post-break data, not adding more lags.
Recap
- AR(p) = momentum ($Y_t=Σφ_iY_{t-i}+ε_t$); MA(q) = shocks; I(d) = differencing removes trend.
- One difference removes a linear trend — ARIMA(p,1,q) for trending series; d=2 usually over-differencing.
- ACF/PACF = starting point, not recipe: clean cutoffs only in simulated data; compare candidates by AIC.
- Ljung-Box must pass at ALL lags — overall p=0.42 can hide a lag-12 seasonal spike → go SARIMA.
- Structural break = worst failure: model absorbs regime shift as long-lag effect, biased forecasts forever.
- Fix a break by re-identifying orders on post-break data, not adding more lags.
- Per-SKU ARIMA doesn't scale (50k series, cold start, intermittent) → global models / Croston's.
Check your understanding
Q1. Your ACF shows significant spikes at lags 1, 2, 3 that decay, and your PACF shows a single significant spike at lag 1 that cuts off. What model do you fit and why?
- A) MA(3) — the three significant ACF spikes directly indicate three MA terms; the observed PACF pattern is fully consistent with this MA(3) order selection under Box-Jenkins rules.
- B) AR(1) — PACF cuts off at lag 1 and ACF decays, the textbook signature of an AR(1) process. Fit ARIMA(1,0,0), check residuals with Ljung-Box; if autocorrelation remains at lag 2+, try AR(2).
- C) ARMA(1,1) — whenever both ACF and PACF show any non-zero values at early lags at all, a mixed model is always strictly required regardless of the specific decay-versus-cutoff pattern actually observed.
- D) ARIMA(3,1,0) — differencing is needed because three lags in the ACF are significant, which by itself indicates a unit root paired with an AR structure of order exactly 3, before checking PACF at all.
Q2. You fit ARIMA(2,1,2) to a monthly sales series; Ljung-Box on residuals passes overall (p=0.42) but the residual ACF shows a spike at lag 12. Which TWO statements are correct?
- A) A passing overall Ljung-Box p-value does not guarantee every individual lag is clean; a seasonal-lag spike can hide underneath an acceptable aggregate p-value across the tested lag range.
- B) The significant lag-12 spike indicates the non-seasonal differencing order d should be increased from 1 to 2, which will remove the remaining autocorrelation structure entirely and immediately.
- C) The lag-12 spike indicates uncaptured seasonal structure; the fix is upgrading to a seasonal model such as SARIMA(2,1,2)(1,0,1)[12] or adding seasonal differencing, not more non-seasonal lags.
- D) The model has overfit at short lags, causing residual energy to concentrate artificially at lag 12; reducing to a simpler ARIMA(1,1,1) would redistribute the autocorrelation more evenly across all lags.
Q3. Your e-commerce platform has 50,000 product SKUs. You need daily sales forecasts for each. Why is per-SKU ARIMA unrealistic and what do you use instead?
- A) Order selection is expensive at 50k series and many SKUs have intermittent demand ARIMA can't handle. Better: global neural models, Croston's for intermittent demand, hierarchical MinT reconciliation.
- B) Per-SKU ARIMA is unrealistic only because of raw computational cost; once fully parallelised across a large enough cluster, ARIMA remains the single most accurate method for individual SKU-level forecasting.
- C) Per-SKU ARIMA fails primarily because ARIMA requires at least 5 full years of history per series; for any SKU with less data, use a simple seasonal-naive forecast as a drop-in replacement instead.
- D) Per-SKU ARIMA is unrealistic due to the cold-start problem alone; the correct fix is to pre-train one global ARIMA model on aggregated demand and then fine-tune its coefficients separately per SKU.
Q4. You fit ARIMA(0,1,1) and ARIMA(1,1,0) to the same series. Both pass diagnostics. The MA model has lower AIC. A colleague argues the AR model is more interpretable for business stakeholders. How do you decide?
- A) Always choose the model with strictly lower AIC regardless of any stakeholder considerations; interpretability arguments are never a statistically valid reason to prefer a higher-AIC model.
- B) Prefer the AR model unconditionally in every single case — AR models are always more interpretable because lagged values are directly observable, while MA error terms are latent and totally unexplainable to non-statisticians.
- C) ARIMA(0,1,1) with θ≈1 approximates exponential smoothing; ARIMA(1,1,0) with φ≈1 approximates a random walk. Use lower-AIC MA for forecasting, AR for stakeholder comms; verify on a holdout, deploy the simpler if tied.
- D) Fit both models simultaneously as an equally weighted mixture ensemble; this sidesteps the model-selection debate entirely and always produces better-calibrated prediction intervals than either model alone.
Try it interactively
ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.
Open ML Systems Lab →