Stationarity & Differencing
Unit root tests, spurious regression, integration order, cointegration
Regress two independent random walks against each other and you'll get R² near 1 and t-statistics in double digits — not because they're related, but because both are trending. That's spurious regression, and it invalidates every downstream conclusion.
It's the reason stationarity matters: non-stationary series have growing variance and shifting means, so the statistical tests that assume constant moments produce completely unreliable results. The ADF and KPSS tests tell you whether you have a unit root; differencing removes trends; seasonal differencing removes periodicity. The flip side of non-stationarity is cointegration — two non-stationary series can share a long-run equilibrium whose spread is stationary, and error correction models exploit that structure rather than discarding it.
Key points
- Two random walks regressed against each other will look strongly related — R² > 0.5, |t| > 2 — with zero true relationship. The tell is Durbin-Watson near 0: residuals are nearly perfectly autocorrelated, which is the signature of a spurious regression between two non-stationary series. OLS standard errors assume independent residuals; when they're serially correlated, standard errors are severely underestimated and all inference collapses. This is not a small-sample problem — it gets worse with more data.
- Weak (covariance) stationarity requires three properties: constant mean E[Y_t] = μ, constant variance Var(Y_t) = σ², and autocovariance Cov(Y_t, Y_{t-k}) = γ(k) that depends only on lag k, not on t. A random walk Y_t = Y_{t-1} + ε_t violates two of the three — variance grows as tσ² and the autocovariance depends on t, so the series wanders without bound (the unconditional mean E[Y_t] = Y_0 stays constant for this driftless walk; a random walk with drift would violate the mean too). One difference gives ΔY_t = ε_t, which is stationary. The I(d) notation means d differences are needed to achieve stationarity.
- ADF test has H₀: unit root present, H₁: stationary. Includes lags of ΔY_t to purge residual autocorrelation. Critical values are non-standard (more negative than t-distribution). The test has low power in small samples and often fails to reject even when the series is trend-stationary. KPSS flips the null: H₀ is stationary, H₁ is unit root. Running both together resolves ambiguity: ADF rejects and KPSS doesn't reject → stationarity. ADF doesn't reject and KPSS rejects → unit root. Both rejecting → possible fractional integration or local non-stationarity.
- Trend-stationary and difference-stationary series look similar in plots but require different treatment. A trend-stationary series (Y_t = α + βt + ε_t) should be detrended — regress out the linear trend and model residuals. A difference-stationary series needs differencing. The error matters: detrending a difference-stationary series leaves residuals that still have a unit root. Differencing a trend-stationary series removes real signal and induces unnecessary MA structure.
- Cointegration is the productive flip side of non-stationarity. Two I(1) series X_t and Y_t are cointegrated when there exists β such that Y_t − βX_t is I(0). The spread is stationary even though the individual series are not. Economically, the series share a long-run equilibrium. The Engle-Granger test runs ADF on the residuals of a regression of Y on X. The Johansen trace/max-eigenvalue test handles multiple cointegrating vectors simultaneously.
- Error Correction Model: ΔY_t = α(Y_{t-1} − βX_{t-1}) + short-run dynamics. The error-correction term α(Y_{t-1} − βX_{t-1}) pulls the system back toward equilibrium; α < 0 ensures mean reversion. If you difference cointegrated series without including the error-correction term, you throw away the long-run relationship. The short-run model (first differences) tells you about dynamics; the error-correction term tells you about the equilibrium.
- Stationarity is not a one-time check at model training time. A series that was stationary from 2015-2019 can develop a unit root after a structural shift (new competitor, regulatory change, macroeconomic shock). Rolling ADF tests with a sliding window, CUSUM tests on residuals, or online variance monitoring catch this. A single ADF on historical data at training time is insufficient for a deployed model — stationarity must be monitored continuously in production.
Stationarity is not a box to check once at model training time — spurious regression is the immediate consequence of skipping it, and structural breaks mean a series that was stationary at training time may not be stationary in production. The most important inference to demonstrate is knowing when two non-stationary series should be modelled jointly (cointegration + ECM preserves the long-run relationship) versus separately in first differences (when no cointegrating vector exists and the long-run relationship is meaningless).
Recap
- Spurious regression: two random walks → R²>0.5, |t|>2, DW≈0, zero true relation.
- Weak stationarity = constant mean + variance + lag-only autocovariance γ(k).
- ADF (H₀: unit root) + KPSS (H₀: stationary) run together resolve ambiguity.
- Trend-stationary → detrend; difference-stationary → difference. Wrong choice leaves a unit root or induces MA noise.
- Cointegration: two I(1) series, spread I(0) — model jointly via ECM, don't difference away the equilibrium.
- ECM: `α(Y_{t-1}−βX_{t-1})` term, α<0 pulls back to equilibrium.
- Not a one-time check: structural breaks kill stationarity in production — monitor with rolling ADF/CUSUM.
Check your understanding
Q1. You regress daily revenue on daily temperature for 3 years and get R² = 0.72 with t-stat = 18. The DW statistic is 0.12. What is wrong and how do you fix it?
- A) The model is correctly specified; DW near 0 actually confirms no autocorrelation is present, so the high R² and large t-stat reflect genuine, causal co-movement between revenue and temperature.
- B) DW ≈ 0.12 signals near-perfect residual autocorrelation — the hallmark of spurious regression. Test with ADF/KPSS, then work in first differences or use cointegration/ECM if a long-run relationship exists.
- C) The issue is heteroskedasticity, not autocorrelation. Apply a Newey-West or White robust standard error correction after a Breusch-Pagan pre-test, and the OLS regression remains fully valid.
- D) DW near 0 means the model is over-differenced at lag one, inducing artificial negative MA(1) noise into the residual structure. Re-fit with a Cochrane-Orcutt correction instead of differencing and the spurious correlation fully disappears.
Q2. Which TWO of the following statements about ADF and KPSS are correct?
- A) ADF's null hypothesis is a unit root while KPSS's null is stationarity — the tests are complementary because they test in opposite directions, which is exactly why running both resolves ambiguous cases.
- B) ADF is always the more statistically reliable test regardless of sample size, so when the two tests disagree the correct rule is to simply trust ADF and discard the KPSS result entirely without further checks.
- C) When ADF rejects the unit-root null (implying stationarity) and KPSS also rejects the stationarity null (implying a unit root) — the two tests pointing in contradictory directions — this can indicate fractional integration or a structural break, and blindly differencing further is not the right fix.
- D) Both ADF and KPSS require at least exactly 100 observations to produce statistically valid results, and any sample below that threshold makes both tests entirely uninterpretable regardless of context.
Q3. You have two financial time series (stock price and its futures contract price) that are both I(1). How do you decide whether to model them separately in first differences or jointly?
- A) Test for cointegration with the Johansen trace test — spot and futures are cointegrated by no-arbitrage. If a vector is found, use a VECM instead of separate first differences, which discards the equilibrium.
- B) Always model them separately in first differences regardless of theory or context; cointegration is a purely academic construct that almost never survives contact with noisy real-world financial tick data feeds.
- C) Use the Engle-Granger two-step regression on the raw levels; if residuals pass an ADF test at I(0), cointegration is confirmed and the OLS coefficients on levels are directly interpretable and unbiased.
- D) Run a simple Pearson cross-correlation between the two raw level series; if the coefficient exceeds 0.9 then model them jointly with a VAR, otherwise fall back to first differences independently.
Q4. Your revenue series passes ADF stationarity test. You fit an ARIMA(1,0,1) and the residuals look clean. Six months later the model performance degrades sharply. What likely happened and how do you detect it earlier?
- A) The ARIMA order was mis-specified from the start; re-running auto-ARIMA with a wider search grid on the full dataset, including the degraded period, will identify the correct orders retroactively.
- B) The model likely overfit during the original training window by capturing noise as signal; reduce the AR and MA orders to a simpler ARIMA(1,0,1) specification and performance will recover on its own.
- C) A structural break changed the underlying data-generating process. Detect it earlier with rolling ADF tests, CUSUM/MOSUM on rolling mean and variance, and a CUSUM chart on forecast residuals.
- D) Stationarity, once confirmed by ADF, guarantees model stability indefinitely for the life of the series; the degradation is instead caused by an upstream data pipeline bug, not a modelling problem.
Try it interactively
ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.
Open ML Systems Lab →