Causal Inference in Time Series
Granger causality, ITS, synthetic control, CausalImpact, DiD, temporal autocorrelation
Granger causality is the most widely misused concept in applied time series work. It answers a predictive question — does X help predict Y beyond Y's own past? — not a causal one. A shared upstream cause Z that affects both X and Y with different lags produces Granger causality between X and Y with zero direct relationship. Knowing that search volume Granger-causes sales tells you nothing about whether investing in SEO will increase sales. The tools that actually support causal claims — synthetic control, CausalImpact, interrupted time series, difference-in-differences — all require a credible counterfactual: what would have happened to the treated unit absent the intervention.
The harder problem in practice is staggered rollouts, where different units receive treatment at different times. Standard two-way fixed effects DiD is biased under treatment effect heterogeneity in this case — already-treated units contaminate the control group — and the fix (Callaway-Sant'Anna) is not widely known.
Key points
- Granger causality tests whether lagged X improves prediction of Y beyond lagged Y alone — it is a predictive test, not a causal test. A shared upstream cause Z that affects both X and Y with different lags produces Granger causality X → Y with no direct mechanism. TV advertising that simultaneously increases search volume and sales will produce Granger causality from search to sales even if search has no causal effect on sales. The correct interpretation: "X has predictive information about Y beyond Y's own past." The incorrect interpretation: "changing X will change Y."
- Interrupted Time Series (ITS): OLS regression: Y_t = β₀ + β₁T + β₂D_t + β₃(T − T*)D_t + ε_t, where D_t = 1{t ≥ T*}. β₂ is the immediate level change; β₃ is the slope change post-intervention. Critical assumptions: no other intervention coincides at T*, the pre-period trend extrapolates cleanly, and residuals are not autocorrelated (autocorrelated residuals underestimate SE — use Prais-Winsten or Newey-West correction). ITS works without any control unit but requires a credible counterfactual from the pre-period trend extrapolation.
- Synthetic control (Abadie et al.): constructs a weighted combination of control units that matches the treated unit's pre-treatment outcomes as closely as possible. The post-treatment gap (treated − synthetic control) is the estimated treatment effect. The critical advantage over DiD: pre-treatment fit quality is directly observable — you can see whether the synthetic control matches the treated unit before claiming an effect. If pre-period fit is poor, the counterfactual is unreliable and the treatment effect estimate is invalid.
- CausalImpact (Brodersen et al., Google 2015): Bayesian Structural Time Series fitted on the pre-intervention treated series using control series as regressors. Post-intervention: extrapolate the counterfactual; treatment effect = observed − counterfactual with full posterior credible intervals. Key assumptions: the relationship between treated and control series is stable across the intervention, and the pre-period is long enough and rich enough for BSTS to estimate that relationship reliably — a short or noisy pre-period gives a wide, unreliable counterfactual regardless of model correctness. If control series were independently affected by the intervention (spillover), the counterfactual is polluted and the effect estimate is biased. Unlike DiD, CausalImpact does not require strict parallel trends — only that the treated/control relationship stays stable, a weaker and often more realistic condition.
- Difference-in-Differences: DiD = (Y_{treated,post} − Y_{treated,pre}) − (Y_{control,post} − Y_{control,pre}). The parallel trends assumption requires that the treated and control units would have moved identically absent the treatment. Temporal autocorrelation within units inflates t-statistics — cluster standard errors at the unit level. Always plot pre-trend event study coefficients to validate parallel trends before reporting a single DiD estimate.
- Staggered DiD and the TWFE bias: when different units receive treatment at different times, two-way fixed effects (TWFE) regression uses already-treated units as controls for later-treated units. With heterogeneous treatment effects (e.g., the effect grows over time), TWFE produces estimates that can be negatively weighted — some comparisons literally subtract true treatment effects. Callaway-Sant'Anna computes cohort-specific ATTs (average treatment effects for each treatment cohort) using only clean controls (never-treated or not-yet-treated) and aggregates properly. Sun-Abraham is a closely related fix: it uses cohort-by-relative-time interaction terms in an event-study regression (rather than CS's group-time ATT estimation) to get unbiased dynamic treatment-effect estimates under the same staggered-timing conditions. Either estimator resolves the TWFE bias, and the two are commonly cited together as "heterogeneity-robust" staggered-DiD estimators.
- Temporal autocorrelation breaks standard causal methods in specific ways. IV exclusion restrictions become implausible when instrument and outcome share autocorrelated common trends. Regression discontinuity with time as the running variable is especially fragile — observations near the cutoff are highly correlated, conflating the treatment effect with local autocorrelation. Time-varying confounding requires marginal structural models (MSM) with inverse probability of treatment weighting (IPTW), not cross-sectional propensity score matching.
- Event study (dynamic DiD): estimate effects at each relative time period with Y_{it} = αᵢ + λ_t + Σ_{k≠-1} δ_k 1{t − G_i = k} + ε_{it}. Pre-treatment δ_k coefficients (k < 0) test parallel trends — a joint F-test on pre-treatment periods is the validation. Post-treatment coefficients show dynamic effect trajectory. Always visualise as an event study plot before reporting a single DiD estimate — if pre-treatment effects are nonzero, the parallel trends assumption fails and the DiD estimate is biased.
Granger causality is the most frequently misused concept in applied time series work: it measures predictive priority, not causation, and a common upstream cause produces Granger causality between two otherwise unrelated series. The second most important insight for applied causal time series is that TWFE DiD is biased under staggered rollouts with heterogeneous treatment effects — already-treated units contaminate the control group, and the fix is Callaway-Sant'Anna, not just clustering standard errors. Always run an event study plot before reporting any DiD estimate.
Recap
- Granger = predictive, NOT causal: a shared upstream cause Z produces Granger causality with no direct mechanism.
- ITS: pre-period trend extrapolation as counterfactual; correct autocorrelated residuals (Newey-West/Prais-Winsten).
- Synthetic control: weighted control units matching pre-treatment — pre-fit quality is directly observable.
- CausalImpact (BSTS): counterfactual from control regressors; breaks if controls hit by spillover.
- DiD needs parallel trends — always plot event-study pre-trend coefficients before reporting an estimate.
- Staggered TWFE is biased under heterogeneous effects (already-treated as controls, negative weights) → Callaway-Sant'Anna.
- No holdout = strong untestable assumptions; the real fix is a prospective holdout design for future launches.
Check your understanding
Q1. Google search volume Granger-causes weekly sales (p<0.001). A PM wants to invest in SEO to raise sales. Which TWO statements identify the problem correctly?
- A) Granger causality shows lagged search predicts sales beyond sales' own past values — it is fundamentally a predictive test, not proof that deliberately raising search volume will causally raise sales.
- B) A shared upstream cause, such as TV advertising simultaneously driving both search volume and sales, can produce Granger causality between search and sales with no direct causal mechanism between them at all.
- C) The reasoning here is correct as stated; Granger causality at p<0.001 is strong enough statistical evidence on its own to justify the SEO investment, since it demonstrates search is a leading indicator of sales.
- D) The Granger test is invalid because weekly sales data violates the stationarity assumption required for VAR estimation entirely; first-difference both series and re-run the whole test before any conclusion.
Q2. You are measuring the impact of a new feature launched to users in Germany on January 15. You have daily active users (DAU) data for Germany (treated) and France (control) from January 2023 onwards. How do you use CausalImpact and what assumptions must hold?
- A) Fit BSTS on France DAU using Germany DAU as the regressor during the pre-period, then extrapolate post-launch to estimate the counterfactual France trajectory, assuming a stable pre-period relationship.
- B) Run a simple pre-post t-test comparing Germany DAU before and after January 15; CausalImpact is only ever needed when you have multiple control units available, not a single control country like France.
- C) Use both Germany and France data together in a DiD estimator with country and time fixed effects instead; CausalImpact is only appropriate when there is no control group available at all.
- D) Pre=before Jan 15, post=after. Fit BSTS on Germany DAU with France as regressor, extrapolate post-launch. Assumptions: stable relationship, no spillover, adequate pre-period for BSTS to fit reliably — and unlike DiD, no strict parallel-trends requirement.
Q3. Your company rolls out a pricing change to different markets in different months over a 6-month window. You use TWFE DiD to estimate the effect. A colleague says your estimate is biased. Why and what do you do?
- A) TWFE is biased because staggered rollout violates the parallel trends assumption entirely; adding market-specific linear time trends into the TWFE regression will correct the bias without changing the estimator itself.
- B) TWFE DiD is fully unbiased under staggered rollout as long as you cluster standard errors at the market level; the colleague is simply confusing standard-error bias with point-estimate bias in this case.
- C) Staggered timing invalidates TWFE — already-treated markets act as controls for later ones, and with heterogeneous effects TWFE weights can go negative. Use Callaway-Sant'Anna or Sun-Abraham instead.
- D) TWFE is biased only when treatment assignment is non-random; since the rollout schedule was set by the company itself, treatment is as-good-as-random and TWFE gives an unbiased estimate of the average treatment effect.
Q4. You want to estimate the causal effect of an algorithm change on user engagement, but the change was rolled out gradually to all users with no holdout group. CausalImpact, synthetic control, and ITS all require a control group or counterfactual. What do you do?
- A) Use a simple pre-post comparison with a paired t-test on engagement metrics measured before and after the rollout date; without any control group, this is the only statistically valid causal identification strategy.
- B) Retrospectively identify a set of users who happened to adopt the new algorithm later as a natural control group; compare early adopters against late adopters directly using a standard DiD estimator.
- C) Apply CausalImpact using a competitor platform's engagement metric as the control regressor; competitor metrics are always valid synthetic controls to use for internal product changes like this.
- D) Without a holdout: ITS with pre-change BSTS extrapolation, regression discontinuity in time, or external synthetic-control donors. Honest answer: no holdout means untestable assumptions.
Try it interactively
ML Systems Lab is a free interview-prep platform for ML engineers — work through the full interactive module, quizzes, and drills.
Open ML Systems Lab →