Key Concepts in Time Series Models

Why This Matters

Time series models form the backbone of forecasting and dynamic analysis in quantitative methods—and you're being tested on your ability to choose the right model for the right situation. Whether you're predicting stock prices, analyzing economic indicators, or modeling climate patterns, understanding stationarity requirements, volatility dynamics, and multivariate relationships separates surface-level knowledge from genuine analytical skill.

Don't just memorize model acronyms and their parameters. Know why each model exists, what data characteristics it addresses, and when to deploy it over alternatives. Exam questions will ask you to identify appropriate models given specific data features, interpret parameter meanings, and explain the theoretical foundations that make each approach valid.

---

Foundational Univariate Models

These models capture temporal dependence in a single variable using either past values, past errors, or both. The key distinction lies in whether the model looks backward at the variable itself or at the mistakes made in predicting it.

Autoregressive (AR) Models

• Past values drive predictions—the model uses $p$ lagged observations of the variable itself to forecast future values
• Coefficients measure persistence—each $\phi_i$ parameter quantifies how strongly value $t-i$ influences the current observation
• Stationarity required—AR models assume the underlying process has constant mean and variance over time

Moving Average (MA) Models

• Past forecast errors drive predictions—the model uses $q$ lagged error terms (shocks) rather than the variable's own history
• Captures short-term dynamics—MA models excel at modeling temporary fluctuations that dissipate quickly
• Invertibility condition—for valid estimation, MA coefficients must satisfy constraints ensuring the process can be expressed in AR form

Autoregressive Moving Average (ARMA) Models

• Combines AR and MA components—uses both $p$ past values and $q$ past errors for more flexible modeling
• Parsimony advantage—often achieves better fit with fewer parameters than pure AR or MA specifications
• Stationary data only—ARMA assumes no trends, no unit roots, and no seasonal patterns in the series

---

Handling Non-Stationarity

Real-world data rarely stays put. These models address trends, unit roots, and deterministic drift that violate the assumptions of basic ARMA frameworks.

Autoregressive Integrated Moving Average (ARIMA) Models

• Differencing creates stationarity—the $d$ parameter specifies how many times to difference the series before applying ARMA
• Notation $ARIMA(p,d,q)$—combines autoregressive order, integration order, and moving average order in one framework
• Most common specification—$ARIMA(1,1,1)$ handles many economic series with a single difference and simple dynamics

Seasonal ARIMA (SARIMA) Models

• Adds seasonal differencing and lags—parameters $(P, D, Q)_s$ capture patterns repeating every $s$ periods
• Full notation $SARIMA(p,d,q)(P,D,Q)_s$—separates non-seasonal and seasonal components explicitly
• Essential for periodic data—quarterly GDP, monthly retail sales, and daily temperature all exhibit predictable seasonal swings

---

Modeling Volatility Dynamics

Financial returns often exhibit volatility clustering—periods of high variance followed by more high variance. These models treat variance itself as a time-varying process.

Generalized Autoregressive Conditional Heteroskedasticity (GARCH) Models

• Variance depends on past variance and past shocks—$\sigma_t^2 = \omega + \alpha \epsilon_{t-1}^2 + \beta \sigma_{t-1}^2$ in the standard $GARCH(1,1)$
• Captures volatility clustering—large price movements tend to follow large movements, which GARCH models explicitly
• Foundation for risk management—Value-at-Risk calculations and option pricing rely heavily on GARCH-based volatility forecasts

Exponential Smoothing Models

• Weighted averages with exponential decay—recent observations receive more weight than distant ones via smoothing parameter $\alpha$
• Holt-Winters extends to trends and seasons—additive or multiplicative components handle level, slope, and periodic patterns
• Computationally simple—no likelihood estimation required, making these models fast benchmarks for forecasting competitions

---

Multivariate and Regime-Based Models

When multiple variables interact or when the underlying data-generating process shifts over time, these advanced frameworks become necessary.

Vector Autoregression (VAR) Models

• All variables are endogenous—each series is regressed on its own lags and the lags of every other variable in the system
• Granger causality testing—VAR enables formal tests of whether one variable's past helps predict another
• Impulse response functions—trace how a shock to one variable propagates through the system over time

State Space Models

• Separates observed from unobserved components—measurement equation links data to latent states; state equation governs state evolution
• Kalman filter estimation—recursive algorithm updates state estimates as new observations arrive
• Handles missing data and irregular timing—the framework naturally accommodates gaps and uneven sampling intervals

Markov Switching Models

• Regime-dependent parameters—model coefficients change based on an unobserved discrete state variable
• Transition probabilities govern switching—the probability of moving between regimes follows a Markov chain
• Captures structural breaks endogenously—rather than imposing break dates, the model infers when shifts occurred

---

Quick Reference Table

---

Self-Check Questions

1. Which two models both use differencing to achieve stationarity, and what distinguishes the type of non-stationarity each addresses?

2. You observe that large forecast errors in a financial return series tend to cluster together. Which model family is designed for this phenomenon, and what is the key equation governing variance dynamics?

3. Compare and contrast VAR and State Space models: under what data conditions would you choose one over the other?

4. An FRQ presents quarterly retail sales data with both upward trend and repeating December spikes. Write the general notation for the appropriate model and explain what each parameter group captures.

5. A researcher suspects that interest rate dynamics fundamentally changed after a financial crisis but doesn't know exactly when. Which model allows the data to identify regime shifts endogenously, and what probabilistic structure governs transitions between states?