Key Time Series Analysis Techniques

Why This Matters

Time series analysis sits at the heart of forecasting—you're being tested on your ability to select the right technique for the right data pattern. The exam doesn't just want you to define ARIMA; it wants you to know when ARIMA beats exponential smoothing, why stationarity matters, and how to diagnose model fit using ACF and PACF plots. These techniques connect directly to core forecasting principles: trend identification, seasonality detection, stationarity requirements, and model selection.

Think of these methods as tools in a diagnostic toolkit. Some handle short-term noise (moving averages), others tackle long-term memory (ARFIMA), and still others manage multiple interacting variables (VAR). Don't just memorize formulas—know what problem each technique solves and what assumptions it requires. When an FRQ asks you to recommend a forecasting approach, your answer should demonstrate understanding of the underlying mechanism, not just vocabulary recall.

---

Foundational Concepts: Preparing Your Data

Before applying any model, you need to understand your data's structure. These techniques help you diagnose patterns and transform data into a form that models can handle effectively. Stationarity—constant mean and variance over time—is the gateway requirement for most classical forecasting methods.

Stationarity and Differencing

• Stationarity—a time series with constant mean, variance, and autocovariance over time; most forecasting models assume this property
• Differencing transforms non-stationary data by computing $Y_t - Y_{t-1}$, removing trends and stabilizing the mean
• Unit root tests (like Augmented Dickey-Fuller) formally assess whether differencing is needed before model fitting

Autocorrelation and Partial Autocorrelation Functions

• ACF (Autocorrelation Function) measures correlation between a series and its lagged values at all lag distances
• PACF (Partial Autocorrelation Function) isolates the direct relationship at each lag, controlling for intermediate lags
• Model identification relies on ACF/PACF patterns: AR models show PACF cutoff, MA models show ACF cutoff

Trend Analysis and Decomposition

• Decomposition separates a time series into three components: trend, seasonal, and irregular (residual)
• Additive vs. multiplicative decomposition depends on whether seasonal fluctuations are constant or proportional to trend level
• Detrending isolates cyclical patterns and improves forecast accuracy by modeling components separately

---

Error-Based Models: Learning from Past Mistakes

These models improve forecasts by incorporating information from previous prediction errors. The core insight: systematic patterns in forecast errors indicate missed structure that can be exploited.

Moving Average (MA) Models

• Past forecast errors (not past values) drive predictions; the model corrects for recent shocks to the system
• Order $q$ specifies how many lagged error terms are included; higher $q$ captures longer error persistence
• Best for short-term fluctuations and data where shocks have temporary, decaying effects

Exponential Smoothing Methods

• Weighted averaging applies exponentially decreasing weights to older observations; recent data matters most
• Three variants address different patterns: simple (level only), Holt's (level + trend), Holt-Winters (level + trend + seasonality)
• Smoothing parameters ($\alpha$, $\beta$, $\gamma$) control how quickly the model adapts to new information

---

Autoregressive Models: Using the Past to Predict the Future

Autoregressive approaches assume that past values of a variable contain predictive information about future values. The key assumption: the future resembles the past in systematic, quantifiable ways.

Autoregressive (AR) Models

• Past values of the series itself predict future values; $Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + ... + \epsilon_t$
• Order $p$ indicates how many lagged observations enter the model; identified via PACF cutoff
• Stationarity required—AR models on non-stationary data produce spurious results and unreliable forecasts

Vector Autoregression (VAR) Models

• Multivariate extension models several time series simultaneously, capturing interdependencies between variables
• Each variable is regressed on its own lags plus lags of all other variables in the system
• Impulse response functions trace how a shock to one variable propagates through the system over time

---

Integrated Models: Handling Non-Stationarity

When data exhibits trends or other non-stationary behavior, these models incorporate differencing directly into the framework. Integration (the "I" in ARIMA) bridges the gap between raw data and the stationarity that models require.

Autoregressive Integrated Moving Average (ARIMA) Models

• ARIMA(p, d, q) combines autoregression, differencing, and moving average in one flexible framework
• Parameter $d$ specifies differencing order; $d=1$ removes linear trend, $d=2$ removes quadratic trend
• Box-Jenkins methodology provides systematic identification, estimation, and diagnostic checking procedures

Seasonal ARIMA (SARIMA) Models

• SARIMA(p,d,q)(P,D,Q)$_s$ adds seasonal AR, differencing, and MA terms at seasonal lag $s$
• Seasonal differencing ($D$) removes repeating seasonal patterns; $Y_t - Y_{t-s}$
• Ideal for monthly, quarterly, or annual data with predictable seasonal cycles like retail sales or energy demand

Box-Jenkins Methodology

• Three-stage process: identification (using ACF/PACF), estimation (maximum likelihood), diagnostic checking (residual analysis)
• Iterative refinement continues until residuals resemble white noise with no remaining autocorrelation
• Parsimony principle favors simpler models; use information criteria (AIC, BIC) to balance fit against complexity

---

Advanced Structures: Hidden States and Frequencies

These sophisticated techniques model aspects of time series that aren't directly observable or that operate in the frequency domain rather than the time domain.

State Space Models and Kalman Filtering

• Hidden states represent unobserved components (like true underlying trend) that generate the observed data
• Kalman filter recursively updates state estimates as new observations arrive; optimal for Gaussian linear systems
• Real-time applications include navigation, tracking, and dynamic economic modeling where conditions change continuously

Spectral Analysis

• Frequency domain perspective decomposes a series into cyclical components of different periodicities
• Periodogram identifies dominant frequencies; peaks indicate important cycles in the data
• Filtering applications separate signal from noise by isolating specific frequency bands

---

Long-Range Dependence and Volatility

Some time series exhibit memory effects or variance patterns that standard models miss. Financial data especially requires these specialized techniques.

Long Memory Models (ARFIMA)

• Fractional differencing allows non-integer values of $d$, capturing persistent but eventually decaying effects
• Long-range dependence means distant observations remain correlated; ACF decays slowly (hyperbolically, not exponentially)
• Applications include financial returns, network traffic, and climate data where effects persist over long horizons

GARCH Models for Volatility Forecasting

• Time-varying volatility is modeled directly; variance depends on past squared errors and past variances
• Volatility clustering—high volatility periods tend to follow high volatility; captured by GARCH structure
• Risk management applications include Value-at-Risk calculations, option pricing, and portfolio optimization

Cointegration and Error Correction Models

• Cointegration identifies long-run equilibrium relationships between non-stationary series that "move together"
• Error correction models (ECM) capture short-term deviations while maintaining long-run equilibrium
• Economic applications include purchasing power parity, interest rate relationships, and supply-demand dynamics

---

Quick Reference Table

---

Self-Check Questions

1. You observe that a time series ACF decays slowly while the PACF cuts off sharply after lag 2. Which model family is most appropriate, and what order would you specify?

2. Compare and contrast ARIMA and exponential smoothing: What assumptions differ, and when would you choose one over the other for short-term forecasting?

3. A colleague proposes using standard ARIMA on monthly ice cream sales data. What limitation should you point out, and which alternative model addresses it?

4. How do cointegration and error correction models work together? Give an example of two economic variables where this approach would be appropriate.

5. An FRQ presents stock return data showing periods of high volatility followed by more high volatility. Which modeling technique specifically addresses this pattern, and what is the key mechanism it captures?