Key Time Series Forecasting Models

Why This Matters

Time series forecasting sits at the heart of business decision-making—from inventory planning and revenue projections to workforce scheduling and capital budgeting. You're being tested not just on whether you can name these models, but on whether you understand when each approach works best and why certain models outperform others in specific situations. The underlying principles here—stationarity, autocorrelation, seasonality, and the bias-variance tradeoff—show up repeatedly in exam questions.

Think of these models as tools in a toolkit: a skilled forecaster knows that an ARIMA model excels with stationary data, while Holt-Winters handles seasonal trends, and LSTM networks capture complex nonlinear patterns. Don't just memorize the acronyms—know what data characteristics each model addresses and when you'd choose one over another. That comparative thinking is what separates strong exam performance from mediocre recall.

---

Classical Smoothing Approaches

These foundational models work by averaging or weighting historical observations to filter out noise and reveal underlying patterns. They're computationally simple and often surprisingly effective for stable, well-behaved data.

Moving Average (MA) Models

• Smooths short-term fluctuations—by averaging residual errors from lagged observations, MA models reveal longer-term trends and cycles
• Order parameter (q) indicates how many lagged forecast errors are included; higher orders capture more complex error structures
• Best for stationary data where you need to model the error component rather than the values themselves

Exponential Smoothing Models

• Applies decreasing weights to past observations—recent data matters more, older data fades exponentially
• Three variants address different data types: simple (no trend/seasonality), double (trend only), triple (trend + seasonality)
• Minimal parameter tuning makes these models practical for quick forecasts when patterns are relatively stable

---

Autoregressive Frameworks

These models exploit autocorrelation—the idea that past values of a series help predict future values. They're the workhorses of classical time series analysis.

Autoregressive (AR) Models

• Predicts using past values directly—the current observation is a weighted sum of previous observations plus error
• Order parameter (p) specifies how many lagged values feed into the prediction; selection often guided by partial autocorrelation plots
• Requires stationarity—statistical properties like mean and variance must remain constant over time

Autoregressive Integrated Moving Average (ARIMA) Models

• Combines AR and MA with differencing—the "I" (integrated) component transforms non-stationary data into stationary form
• Three parameters: $p$, $d$, $q$—representing AR order, differencing degree, and MA order respectively
• Highly versatile and considered the gold standard for univariate forecasting when data lacks strong seasonality

Seasonal ARIMA (SARIMA) Models

• Extends ARIMA with seasonal terms—adds parameters $(P, D, Q)_m$ to capture periodic patterns at interval $m$
• Handles dual structure by modeling both short-term autocorrelation and seasonal autocorrelation simultaneously
• Essential for retail, tourism, and energy data where monthly, quarterly, or annual cycles dominate

---

Multivariate and Systems Models

When multiple time series interact—like how advertising spend affects sales, which affects inventory—you need models that capture cross-variable dependencies.

Vector Autoregression (VAR) Models

• Captures interdependencies among multiple series—each variable is modeled as a function of its own lags and lags of all other variables
• Enables impulse response analysis—you can trace how a shock to one variable ripples through the system over time
• Assumes all variables are endogenous—no predetermined causal direction, making it ideal for exploratory analysis

State Space Models

• Represents time series as unobserved states—separates the measurement equation (what you see) from the transition equation (how states evolve)
• Kalman filter estimation provides optimal forecasts by recursively updating state estimates as new data arrives
• Highly flexible framework—can incorporate trends, seasonality, regression effects, and irregularities in a unified structure

---

Trend and Seasonality Specialists

Some models are purpose-built to decompose time series into interpretable components—level, trend, and seasonal effects.

Holt-Winters Method

• Triple exponential smoothing—maintains three separate equations updating level, trend, and seasonal components
• Additive vs. multiplicative variants—additive when seasonal swings are constant; multiplicative when they scale with the level
• Practical and interpretable for business users who need transparent forecasts with clear seasonal adjustments

Prophet Model

• Developed by Meta for business forecasting—handles missing values, outliers, and irregular holiday effects gracefully
• Decomposition approach separates trend (with changepoints), yearly/weekly seasonality, and user-specified holiday effects
• Designed for analysts, not statisticians—intuitive parameters and automatic handling of common data issues

---

Machine Learning Approaches

When relationships are nonlinear or patterns are too complex for classical models, neural networks offer powerful alternatives—at the cost of interpretability.

Long Short-Term Memory (LSTM) Networks

• Recurrent neural network architecture—processes sequential data while maintaining memory of long-term dependencies through gate mechanisms
• Handles nonlinear patterns that defeat classical models; excels when relationships between past and future are complex
• Requires substantial data and tuning—prone to overfitting on small datasets; best suited for high-frequency data with thousands of observations

---

Quick Reference Table

---

Self-Check Questions

1. Which two models both handle seasonality but differ in their treatment of irregular events like holidays? What makes one more suitable for messy real-world data?

2. If you're given a non-stationary time series with no seasonal pattern, which model family would you choose, and what does the "I" component accomplish?

3. Compare and contrast VAR and ARIMA: when would you choose a multivariate approach over a univariate one, and what assumption does VAR make about causality?

4. A retail company has 10 years of daily sales data with strong weekly and annual seasonality, occasional outliers from promotions, and several structural changes in trend. Rank Prophet, SARIMA, and LSTM in terms of suitability and justify your ranking.

5. What distinguishes state space models from other approaches in this list, and why might a forecaster choose the Kalman filter framework over direct ARIMA estimation?