6.2 Moving average (MA) models

Moving Average (MA) Models

Moving Average (MA) models describe how an observation depends on past forecast errors (also called residuals or shocks). Unlike autoregressive models, which relate an observation to its own past values, MA models relate it to past mistakes in prediction. This makes them especially useful for capturing short-term dependencies in time series data.

MA models are always stationary: their mean, variance, and autocovariance don't change over time. A key property is that shocks have a finite memory. A past error only influences the series for a limited number of time steps, then its effect disappears entirely.

Moving Average Models in Time Series

An MA model says: "The current value equals the series mean, plus the current error, plus some weighted combination of recent past errors."

• Forecast errors ($\varepsilon_t$) are the differences between what actually happened and what the model predicted. These are assumed to be white noise (independent, identically distributed with mean zero).
• Because the model only looks back a fixed number of steps, the influence of any single shock eventually drops to zero. This is what "finite memory" means.
• MA models are inherently stationary. You don't need to difference the data or check for unit roots to ensure stationarity when working with a pure MA process.

Order of MA Models

The order of an MA model, written as $MA(q)$, tells you how many past error terms the model uses. An $MA(1)$ uses only the immediately previous error; an $MA(2)$ uses the two most recent errors, and so on.

The general equation for an $MA(q)$ model is:

$y_t = \mu + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \theta_2 \varepsilon_{t-2} + \ldots + \theta_q \varepsilon_{t-q}$

Where:

• $y_t$ = observed value at time $t$
• $\mu$ = mean of the time series
• $\varepsilon_t$ = white noise error term at time $t$
• $\theta_1, \theta_2, \ldots, \theta_q$ = moving average coefficients (these are the parameters you estimate)

For a concrete example, an $MA(1)$ model looks like $y_t = \mu + \varepsilon_t + \theta_1 \varepsilon_{t-1}$. The current value depends on the mean, the current shock, and one past shock weighted by $\theta_1$.

Parameter Estimation for MA Models

Estimating MA parameters is trickier than for AR models because the error terms $\varepsilon_t$ are not directly observed. Two main approaches are used:

Least Squares Estimation

• Minimizes the sum of squared residuals between observed and predicted values.
• Provides estimates for the moving average coefficients ($\theta$) and the error variance.
• Because past errors aren't observed, this typically requires an iterative procedure where initial error values are assumed (often set to zero) and then refined.

Maximum Likelihood Estimation (MLE)

• Finds the parameter values that make the observed data most probable under the assumed model.
• Also iterative, but generally produces more efficient estimates (lower variance) than least squares, especially for shorter series.
• MLE is the default in most statistical software packages for fitting MA models.

The choice between methods matters less as your sample size grows. For short series, MLE tends to perform better.

Invertible vs. Non-Invertible MA Models

Invertibility is a condition that guarantees a unique MA model representation for a given autocorrelation structure. Without it, multiple different sets of parameters could produce the same autocorrelation pattern, which creates ambiguity.

An invertible MA model can be rewritten as an infinite-order AR model. This is the MA analog of stationarity for AR models: just as we require AR models to be stationary, we require MA models to be invertible.

Invertibility conditions for $MA(q)$:

• The roots of the characteristic polynomial $1 + \theta_1 z + \theta_2 z^2 + \ldots + \theta_q z^q = 0$ must all lie outside the unit circle (i.e., have absolute value greater than 1).
• For the simple $MA(1)$ case, this reduces to $|\theta_1| < 1$.

When invertibility fails:

• Multiple parameter sets can generate the same ACF, so you can't uniquely identify the model.
• Forecasting becomes unreliable because the mapping between errors and observations is ambiguous.
• In practice, most software will estimate the invertible version by default, but it's worth checking.

Application of MA Models to Data

Fitting an MA model follows a structured workflow:

1. Identify the model order by examining the autocorrelation function (ACF) and partial autocorrelation function (PACF). For a pure $MA(q)$ process, the ACF cuts off sharply after lag $q$ (drops to zero), while the PACF decays gradually. This sharp cutoff in the ACF is the signature clue that you're dealing with an MA process.
2. Estimate parameters using MLE or least squares.
3. Check model fit with diagnostic tools: plot the residuals to verify they look like white noise, run a Ljung-Box test for remaining autocorrelation, and compare models using AIC or BIC (lower is better).
4. Forecast using the fitted model and interpret results in context.

Interpreting the coefficients:

• Each $\theta_i$ represents how much a past forecast error at lag $i$ influences the current observation.
• A positive $\theta_1$ means that when the model under-predicted in the previous period (positive error), the current value tends to be pushed higher as well.
• A negative $\theta_1$ means past positive errors tend to pull the current value down, creating a corrective or oscillating pattern.
• Larger absolute values of $\theta$ indicate stronger influence from that lag's error on the present observation.