ARIMA Model Parameters

Why This Matters

ARIMA models are the workhorses of time series forecasting, and understanding their parameters is essential for any exam question involving model specification, diagnostics, or prediction. You're being tested on your ability to connect parameter choices to data characteristics—knowing that a time series with a strong trend needs differencing (the I component), or that a slowly decaying ACF suggests autoregressive behavior (the AR component), separates students who memorize from those who truly understand.

The concepts here span stationarity, autocorrelation structure, model parsimony, and diagnostic validation. These aren't isolated facts—they form a complete workflow from raw data to reliable forecasts. Don't just memorize that $p$ is the AR parameter; know how to identify the right $p$ from a PACF plot and why choosing too high a value leads to overfitting. That's the level of understanding that earns full credit on FRQs.

---

The Core Parameters: Building Blocks of ARIMA

Every ARIMA model is defined by three parameters—$p$, $d$, and $q$—written as ARIMA($p$, $d$, $q$). Each addresses a different aspect of time series behavior: autoregressive memory, trend removal, and error smoothing.

AR (Autoregressive) Parameter ($p$)

• Captures how past values influence the present—the model uses $p$ lagged observations as predictors, essentially saying "today depends on yesterday, the day before, etc."
• Higher $p$ values model more complex dependencies but risk overfitting; start simple and increase only if diagnostics demand it
• Identified using the PACF—significant spikes at lags 1 through $k$ followed by cutoff suggest $p = k$

I (Integrated) Parameter ($d$)

• Represents differencing operations needed to achieve stationarity—each difference subtracts the previous value: $y'_t = y_t - y_{t-1}$
• Typical values are 0, 1, or 2; $d = 1$ removes linear trends, $d = 2$ handles quadratic trends (higher values are rare and often signal model misspecification)
• Over-differencing introduces problems—if $d$ is too high, you create artificial patterns and inflate variance

MA (Moving Average) Parameter ($q$)

• Models the relationship between observations and past forecast errors—the series depends on $q$ lagged residual terms from a moving average process
• Smooths short-term fluctuations by averaging out random shocks, helping isolate underlying signal from noise
• Identified using the ACF—significant spikes at lags 1 through $k$ followed by cutoff suggest $q = k$

---

Stationarity: The Foundation of Valid Modeling

ARIMA assumes stationarity after differencing. Without it, parameter estimates become unreliable and forecasts meaningless. Stationarity means the statistical properties of the series don't change over time.

Stationarity Requirement

• A stationary series has constant mean, variance, and autocovariance over time—no trends, no changing volatility, no structural breaks
• Non-stationary data produces spurious results—you might find "significant" relationships that are actually just shared trends (classic exam trap)
• Achieved through differencing or transformation—differencing removes trends; log or Box-Cox transformations stabilize variance

---

Diagnostic Tools: Identifying the Right Parameters

Before fitting a model, you need to determine appropriate values for $p$ and $q$. The ACF and PACF are your primary diagnostic tools, each revealing different aspects of the correlation structure.

ACF (Autocorrelation Function)

• Measures correlation between $y_t$ and $y_{t-k}$ for all lags $k$—shows the total linear relationship including indirect effects through intermediate lags
• Key pattern for MA processes: ACF cuts off sharply after lag $q$, while PACF decays gradually
• Slow decay in ACF suggests non-stationarity or strong AR component—this is a critical diagnostic signal

PACF (Partial Autocorrelation Function)

• Measures correlation between $y_t$ and $y_{t-k}$ after removing effects of intermediate lags—isolates the direct relationship at each lag
• Key pattern for AR processes: PACF cuts off sharply after lag $p$, while ACF decays gradually
• Significant spike at lag $k$ only suggests including that specific lag in the model

---

Model Selection and Validation

Choosing between competing models and verifying your final choice requires formal criteria and diagnostic checks. A good model balances fit against complexity.

Model Selection Criteria (AIC, BIC)

• AIC and BIC quantify the trade-off between goodness-of-fit and model complexity—both penalize additional parameters to prevent overfitting
• Lower values indicate better models—compare AIC (or BIC) across candidate models with different $(p, d, q)$ combinations
• BIC penalizes complexity more heavily than AIC, often selecting simpler models; use AIC when prediction accuracy matters most, BIC when parsimony is priority

Residual Analysis

• Residuals should behave like white noise—no autocorrelation, constant variance, approximately normal distribution
• Ljung-Box test checks for remaining autocorrelation; significant results indicate model inadequacy
• Patterns in residual plots reveal model failures—trends suggest wrong $d$, autocorrelation suggests wrong $p$ or $q$

---

Extensions and Applications

Once you understand basic ARIMA, you can extend to seasonal data and apply models to generate forecasts with uncertainty quantification.

Forecasting with ARIMA Models

• Forecasts combine AR and MA components—predicted values use recent observations (AR) and recent forecast errors (MA) with estimated coefficients
• Uncertainty grows with forecast horizon—confidence intervals widen as you predict further into the future
• Point forecasts eventually converge to the mean—for stationary series, long-horizon forecasts approach the unconditional mean

Seasonal ARIMA (SARIMA) Parameters

• Adds seasonal components $(P, D, Q)_s$ to capture repeating patterns at seasonal lag $s$ (e.g., $s = 12$ for monthly data with yearly cycles)
• Full notation is ARIMA$(p, d, q)(P, D, Q)_s$—non-seasonal parameters handle short-term dynamics, seasonal parameters handle periodic patterns
• Seasonal differencing ($D$) removes seasonal trends; $D = 1$ means subtracting the value from $s$ periods ago: $y'_t = y_t - y_{t-s}$

---

Quick Reference Table

---

Self-Check Questions

1. You observe an ACF that decays slowly and a PACF with a significant spike only at lag 1. What ARIMA order would you specify, and why might you need to difference first?

2. Compare how you would use ACF vs. PACF to determine the $p$ and $q$ parameters. Which plot identifies which parameter?

3. A colleague fits ARIMA(3,1,3) and ARIMA(1,1,1) to the same data. The complex model has slightly lower AIC but higher BIC. Which would you choose and under what circumstances?

4. What three properties should residuals exhibit if your ARIMA model is adequate? What does autocorrelation in residuals suggest about your parameter choices?

5. Explain the difference between the $d$ parameter and the $D$ parameter in a SARIMA model. When would you need both to be non-zero?