⏳Intro to Time Series Unit 6 Review

Autoregressive (AR) models predict the current value of a time series using its own past values. They're one of the foundational building blocks of ARIMA modeling, and understanding them well makes the rest of Unit 6 much easier to follow.

The core idea is simple: what happened recently in a time series often tells you something about what's happening now. AR models formalize that intuition by assuming a linear relationship between the current observation and some number of previous observations.

Key characteristics of AR models

The time series must be stationary, meaning its mean, variance, and autocorrelation structure don't change over time.
They capture autocorrelation, which is the correlation between an observation and its own lagged values.
The relationship between current and past values is assumed to be linear.
The goal is to describe the series's dynamics with as few parameters as possible.

Order of AR models

The order of an AR model, written as AR(p), tells you how many lagged terms the model uses. An AR(1) model uses only the immediately previous value; an AR(2) model uses the two most recent values, and so on.

The general equation for an AR(p) model:

$y_t = c + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \ldots + \phi_p y_{t-p} + \varepsilon_t$

Where:

$y_t$ is the current observation at time $t$
$c$ is a constant term (related to the series mean)
$\phi_1, \phi_2, \ldots, \phi_p$ are the autoregressive coefficients
$y_{t-1}, y_{t-2}, \ldots, y_{t-p}$ are the lagged observations
$\varepsilon_t$ is the error term (white noise) at time $t$

How to choose p: You determine the order by looking at the partial autocorrelation function (PACF) plot. The PACF shows the correlation between $y_t$ and $y_{t-k}$ after removing the effects of all intermediate lags. If the PACF has significant spikes at lags 1 and 2 but cuts off after that, an AR(2) model is a reasonable choice.

Parameter Estimation and Model Assessment

Parameter estimation for AR models

Once you've chosen the order, you need to estimate the coefficients ( $\phi_1, \phi_2, \ldots$ ) and the constant $c$ . Two main methods are used:

Ordinary Least Squares (OLS):

Minimizes the sum of squared residuals between observed and predicted values.
Computationally straightforward since it reduces to solving a system of linear equations.
Provides unbiased estimates when the errors are uncorrelated and have constant variance.

Maximum Likelihood Estimation (MLE):

Finds the parameter values that make the observed data most probable under the model.
More flexible than OLS because it can handle more complex error structures.
Requires numerical optimization, so it's more computationally intensive.

For most intro-level AR modeling with well-behaved data, OLS and MLE produce very similar results. OLS is the simpler starting point; MLE becomes more useful when error assumptions get more complicated.

Stationarity assessment of AR models

An AR model only makes sense if the series is stationary. Checking stationarity comes down to examining the autoregressive coefficients.

For an AR(1) model $y_t = c + \phi_1 y_{t-1} + \varepsilon_t$ :

Stationary if $|\phi_1| < 1$ . The effect of past shocks gradually fades.
Random walk if $\phi_1 = 1$ . Shocks persist forever, and the series drifts without reverting to a mean.
Explosive if $|\phi_1| > 1$ . Values grow without bound, which isn't realistic for forecasting.

For higher-order AR(p) models, you check the characteristic equation:

$1 - \phi_1 z - \phi_2 z^2 - \ldots - \phi_p z^p = 0$

The model is stationary if all roots of this equation lie outside the unit circle in the complex plane. You don't typically solve this by hand; statistical software handles it for you.

If your model turns out to be non-stationary, the standard fix is differencing the series (subtracting each value from the previous one) to induce stationarity before fitting the AR model. This is exactly where the "I" in ARIMA comes from.

Application and Interpretation

Autoregressive models in time series, Autocorrelation - wikidoc

Applying AR models step by step

Identify the order using the PACF plot, or use model selection criteria like AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion) to compare candidate models.
Estimate the parameters using OLS or MLE.
Assess the fit through diagnostic checks: examine residuals for remaining autocorrelation, check for normality, and look for patterns the model missed.
Forecast future values using the fitted model.

Interpreting the coefficients

A positive $\phi$ coefficient means the current value tends to move in the same direction as the lagged value. A negative coefficient means they tend to move in opposite directions.
Larger magnitudes (closer to 1) mean past values have a stronger, more persistent influence on the present. Smaller magnitudes mean the effect fades quickly.
The constant $c$ is related to the overall mean of the series. When all lagged terms are at zero, $c$ represents the baseline level.

Common applications

AR models are widely used for forecasting stock prices, exchange rates, and economic indicators. They also appear in energy demand modeling, sales forecasting, and web traffic analysis.

That said, AR models have real limitations. They assume linearity and rely solely on the series's own past. If external factors (like policy changes or weather events) drive the series, you may need to incorporate exogenous variables or use a more flexible model.

⏳Intro to Time Series Unit 6 Review