8.3 Autoregressive models

Autoregressive models are a key tool in econometrics for analyzing time series data. They capture how past values of a variable influence its current value, allowing economists to model and forecast economic trends.

AR models assume that a variable's current value depends linearly on its past values plus random error. By estimating these relationships, economists can understand persistence in economic data and make short-term predictions about future values.

Definition of autoregressive models

• Autoregressive (AR) models are a class of time series models used to capture the linear dependence of a variable on its own past values
• AR models are widely used in econometrics to model and forecast economic and financial time series data
• The basic idea behind AR models is that the current value of a variable can be expressed as a linear combination of its past values plus an error term

Components of AR models

Dependent variable vs lagged variables

• In an AR model, the dependent variable is the current value of the time series being modeled
• Lagged variables, also known as autoregressive terms, are the past values of the dependent variable used as predictors
• The number of lagged variables included in the model determines the order of the AR model (e.g., AR(1), AR(2), etc.)

Autoregressive coefficients

• Autoregressive coefficients are the parameters that determine the relationship between the dependent variable and its lagged values
• These coefficients indicate the magnitude and direction of the influence of past values on the current value
• The coefficients are estimated using statistical methods such as ordinary least squares (OLS) or maximum likelihood estimation (MLE)

Error term assumptions

• The error term in an AR model represents the random shock or innovation that is not explained by the lagged variables
• For valid inference and estimation, the error term is assumed to be independently and identically distributed (i.i.d.) with a mean of zero and constant variance
• The error term is also assumed to be uncorrelated with the lagged variables and normally distributed

Stationarity in AR models

Definition of stationarity

• Stationarity is a crucial assumption in AR models, which means that the statistical properties of the time series (mean, variance, and autocovariance) do not change over time
• A stationary time series exhibits constant mean and variance, and the covariance between any two observations depends only on the time lag between them
• Stationarity ensures that the relationships estimated by the AR model are stable and reliable

Importance for valid inference

• Stationarity is essential for valid inference in AR models because non-stationary data can lead to spurious regression results
• If the time series is non-stationary, the estimated coefficients and standard errors may be biased and inconsistent, leading to incorrect conclusions
• Stationarity allows for meaningful interpretation of the model coefficients and enables accurate forecasting

Testing for stationarity

• Various statistical tests can be used to assess the stationarity of a time series before fitting an AR model
• Common tests include the Augmented Dickey-Fuller (ADF) test, Phillips-Perron (PP) test, and Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test
• These tests examine the presence of unit roots in the time series, which indicate non-stationarity
• If the time series is found to be non-stationary, differencing or other transformations may be applied to achieve stationarity

Estimation of AR models

Ordinary least squares (OLS)

• OLS is a widely used method for estimating the coefficients of an AR model
• It minimizes the sum of squared residuals between the observed values and the predicted values based on the lagged variables
• OLS provides unbiased and consistent estimates of the coefficients under certain assumptions (e.g., no autocorrelation in the error term)
• The OLS estimator is computationally simple and has desirable statistical properties when the assumptions are met

Maximum likelihood estimation (MLE)

• MLE is an alternative method for estimating the coefficients of an AR model
• It finds the parameter values that maximize the likelihood function, which measures the probability of observing the data given the model
• MLE is particularly useful when the error term follows a non-normal distribution or when there are missing observations in the data
• MLE estimates are asymptotically efficient and have desirable properties, such as consistency and asymptotic normality

Model selection for AR models

Determining AR order

• Selecting the appropriate order (number of lagged variables) is crucial in building an AR model
• The order determines the complexity of the model and affects its ability to capture the dynamics of the time series
• Various techniques can be used to determine the optimal AR order, such as examining the partial autocorrelation function (PACF) or using information criteria

Information criteria (AIC, BIC)

• Information criteria, such as the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), are commonly used for model selection in AR models
• These criteria balance the goodness of fit of the model with the number of parameters estimated
• AIC and BIC assign penalties for the number of parameters, favoring more parsimonious models
• The model with the lowest AIC or BIC value is generally selected as the preferred model

Diagnostic checking of AR models

Residual analysis

• Residual analysis involves examining the properties of the residuals (estimated error terms) from the fitted AR model
• The residuals should be uncorrelated, normally distributed, and have constant variance (homoscedasticity)
• Plotting the residuals against time or the fitted values can help identify patterns or anomalies that may indicate model misspecification
• Statistical tests, such as the Durbin-Watson test or the Breusch-Godfrey test, can be used to detect autocorrelation in the residuals

Ljung-Box test for autocorrelation

• The Ljung-Box test is a commonly used diagnostic test for assessing the presence of autocorrelation in the residuals of an AR model
• It tests the null hypothesis that the residuals are independently distributed against the alternative hypothesis of autocorrelation
• The test statistic is based on the sample autocorrelation coefficients of the residuals up to a specified lag
• A significant test result indicates the presence of autocorrelation, suggesting that the AR model may not adequately capture the dynamics of the time series

Forecasting with AR models

One-step ahead forecasts

• One-step ahead forecasting involves predicting the value of the time series one period ahead based on the estimated AR model
• The forecast is obtained by plugging in the observed values of the lagged variables and the estimated coefficients into the AR equation
• One-step ahead forecasts are relatively straightforward to compute and can be used for short-term prediction

Multi-step ahead forecasts

• Multi-step ahead forecasting involves predicting the values of the time series multiple periods ahead based on the estimated AR model
• The forecasts are generated iteratively, using the previously forecasted values as inputs for subsequent forecasts
• Multi-step ahead forecasting becomes more challenging as the forecast horizon increases due to the accumulation of forecast errors
• Techniques such as bootstrapping or simulation can be used to obtain prediction intervals and assess the uncertainty associated with multi-step forecasts

Advantages vs disadvantages of AR models

• Advantages of AR models include their simplicity, interpretability, and ability to capture the linear dependence structure of a time series
• AR models are particularly useful for short-term forecasting and can provide accurate predictions when the underlying assumptions are met
• Disadvantages of AR models include their inability to capture non-linear relationships or handle structural breaks in the data
• AR models may not be suitable for long-term forecasting or for time series with complex dynamics or external factors influencing the variable of interest

Extensions of AR models

Autoregressive moving average (ARMA)

• ARMA models extend AR models by incorporating moving average (MA) terms, which capture the dependence of the current value on past error terms
• ARMA models combine the autoregressive and moving average components to provide a more flexible and parsimonious representation of the time series
• The order of an ARMA model is specified by the number of AR and MA terms included (e.g., ARMA(p,q))
• ARMA models can handle a wider range of time series patterns compared to pure AR models

Vector autoregressive (VAR) models

• VAR models extend the concept of AR models to multivariate time series analysis
• In a VAR model, each variable is modeled as a linear function of its own past values and the past values of other variables in the system
• VAR models capture the dynamic interactions and feedback effects among multiple time series variables
• VAR models are commonly used in macroeconomic analysis to study the relationships and transmission mechanisms among economic variables

Applications of AR models in econometrics

Modeling economic time series

• AR models are widely used to model various economic time series, such as GDP growth, inflation rates, unemployment rates, and exchange rates
• AR models can capture the persistence and cyclical behavior often observed in economic variables
• By estimating AR models, economists can analyze the dynamic properties of economic time series and study the impact of past values on current outcomes

Forecasting financial variables

• AR models are also applied in financial econometrics to forecast variables such as stock prices, returns, volatility, and trading volumes
• AR models can capture the autocorrelation and persistence in financial time series, which can be exploited for short-term forecasting
• AR models can be used in conjunction with other techniques, such as GARCH models, to account for time-varying volatility in financial markets
• Forecasts from AR models can assist in risk management, portfolio optimization, and trading strategies in financial applications