2.4 Stationarity and Differencing

Time series analysis hinges on stationarity, where statistical properties remain constant over time. This concept is crucial for accurate modeling and forecasting. Non-stationary data can lead to misleading results, so identifying and addressing it is key.

Differencing is a powerful tool to achieve stationarity by removing trends and seasonality. It involves calculating differences between observations, transforming the data into a more stable form. Proper application of differencing techniques can significantly improve the reliability of time series models.

Stationarity in Time Series

Understanding Stationarity

• Stationarity is a critical assumption in time series analysis where the statistical properties of a time series remain constant over time
• A stationary time series has constant mean, variance, and autocovariance structure, which allows for more accurate modeling and forecasting
  • For example, a stationary time series of daily stock returns would have a consistent average return and volatility over time
• Non-stationary time series can lead to spurious correlations and unreliable forecasts, making it essential to identify and address non-stationarity before modeling
• Stationarity is a prerequisite for many time series models, such as ARIMA and SARIMA, which assume that the underlying process generating the data is stationary

Sources of Non-Stationarity

• Trend and seasonality are common sources of non-stationarity in time series data, and they need to be removed or accounted for to achieve stationarity
  • A time series with an upward trend, such as increasing sales over years, would be non-stationary
  • Seasonal patterns, like higher ice cream sales in summer months, also introduce non-stationarity
• Other sources of non-stationarity include changes in variance over time (heteroscedasticity) and structural breaks or regime shifts in the data generating process
• Ignoring non-stationarity can lead to misleading results and poor forecast performance, emphasizing the importance of assessing and addressing it in time series analysis

Assessing Time Series Stationarity

Visual Inspection

• Visual inspection of time series plots can provide initial insights into the presence of trend, seasonality, and changes in variance over time, which are indicators of non-stationarity
  • Plotting the time series against time can reveal obvious trends or seasonal patterns
  • Changes in the spread or volatility of the data over time can also be visually detected
• The autocorrelation function (ACF) plot can help identify non-stationarity by showing slow decay or persistence in the autocorrelations at different lags
  • For a stationary time series, the ACF should decay quickly to zero as the lag increases
  • Slowly decaying or persistent ACF values suggest the presence of non-stationarity

Statistical Tests

• Statistical tests, such as the Augmented Dickey-Fuller (ADF) test and the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test, can formally assess the presence of unit roots and stationarity in a time series
  • The ADF test has a null hypothesis of non-stationarity (presence of a unit root), while the alternative hypothesis is stationarity
  • The KPSS test has a null hypothesis of stationarity, while the alternative hypothesis is non-stationarity (presence of a unit root)
• The ADF and KPSS tests can be used in conjunction to cross-validate the results and make more robust conclusions about the stationarity of a time series
  • If both tests agree on the stationarity or non-stationarity of the series, the conclusion is more reliable
  • Conflicting results may indicate the need for further investigation or the use of alternative methods

Differencing for Stationarity

First-Order Differencing

• Differencing is a common technique used to remove trend and seasonality from non-stationary time series, making them stationary
• First-order differencing involves computing the differences between consecutive observations in a time series, which can help eliminate linear trends
  • For example, if the original time series is $y_t$, the first-order differenced series would be $\Delta y_t = y_t - y_{t-1}$
• The differenced time series represents the changes or growth rates in the original time series, rather than the absolute values
  • Interpreting the differenced series requires considering the original units of measurement and the time interval between observations

Seasonal Differencing

• Seasonal differencing involves computing the differences between observations separated by a fixed seasonal period, which can help remove seasonal patterns
  • For monthly data with an annual seasonal cycle, seasonal differencing would involve subtracting the value from 12 months prior: $\Delta_{12} y_t = y_t - y_{t-12}$
• Seasonal differencing can be applied in addition to first-order differencing to remove both trend and seasonality
• The choice of the seasonal period depends on the frequency of the data and the observed seasonal patterns
  • Quarterly data may require seasonal differencing with a period of 4, while weekly data may use a period of 52

Avoiding Over-Differencing

• Differencing can be applied multiple times if necessary, depending on the complexity of the trend and seasonality in the data
• However, over-differencing should be avoided, as it can introduce unnecessary noise and complicate the modeling process
  • Over-differenced series may exhibit rapid oscillations or appear "rough" compared to the original data
• It is essential to assess the stationarity of the differenced series and stop differencing once stationarity is achieved

Order of Differencing

Determining the Appropriate Order

• The order of differencing refers to the number of times differencing is applied to a time series to achieve stationarity
• The appropriate order of differencing can be determined by iteratively applying differencing and assessing the resulting time series for stationarity using visual and statistical methods
  • After each round of differencing, plot the differenced series and examine the ACF and PACF plots for signs of stationarity
  • Perform statistical tests (ADF and KPSS) on the differenced series to confirm stationarity
• In practice, it is common to use a combination of first-order and seasonal differencing to achieve stationarity in time series with both trend and seasonality
  • For example, a monthly time series with a linear trend and annual seasonality may require first-order differencing ($\Delta y_t$) followed by seasonal differencing ($\Delta_{12} \Delta y_t$)

Guidance from ACF and PACF Plots

• The ACF and PACF (partial autocorrelation function) plots can provide guidance on the required order of differencing
  • If the ACF decays slowly and the PACF has a significant spike at lag 1, first-order differencing may be sufficient
  • If the ACF and PACF exhibit seasonal patterns, seasonal differencing may be necessary
• The goal is to achieve a stationary series where the ACF and PACF plots show rapid decay and no significant spikes at higher lags
• The ADF and KPSS tests can be used to confirm the stationarity of the differenced time series and determine if further differencing is required