🎲Data, Inference, and Decisions Unit 9 – Time Series Analysis & Forecasting

Key Concepts and Terminology

• Time series data consists of observations collected sequentially over time, often at regular intervals (hourly, daily, monthly)
• Trend refers to the long-term increase or decrease in the data over time
  • Can be linear, exponential, or more complex patterns
• Seasonality captures recurring patterns or cycles within the data, such as daily, weekly, or annual cycles
• Autocorrelation measures the correlation between observations at different time lags
  • Positive autocorrelation indicates observations are similar to nearby observations
  • Negative autocorrelation suggests observations are dissimilar to nearby observations
• White noise is a series of uncorrelated random variables with constant mean and variance
• Differencing involves computing the differences between consecutive observations to remove trend and seasonality
• Stationarity is a property where the statistical properties (mean, variance, autocorrelation) remain constant over time

Components of Time Series Data

• Level represents the average value or baseline around which the series fluctuates
• Trend captures the long-term increase or decrease in the data
  • Can be modeled using linear, polynomial, or exponential functions
• Seasonality refers to recurring patterns or cycles within the data
  • Additive seasonality assumes the seasonal effect is constant over time
  • Multiplicative seasonality assumes the seasonal effect varies with the level of the series
• Cyclical component captures longer-term fluctuations or business cycles, typically over several years
• Irregular or residual component represents the random, unpredictable fluctuations in the data
  • Often modeled as white noise or a stochastic process
• Decomposition methods, such as classical decomposition or STL (Seasonal and Trend decomposition using Loess), can be used to separate these components

Exploratory Data Analysis for Time Series

• Plotting the time series helps identify trends, seasonality, outliers, and structural breaks
  • Line plots display observations over time
  • Seasonal subseries plots group observations by seasonal periods to assess seasonal patterns
• Summary statistics, such as mean, variance, and autocorrelation, provide insights into the properties of the series
• Autocorrelation Function (ACF) plots the correlation between observations at different time lags
  • Helps identify the order of autoregressive models
• Partial Autocorrelation Function (PACF) plots the correlation between observations at different lags, controlling for intermediate lags
  • Helps identify the order of moving average models
• Spectral analysis or periodograms can reveal hidden periodicities or cycles in the data
• Identifying and handling missing values, outliers, and structural breaks is crucial for accurate modeling and forecasting

Stationarity and Transformations

• Stationarity is a key assumption for many time series models
  • A stationary series has constant mean, variance, and autocorrelation over time
• Visual inspection of the time series plot can provide initial insights into stationarity
• Statistical tests, such as the Augmented Dickey-Fuller (ADF) test or Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test, can formally assess stationarity
• Differencing is a common technique to remove trend and seasonality and achieve stationarity
  • First-order differencing computes the differences between consecutive observations
  • Seasonal differencing computes the differences between observations separated by a seasonal period
• Logarithmic or power transformations can stabilize the variance of the series
• Detrending methods, such as linear regression or polynomial regression, can remove deterministic trends

Time Series Models and Techniques

• Autoregressive (AR) models express the current observation as a linear combination of past observations
  • The order of an AR model, denoted as AR(p), indicates the number of lagged observations used
• Moving Average (MA) models express the current observation as a linear combination of past forecast errors
  • The order of an MA model, denoted as MA(q), indicates the number of lagged forecast errors used
• Autoregressive Moving Average (ARMA) models combine AR and MA components
  • ARMA(p, q) models include p autoregressive terms and q moving average terms
• Autoregressive Integrated Moving Average (ARIMA) models extend ARMA models to handle non-stationary series
  • ARIMA(p, d, q) models include p autoregressive terms, d differencing operations, and q moving average terms
• Seasonal ARIMA (SARIMA) models incorporate seasonal components into the ARIMA framework
  • SARIMA(p, d, q)(P, D, Q)m models include seasonal autoregressive, differencing, and moving average terms
• Exponential Smoothing methods, such as Simple Exponential Smoothing (SES), Holt's Linear Trend, and Holt-Winters' Seasonal methods, use weighted averages of past observations for forecasting

Forecasting Methods and Evaluation

• Forecasting involves predicting future values of a time series based on historical data
• Rolling origin or time series cross-validation can be used to assess the performance of forecasting models
  • Data is split into training and testing sets, and the model is repeatedly trained and evaluated on different subsets
• Forecast accuracy measures, such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE), quantify the difference between forecasted and actual values
• Residual diagnostics, such as the Ljung-Box test or the Durbin-Watson test, assess the independence and randomness of forecast errors
• Forecast intervals provide a range of likely future values, accounting for uncertainty
  • Prediction intervals consider the uncertainty in future observations
  • Confidence intervals consider the uncertainty in the model parameters
• Ensemble methods, such as bagging or boosting, can improve forecast accuracy by combining multiple models

Practical Applications and Case Studies

• Demand forecasting helps businesses optimize inventory management and production planning
  • Forecasting sales, customer traffic, or product demand based on historical data
• Economic forecasting assists in predicting macroeconomic indicators, such as GDP, inflation, or unemployment rates
  • Guiding monetary and fiscal policies based on forecasted economic conditions
• Energy load forecasting enables utility companies to balance supply and demand
  • Predicting electricity consumption based on historical usage patterns, weather, and other factors
• Financial market forecasting supports investment decisions and risk management
  • Forecasting stock prices, exchange rates, or volatility using historical price and volume data
• Epidemiological forecasting helps predict the spread and impact of infectious diseases
  • Modeling disease transmission dynamics and forecasting case counts or hospitalizations
• Supply chain forecasting optimizes inventory levels and minimizes stockouts or overstocking
  • Forecasting demand, lead times, and supplier performance based on historical data and external factors

Common Pitfalls and Limitations

• Overfitting occurs when a model is too complex and fits the noise in the data rather than the underlying pattern
  • Regularization techniques, such as L1 (Lasso) or L2 (Ridge) regularization, can help mitigate overfitting
• Underfitting happens when a model is too simple and fails to capture the true patterns in the data
  • Increasing model complexity or incorporating additional relevant features can address underfitting
• Outliers and anomalies can distort the modeling and forecasting process
  • Robust methods, such as median-based models or outlier detection techniques, can help handle outliers
• Structural breaks or regime changes can invalidate the assumptions of time series models
  • Piecewise modeling or change point detection methods can adapt to structural breaks
• Multicollinearity among predictors can lead to unstable or unreliable forecasts
  • Variable selection techniques, such as stepwise regression or Lasso, can identify relevant predictors
• Limited historical data or short time series can hinder the accuracy and reliability of forecasts
  • Incorporating external data sources or using transfer learning can improve forecasting performance