📉Statistical Methods for Data Science Unit 13 – Time Series Analysis & Forecasting

Key Concepts in Time Series

• Time series data consists of observations collected sequentially over time at regular intervals (hourly, daily, monthly)
• Time series analysis examines patterns, trends, and seasonality in data to make predictions about future values
• Stationarity assumes the statistical properties of a time series remain constant over time (mean, variance, autocorrelation)
  • Non-stationary data requires transformations (differencing, logarithmic) to achieve stationarity before modeling
• Autocorrelation measures the correlation between a time series and its lagged values
  • Positive autocorrelation indicates persistence, while negative autocorrelation suggests mean reversion
• White noise is a purely random time series with no discernible patterns or correlations
• Forecasting involves predicting future values based on historical data and identified patterns
• Time series models include autoregressive (AR), moving average (MA), and autoregressive integrated moving average (ARIMA)

Components of Time Series Data

• Trend represents the long-term increase or decrease in the data over time
  • Can be linear, exponential, or polynomial in nature
• Seasonality refers to regular, predictable fluctuations that occur within a fixed period (weekly, monthly, yearly)
  • Seasonal patterns can be additive (constant amplitude) or multiplicative (varying amplitude)
• Cyclical component captures irregular fluctuations lasting more than a year, often related to economic or business cycles
• Irregular or residual component represents random, unpredictable fluctuations not captured by other components
• Decomposition techniques (additive, multiplicative) separate a time series into its constituent components for analysis
• Smoothing methods (moving average, exponential smoothing) help isolate the trend and seasonality by reducing noise
• Seasonal adjustment removes the seasonal component to focus on the underlying trend and cyclical behavior

Stationarity and Its Importance

• Stationarity is a critical assumption for many time series models, as it simplifies the modeling process
• Stationary time series have constant mean, variance, and autocorrelation over time
  • Enables more accurate forecasting and statistical inference
• Non-stationary data can lead to spurious correlations and unreliable model results
• Unit root tests (Dickey-Fuller, KPSS) assess stationarity by examining the presence of a trend or drift in the data
• Differencing removes the trend by computing the differences between consecutive observations
  • First-order differencing calculates the change between each observation and its previous value
  • Higher-order differencing may be necessary for more complex trends
• Logarithmic transformations stabilize the variance of a time series with increasing or decreasing volatility
• Rolling statistics (mean, variance) help identify changes in the statistical properties of a time series over time

Time Series Decomposition Methods

• Decomposition separates a time series into its constituent components (trend, seasonality, cyclical, irregular)
• Additive decomposition assumes the components are independent and can be summed to form the original series: $Y_t = T_t + S_t + C_t + I_t$
  • Suitable when the seasonal fluctuations have a constant amplitude over time
• Multiplicative decomposition assumes the components interact with each other: $Y_t = T_t \times S_t \times C_t \times I_t$
  • Appropriate when the seasonal fluctuations vary proportionally with the level of the series
• Classical decomposition iteratively estimates and removes the trend and seasonal components using moving averages
• STL (Seasonal and Trend decomposition using Loess) is a robust method that handles missing values and outliers
  • Uses locally weighted regression (Loess) to estimate the trend and seasonal components
• X-11 and X-12-ARIMA are widely used decomposition methods developed by the U.S. Census Bureau
• Decomposition helps identify the underlying patterns and facilitates the selection of appropriate forecasting models

Autocorrelation and Partial Autocorrelation

• Autocorrelation (ACF) measures the linear dependence between a time series and its lagged values
  • Helps identify the presence and strength of serial correlation in the data
• Partial autocorrelation (PACF) measures the correlation between a time series and its lagged values, controlling for the effects of intermediate lags
  • Useful for determining the order of an autoregressive (AR) model
• ACF and PACF plots visually represent the autocorrelation and partial autocorrelation at different lag lengths
  • Significant spikes indicate the presence of correlation at the corresponding lags
• Ljung-Box test assesses the overall significance of autocorrelations in a time series
  • Null hypothesis: the data is independently distributed (no autocorrelation)
• Durbin-Watson test checks for the presence of first-order autocorrelation in the residuals of a regression model
• Autocorrelation and partial autocorrelation help identify the appropriate order of ARMA models
• Seasonal differencing may be necessary to remove seasonal autocorrelation before modeling

ARIMA Models and Their Variations

• ARIMA (Autoregressive Integrated Moving Average) models combine autoregressive (AR), differencing (I), and moving average (MA) components
  • Suitable for modeling non-seasonal, stationary time series
• AR(p) component represents the relationship between an observation and its p lagged values
  • $Y_t = c + \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + ... + \phi_p Y_{t-p} + \epsilon_t$
• MA(q) component models the relationship between an observation and the past q forecast errors
  • $Y_t = c + \epsilon_t + \theta_1 \epsilon_{t-1} + \theta_2 \epsilon_{t-2} + ... + \theta_q \epsilon_{t-q}$
• Integrated (I) component represents the degree of differencing required to achieve stationarity
• ARIMA(p,d,q) notation specifies the order of the AR, I, and MA components
• SARIMA (Seasonal ARIMA) extends ARIMA to handle seasonal patterns by including seasonal AR, I, and MA terms
• ARIMAX (ARIMA with exogenous variables) incorporates external factors (holidays, promotions) into the model
• Box-Jenkins methodology is a systematic approach for identifying, estimating, and diagnosing ARIMA models

Advanced Forecasting Techniques

• Exponential smoothing methods assign exponentially decreasing weights to past observations
  • Simple exponential smoothing (SES) is suitable for data with no trend or seasonality
  • Holt's linear trend method extends SES to capture trends in the data
  • Holt-Winters' method incorporates both trend and seasonality (additive or multiplicative)
• TBATS (Trigonometric seasonality, Box-Cox transformation, ARMA errors, Trend, and Seasonal components) is a flexible exponential smoothing method
  • Handles complex seasonal patterns, including non-integer seasonality and calendar effects
• Neural networks (NN) and deep learning models (LSTM, GRU) can capture non-linear relationships in time series data
  • Require large amounts of data and careful hyperparameter tuning to avoid overfitting
• Ensemble methods combine multiple models to improve forecast accuracy and robustness
  • Simple averaging, weighted averaging, or stacking can be used to combine individual model forecasts
• Hierarchical forecasting reconciles forecasts at different levels of aggregation (product, region, overall)
  • Top-down, bottom-up, and middle-out approaches distribute the forecasts across the hierarchy

Evaluating Forecast Accuracy

• Forecast accuracy measures the discrepancy between the predicted and actual values
• Scale-dependent metrics (MAE, RMSE) express the error in the same units as the data
  • Mean Absolute Error (MAE): $\frac{1}{n} \sum_{t=1}^n |y_t - \hat{y}_t|$
  • Root Mean Squared Error (RMSE): $\sqrt{\frac{1}{n} \sum_{t=1}^n (y_t - \hat{y}_t)^2}$
• Percentage errors (MAPE, sMAPE) provide scale-independent measures of accuracy
  • Mean Absolute Percentage Error (MAPE): $\frac{100%}{n} \sum_{t=1}^n |\frac{y_t - \hat{y}_t}{y_t}|$
  • Symmetric Mean Absolute Percentage Error (sMAPE): $\frac{200%}{n} \sum_{t=1}^n \frac{|y_t - \hat{y}_t|}{|y_t| + |\hat{y}_t|}$
• Theil's U statistic compares the performance of a forecasting model to a naive benchmark (random walk)
  • U < 1 indicates the model outperforms the naive forecast, while U > 1 suggests the opposite
• Cross-validation techniques (rolling origin, time series) assess the model's performance on unseen data
• Residual diagnostics (ACF, PACF, Q-Q plots) help identify any remaining patterns or autocorrelation in the forecast errors
• Forecast value added (FVA) measures the improvement in accuracy compared to a simpler or naive model

Practical Applications and Case Studies

• Demand forecasting predicts future product demand to optimize inventory management and production planning
  • Retail sales, supply chain management, and manufacturing benefit from accurate demand forecasts
• Financial forecasting estimates future financial performance, risk, and economic conditions
  • Stock price prediction, portfolio optimization, and risk management rely on time series analysis
• Energy load forecasting helps utility companies balance supply and demand, ensuring a stable power grid
  • Short-term (hourly, daily) and long-term (monthly, yearly) forecasts inform operational and strategic decisions
• Weather forecasting predicts future weather conditions based on historical data and meteorological models
  • Accurate forecasts are crucial for agriculture, transportation, and disaster preparedness
• Economic forecasting projects future economic indicators (GDP, inflation, unemployment) to guide policy decisions
  • Central banks and governments use economic forecasts to set monetary and fiscal policies
• Marketing and sales forecasting helps businesses allocate resources and plan promotional activities
  • Customer demand, market trends, and competitor actions inform marketing strategies and budgets
• Healthcare and epidemiology use time series analysis to monitor and predict disease outbreaks and patient volumes
  • Early detection and intervention can help control the spread of infectious diseases and optimize resource allocation