6.1 Autoregressive (AR) models

Autoregressive models are key tools in time series analysis. They help us understand how past values influence current observations in data like stock prices or energy consumption. AR models assume a linear relationship between current and past values, capturing the data's autocorrelation structure.

The order of an AR model, denoted as AR(p), shows how many past values are used to predict the current one. We estimate model parameters using methods like least squares or maximum likelihood. It's crucial to check if the model is stationary, ensuring its properties stay constant over time.

Introduction to Autoregressive (AR) Models

Autoregressive models in time series

• AR models describe the relationship between an observation and a number of lagged observations in a time series
• Assume the current value depends linearly on its own previous values
• The order of an AR model, AR(p), indicates the number of lagged terms included
• Key characteristics of AR models:
  • Assume the time series is stationary, meaning its statistical properties (mean, variance, autocorrelation) remain constant over time
  • Capture the autocorrelation structure, which is the correlation between an observation and its lagged values
  • Assume a linear relationship between the current observation and its lagged values
  • Aim to describe the time series using the minimum number of parameters while capturing the essential dynamics (stock prices, energy consumption)

Order of AR models

• The order of an AR model, AR(p), is determined by the number of lagged terms in the model equation
  • AR(1) model includes one lagged term, AR(2) model includes two lagged terms
• General form of an AR(p) model:
  • $y_t = c + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \ldots + \phi_p y_{t-p} + \varepsilon_t$
    • $y_t$ is the current observation at time t
    • $c$ is a constant term
    • $\phi_1, \phi_2, \ldots, \phi_p$ are the autoregressive coefficients
    • $y_{t-1}, y_{t-2}, \ldots, y_{t-p}$ are the lagged observations
    • $\varepsilon_t$ is the error term (white noise) at time t
• The order can be determined by examining the partial autocorrelation function (PACF) plot
  • PACF plot shows the correlation between an observation and its lagged values, controlling for intermediate lags
  • Significant lags in the PACF plot indicate the order of the AR model (web traffic, sales data)

Parameter Estimation and Model Assessment

Parameter estimation for AR models

• Parameters of an AR model (autoregressive coefficients, constant term) can be estimated using statistical methods
• Least squares estimation:
  • Ordinary least squares (OLS) minimizes the sum of squared residuals between observed and predicted values
  • OLS estimates of AR coefficients are obtained by solving a system of linear equations
• Maximum likelihood estimation (MLE):
  • Finds parameter values that maximize the likelihood function, measuring the probability of observing the data under the assumed model
  • MLE estimates are obtained by numerically optimizing the likelihood function
• Choice between least squares and MLE depends on assumptions about the error term and desired properties of estimators
  • OLS is computationally simpler and provides unbiased estimates when errors are uncorrelated with constant variance
  • MLE is more flexible and accommodates complex error structures but may be computationally intensive (exchange rates, economic indicators)

Stationarity assessment of AR models

• Stationarity is crucial for AR models, ensuring the model's properties remain constant over time
• Stationarity can be assessed by examining the values of the autoregressive coefficients
• For an AR(1) model: $y_t = c + \phi_1 y_{t-1} + \varepsilon_t$
  • The model is stationary if $|\phi_1| < 1$
  • If $\phi_1 = 1$, the model is non-stationary and is a random walk
  • If $|\phi_1| > 1$, the model is explosive and not suitable for forecasting
• For an AR(p) model with p > 1, stationarity involves the roots of the characteristic equation
  • Characteristic equation: $1 - \phi_1 z - \phi_2 z^2 - \ldots - \phi_p z^p = 0$
  • AR(p) model is stationary if all roots lie outside the unit circle in the complex plane
• If an AR model is non-stationary, it may need to be transformed (differencing) to achieve stationarity before further analysis (temperature data, population growth)

Application and Interpretation

Application of AR models

• AR models can be applied to various real-world time series data for forecasting and understanding dynamics
• Steps to apply AR models:
  1. Identify the order of the AR model using PACF plot or model selection criteria (AIC, BIC)
  2. Estimate the parameters of the AR model using least squares or MLE
  3. Assess the model's goodness of fit and diagnostic checks (residual analysis, normality tests)
  4. Use the fitted AR model to make predictions for future time points
• Interpreting the results:
  • Estimated autoregressive coefficients indicate the strength and direction of the relationship between current and lagged observations
    • Positive coefficients suggest positive correlation, negative coefficients suggest negative correlation
  • The magnitude of coefficients determines the persistence of the effect of past observations on the current observation
  • The constant term represents the mean level of the time series when all lagged terms are zero
• Real-world applications:
  • Forecasting stock prices, exchange rates, economic indicators
  • Modeling and predicting energy consumption or demand
  • Analyzing and forecasting sales data or web traffic
• Consider the limitations of AR models when interpreting results, such as the assumption of linearity and potential need for additional factors or exogenous variables to capture full dynamics