⏳Intro to Time Series Unit 3 – Stationary vs Non-Stationary Time Series

What's the Deal with Time Series?

• Time series data consists of observations collected sequentially over time at regular intervals (hourly, daily, monthly)
• Analyzing time series data helps uncover patterns, trends, and seasonality to make predictions and inform decision-making
  • Sub-bullet: Time series forecasting uses historical data to predict future values (stock prices, weather patterns)
• Time series differ from other data types as observations are dependent on past values and often exhibit autocorrelation
• Components of a time series include trend, seasonality, cyclical patterns, and irregular fluctuations
• Time series analysis techniques range from simple moving averages to complex models like ARIMA and LSTM neural networks
• Stationarity is a crucial property in time series analysis that affects the choice of modeling techniques and interpretation of results
• Non-stationary time series require special handling, such as differencing or detrending, before applying certain analysis methods

Stationary vs Non-Stationary: The Basics

• Stationarity refers to the statistical properties of a time series remaining constant over time
  • Sub-bullet: In a stationary series, the mean, variance, and autocorrelation structure do not change with time
• Non-stationary time series have statistical properties that vary over time, often exhibiting trends or changing variance
• Stationary time series are easier to model and forecast as their behavior is more predictable and consistent
• Non-stationary time series can lead to spurious correlations and unreliable predictions if not properly addressed
• Stationarity is a requirement for many time series analysis techniques, such as ARMA and ARIMA models
• Differencing is a common method to transform a non-stationary time series into a stationary one by taking the difference between consecutive observations
• Trend-stationary series can be made stationary by removing the deterministic trend, while difference-stationary series require differencing

Spotting the Difference: Key Features

• Visual inspection of the time series plot can provide initial clues about stationarity
  • Sub-bullet: Stationary series typically fluctuate around a constant mean, while non-stationary series may show trends or changing variance
• Stationary time series have constant mean, variance, and autocorrelation over time
• Non-stationary series may exhibit trends (increasing or decreasing mean over time) or seasonality (regular patterns)
• Changing variance, or heteroscedasticity, is another indicator of non-stationarity (volatility clustering in financial data)
• Autocorrelation plots (ACF) can help identify non-stationarity
  • Sub-bullet: Slowly decaying ACF suggests non-stationarity, while quickly decaying ACF indicates stationarity
• Unit root tests, such as Dickey-Fuller or KPSS, provide statistical evidence for the presence of non-stationarity
• Residual plots from fitted models can reveal non-stationarity if patterns or trends are present in the residuals

Why It Matters: Real-World Applications

• Stationarity assumptions are crucial for the validity and reliability of time series forecasting models
• Non-stationary data can lead to spurious regressions, where unrelated variables appear to be significantly correlated
• Forecasting with non-stationary data may result in unreliable predictions and poor decision-making
• Stationary time series are essential for risk management and portfolio optimization in finance (stock returns, volatility modeling)
• Monitoring the stationarity of process variables is critical in quality control and fault detection (manufacturing, sensor data)
• Econometric models, such as vector autoregression (VAR), require stationary inputs for valid inference and policy analysis
• Stationarity is a key assumption in signal processing applications, such as speech recognition and EEG analysis

Testing for Stationarity: Tools and Techniques

• Visual inspection of time series plots and summary statistics can provide initial insights into stationarity
• Augmented Dickey-Fuller (ADF) test is a widely used unit root test for stationarity
  • Sub-bullet: ADF tests the null hypothesis of a unit root (non-stationarity) against the alternative of stationarity
• Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test is another popular stationarity test with the null hypothesis of stationarity
• Phillips-Perron (PP) test is a non-parametric alternative to the ADF test, robust to serial correlation and heteroscedasticity
• Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots can reveal non-stationarity through slow decay
• Seasonal decomposition and subseries plots can help identify seasonality and trend components
• Residual diagnostics from fitted models, such as the Ljung-Box test, can detect remaining non-stationarity

Transforming Non-Stationary to Stationary

• Differencing is a common technique to remove trends and make a series stationary
  • Sub-bullet: First-order differencing calculates the change between consecutive observations: $\Delta y_t = y_t - y_{t-1}$
• Seasonal differencing can remove seasonal non-stationarity by taking differences at the seasonal lag
• Logarithmic or power transformations can stabilize variance and make a series more stationary (Box-Cox transformation)
• Detrending methods, such as linear regression or moving averages, can remove deterministic trends
• Seasonal adjustment techniques, like X-11 or SEATS, can remove seasonal components and yield a stationary series
• Hodrick-Prescott (HP) filter is a popular method for separating trend and cyclical components in macroeconomic data
• Fourier transforms can identify and remove periodic components, making the series more stationary

Common Pitfalls and How to Avoid Them

• Overdifferencing can introduce unnecessary noise and complicate model interpretation
  • Sub-bullet: Use stationarity tests and ACF/PACF plots to determine the appropriate order of differencing
• Failing to account for seasonality can lead to residual non-stationarity and poor model performance
• Applying tests and transformations without understanding the underlying assumptions and limitations
• Relying solely on a single test or method to assess stationarity; use multiple approaches for robustness
• Ignoring the presence of structural breaks or regime shifts, which can affect stationarity (Chow test, CUSUM test)
• Misinterpreting stationarity tests due to small sample sizes or low power; use appropriate critical values and sample sizes
• Neglecting to check the stationarity of residuals from fitted models; non-stationary residuals indicate model misspecification

Putting It All Together: Practice Problems

• Identify the stationarity of given time series datasets using visual inspection, summary statistics, and formal tests
• Apply appropriate transformations to convert non-stationary series to stationary ones (differencing, detrending, seasonal adjustment)
• Assess the impact of stationarity assumptions on the performance of forecasting models (ARMA, ARIMA, exponential smoothing)
• Analyze real-world case studies involving non-stationary data (stock prices, economic indicators, sensor data) and propose suitable modeling approaches
• Conduct residual diagnostics to ensure the stationarity of model residuals and identify potential misspecifications
• Compare the performance of different stationarity tests (ADF, KPSS, PP) and discuss their strengths and limitations
• Develop a decision tree or flowchart for selecting appropriate stationarity tests and transformations based on data characteristics