Forecasting Study Guide

📖 Core Concepts Forecasting – Predicting future values using past & present data; distinct from generic prediction which may lack formal methods. Risk & Uncertainty – Every forecast should state its associated uncertainty (e.g., confidence intervals). Qualitative vs. Quantitative – Qualitative: expert judgment, no historical numbers. Quantitative: statistical models built from past numeric data. Seasonality – Regular, calendar‑driven pattern (e.g., higher sales every December); expressed with a seasonal index > 1 (above average) or < 1 (below average). Cyclic Behaviour – Irregular fluctuations lasting ≥ 2 years with no fixed period; unlike the fixed‑length seasonality. Forecast Accuracy – Measured by error statistics (MAE, MSE, RMSE, MAPE, MASE); good forecasts have zero‑mean, uncorrelated residuals. 📌 Must Remember Forecast error: $Et = Yt - Ft$ (actual – forecast). MAE: $\displaystyle \frac{1}{n}\sum |Et|$ – average absolute error, same scale as data. MSE / RMSE: $\displaystyle \frac{1}{n}\sum Et^2$ ; RMSE = $\sqrt{\text{MSE}}$. MAPE: $\displaystyle \frac{100\%}{n}\sum \left|\frac{Et}{Yt}\right|$ – unstable when $Yt \approx 0$. MASE: $\displaystyle \frac{\text{MAE}}{\text{MAE}{\text{naïve seasonal}}}$ – allows cross‑series comparison. Naïve forecast: $F{t+1}=Yt$ – baseline benchmark. Drift forecast: $F{t+1}=Yt + \frac{Yt-Y1}{t-1}$ – adds average change (slope). Seasonal naïve: $F{t}=Y{t-s}$ where $s$ = seasonal period (e.g., 12 months). Training vs. Test: Fit model on training set; evaluate on unseen test set. Rolling‑origin CV: Expand window forward in time, re‑forecast each step. 🔄 Key Processes Select Forecasting Method Data available? → Quantitative. No historical numbers? → Qualitative. Split Data Reserve last $k$ observations as test set or use rolling‑origin CV. Fit Model on Training Set Estimate parameters (e.g., smoothing constants, ARIMA coefficients). Generate Forecasts Apply chosen method (naïve, drift, seasonal naïve, exponential smoothing, etc.). Compute Accuracy Metrics MAE, RMSE, MAPE, MASE → compare against naïve benchmark. Validate & Iterate If residuals show autocorrelation → adjust model (add AR terms, incorporate exogenous variables). 🔍 Key Comparisons Qualitative vs. Quantitative Qualitative: expert opinion, no numeric history → used for new products or rare events. Quantitative: statistical models, past numbers → higher objectivity, requires data. Naïve vs. Drift vs. Seasonal Naïve Naïve: $F{t+1}=Yt$ – best for random walk without trend/seasonality. Drift: adds average change → good when a linear trend exists. Seasonal Naïve: $F{t}=Y{t-s}$ – optimal when strong seasonal pattern but no trend. Scale‑Dependent vs. Percentage vs. Scaled Errors Scale‑Dependent (MAE, RMSE): same units as data – useful for absolute error judgment. Percentage (MAPE): unit‑free, easy to interpret, fails near zero. Scaled (MASE): compares to naïve benchmark, works across series of different magnitudes. ⚠️ Common Misunderstandings “Lower MAPE is always better.” → MAPE can be misleading when actual values are tiny; prefer RMSE or MASE in those cases. “A perfect fit on training data guarantees a good forecast.” → Over‑fitting; always test on unseen data. “Seasonality = cyclic behaviour.” → Seasonality has a fixed calendar period; cycles are irregular and longer than two years. “Forecasts are predictions of exact numbers.” – Forecasts are estimates with associated uncertainty; treat them as ranges. 🧠 Mental Models / Intuition Benchmark‑First Mindset: Always compute a naïve (or seasonal naïve) forecast first; any sophisticated model must beat this baseline. Error‑Residual Check: If residuals look like a pattern (trend or seasonality), the model missed something – add the corresponding component. Scale‑Awareness: Think of MAE as “average dollars off,” RMSE as “average squared dollars off,” MAPE as “average percent off.” Choose the one that matches the decision context. 🚩 Exceptions & Edge Cases Zero or Near‑Zero Actuals: MAPE becomes infinite or erratic → switch to MAE or SMAPE (symmetrized version). Highly Seasonal but No Trend: Drift method will over‑forecast; use seasonal naïve or seasonal decomposition. Data with Structural Breaks: Models assuming stationarity (e.g., ARIMA) may fail; consider regime‑switching or judgmental adjustments. 📍 When to Use Which | Situation | Recommended Method | |-----------|--------------------| | No historical numbers, expert insight available | Qualitative / Judgmental methods | | Strong, stable seasonal pattern, little trend | Seasonal Naïve or Seasonal Decomposition | | Linear upward/downward trend, no seasonality | Drift method or Linear Extrapolation | | Short‑run forecast (1‑2 periods) | Naïve or Simple Exponential Smoothing | | Multiple exogenous drivers (e.g., holidays, weather) | Relational/ Causal models (Regression, ARMAX) | | Large data set, non‑linear patterns | AI methods (Neural Networks, SVM) | | Need to compare across products of different scales | MASE (scaled error) | | Model selection phase | Use rolling‑origin CV to assess out‑of‑sample performance | 👀 Patterns to Recognize Residual autocorrelation → missing lagged terms or seasonality. Increasing forecast error over horizon → model may lack trend component. MASE ≈ 1 → model no better than seasonal naïve; revisit. Sudden spikes in error coinciding with external events → consider adding exogenous variables (relational methods). 🗂️ Exam Traps Choosing MAPE for data with zeros – will be flagged as “incorrect because MAPE unstable.” Assuming a low RMSE automatically means a good model – ignore bias; residuals must be zero‑mean. Confusing seasonality with cyclic behaviour – exam will test definition; remember fixed calendar vs. irregular multi‑year cycles. Selecting a quantitative method when only expert opinion is available – will lose points; the right answer is a qualitative/judgmental approach. Omitting the benchmark (naïve) comparison – many questions ask whether a method “improves upon naïve”; forgetting this leads to incomplete answer.