Statistical inference - Predictive Inference

Predictive Inference Introduction Predictive inference is a fundamental approach in statistics focused on one primary goal: using existing data to make accurate forecasts about future observations we haven't yet seen. This contrasts with other statistical approaches that might focus on estimating fixed parameters or understanding relationships in your current data. When you care about "what comes next," predictive inference is your tool. The motivation is straightforward—in many real-world problems, we want to know what will happen, not just understand what has happened. Will it rain tomorrow? What will next year's sales look like? How old will a randomly selected person be? These are predictive questions, and predictive inference gives us a principled way to answer them. Forecasting Future Observations from Past Data The core principle of predictive inference is simple: past data contains information about future observations. By analyzing patterns in historical data, we can characterize the likely range and distribution of new values. Here's how it works conceptually: Gather past observations that are relevant to what you want to predict Learn what those observations tell us about the underlying data-generating process Use that learning to forecast what future observations will look like For example, if you have historical data on a person's age from a survey, that data helps you predict the age of the next person who takes the survey. The past observations suggest the likely range and pattern of future ages. The key insight is that we treat future observations as random variables—we don't expect to predict them exactly, but we can characterize their probable values. Posterior Predictive Distributions In modern predictive inference, we rely on a powerful tool called the posterior predictive distribution. This distribution represents all our beliefs about what a future observation will be, combining both what we learned from past data and our uncertainty. What is a Posterior Predictive Distribution? A posterior predictive distribution is a probability distribution over possible future observations. It answers the question: "Given the data I've observed, what are the likely values for a new observation, and how likely is each value?" To build this distribution, we: Start with past data and learn what it tells us about the underlying parameters of the system Combine that learning with the random variability we expect in future observations Produce a distribution that reflects both our learned knowledge and remaining uncertainty This is a distinctly Bayesian approach: we use the posterior distribution (our updated beliefs about parameters, given data) to predict the distribution of future data. Why It Matters The posterior predictive distribution is powerful because it automatically incorporates uncertainty from two sources: Parameter uncertainty: We didn't learn the true parameters exactly; the data gave us a range of possibilities Observation variability: Even if we knew the parameters perfectly, real observations have natural randomness Both sources of uncertainty appear in the posterior predictive distribution, giving us an honest assessment of prediction uncertainty. The histogram above illustrates what a posterior predictive distribution might look like—a distribution showing the probable values of future observations (in this case, ages), with peak probability in the middle and diminishing probability toward the extremes. Prediction Intervals A practical way to communicate predictions is through prediction intervals—ranges of values where we expect future observations to fall with a specified level of confidence. What is a Prediction Interval? A prediction interval is an interval, say [a, b], where we claim a future observation will fall with a certain probability (commonly 95%). For example: "I predict the next person's age will fall between 18 and 72 years old, with 95% confidence." How to Generate Prediction Intervals Prediction intervals come directly from the posterior predictive distribution: For a 95% prediction interval, we find the values that contain the middle 95% of the posterior predictive distribution, leaving 2.5% probability in each tail For other confidence levels (like 90%), we adjust accordingly to capture the appropriate center portion of the distribution Prediction Intervals vs. Confidence Intervals A common source of confusion: prediction intervals are wider than confidence intervals for the same confidence level. This is because: A confidence interval for a parameter reflects uncertainty about a fixed (but unknown) value A prediction interval reflects uncertainty about a random future observation, which includes both parameter uncertainty and the natural variability of individual observations Prediction intervals appropriately reflect that predicting a single observation is harder than pinpointing a population parameter.