3.3 Prediction and Confidence Intervals for Response Variable

Confidence vs Prediction Intervals

In simple linear regression, once you have a fitted line, you can use it to estimate values of the response variable at a given $x_0$. But a single point estimate ($\hat{y}$) doesn't tell you how uncertain that estimate is. That's where confidence intervals and prediction intervals come in. They both give you a range of plausible values, but they answer different questions.

A confidence interval for the mean response answers: What is the plausible range for the average $y$ across the entire population at this $x_0$?

A prediction interval answers: What is the plausible range for a single new observation's $y$ at this $x_0$?

This distinction matters a lot. Predicting where one individual data point will land involves more uncertainty than estimating where the population mean sits, because individual observations scatter around the mean. That extra source of variability is why prediction intervals are always wider than confidence intervals at the same confidence level and the same $x_0$.

Factors Influencing Interval Width

Both types of intervals get wider or narrower depending on:

• Sample size — Larger $n$ gives you more information about the regression line, so both intervals shrink.
• Data variability — Higher $MSE$ (more scatter around the fitted line) widens both intervals.
• Confidence level — A 99% interval is wider than a 95% interval because you need a larger range to be "more sure."
• Distance of $x_0$ from $\bar{x}$ — The farther $x_0$ is from the center of your data, the wider both intervals become. This is reflected in the $(x_0 - \bar{x})^2 / S_{xx}$ term in the formulas below.

Prediction Intervals for Individual Observations

Standard Error of Prediction

The standard error of prediction captures two sources of uncertainty: uncertainty in the estimated regression line itself, plus the natural variability of individual observations around that line. The formula is:

$se(\text{pred}) = \sqrt{MSE \left(1 + \frac{1}{n} + \frac{(x_0 - \bar{x})^2}{S_{xx}}\right)}$

• $MSE$ — mean squared error (estimate of $\sigma^2$, the variance of the errors)
• $n$ — sample size
• $x_0$ — the specific value of the predictor where you're making the prediction
• $\bar{x}$ — mean of the predictor variable in your sample
• $S_{xx}$ — sum of squared deviations of the predictor, $\sum(x_i - \bar{x})^2$

Notice the 1 inside the parentheses. That term accounts for the variability of an individual observation around the true regression line. It's what makes prediction intervals wider than confidence intervals.

Calculating the Prediction Interval

Once you have $se(\text{pred})$, the prediction interval is:

$\hat{y} \pm t_{\alpha/2,\, n-2} \cdot se(\text{pred})$

1. Compute $\hat{y}$ by plugging $x_0$ into your fitted regression equation.

2. Calculate $se(\text{pred})$ using the formula above.

3. Find the critical value $t_{\alpha/2,\, n-2}$ from the t-distribution with $n - 2$ degrees of freedom. For a 95% interval, $\alpha/2 = 0.025$.

4. Multiply the critical value by $se(\text{pred})$ and add/subtract from $\hat{y}$.

Confidence Intervals for Mean Response

Standard Error of the Mean Response

The standard error of the mean response (sometimes called the standard error of $\hat{y}$) captures only the uncertainty in estimating the regression line:

$se(\hat{y}) = \sqrt{MSE \left(\frac{1}{n} + \frac{(x_0 - \bar{x})^2}{S_{xx}}\right)}$

Compare this to the prediction standard error. The only difference is the absence of the 1 inside the parentheses. Without that term, $se(\hat{y}) < se(\text{pred})$ always, which is why confidence intervals for the mean are narrower.

Calculating the Confidence Interval

$\hat{y} \pm t_{\alpha/2,\, n-2} \cdot se(\hat{y})$

The steps mirror the prediction interval calculation:

1. Compute $\hat{y}$ at $x_0$.
2. Calculate $se(\hat{y})$ using the formula above.
3. Look up $t_{\alpha/2,\, n-2}$.
4. Form the interval: $\hat{y} \pm t_{\alpha/2,\, n-2} \cdot se(\hat{y})$.

Interpreting Intervals in Context

Getting the numbers right is only half the job. You also need to state what the interval means in the context of the problem, and the wording differs for the two interval types.

Interpreting a Prediction Interval

A prediction interval gives a range for a single future observation. Correct interpretation:

The interval covers where one individual's response might fall, including all the person-to-person variability.

Interpreting a Confidence Interval for the Mean Response

A confidence interval for the mean response gives a range for the population average at that $x_0$. Correct interpretation:

Notice this interval is narrower. You're estimating a population parameter (the mean), not predicting a single outcome.

A Common Interpretation Mistake

Be careful with the phrase "95% probability." Strictly speaking, the true mean (or the future observation) is a fixed quantity; it either falls in the interval or it doesn't. The 95% refers to the procedure: if you repeated the sampling and interval construction many times, about 95% of those intervals would contain the true value. In practice, saying "we are 95% confident" is the accepted shorthand, but avoid saying "there is a 95% probability that the true mean is in this interval" on an exam, as many instructors will mark that as incorrect.

Why Intervals Widen Away from $\bar{x}$

Both interval types are narrowest when $x_0 = \bar{x}$ and grow wider as $x_0$ moves away from the center of the data. This happens because the $(x_0 - \bar{x})^2 / S_{xx}$ term increases. Visually, if you plot confidence or prediction bands around the regression line, they form a "bowtie" or hyperbolic shape, tightest at the mean of $x$ and flaring out at the edges. This is a good reminder that extrapolating far beyond your data produces very wide (and unreliable) intervals.