Compare the damage done by omitting an explanatory variable that matters with the effects of keeping in a regressor that does not, and illustrate both.
The idea
Leaving out a relevant variable and keeping an irrelevant one do different damage. Suppose a variable that truly affects y is omitted and is correlated with an included regressor. The included coefficient then absorbs part of its effect. Expected slope = β1 + β2 × δ. Here β1 is the included variable's own coefficient, β2 is the omitted variable's true coefficient, and δ is the slope from regressing the omitted variable on the included one. The estimate is biased and inconsistent, and more data do not cure it. If the omitted variable is uncorrelated with the included ones, the slopes stay unbiased but the error variance is larger. Including an irrelevant variable leaves estimates unbiased and consistent but raises their variance, more so when it is correlated with the other regressors. Omission costs bias, and inclusion costs precision. Bias is usually the worse problem.
Formulas
Expected slope when a relevant variable is omitted
E(b̂₁) = β₁ + β₂ × δ
- β₁
- the true coefficient on the included regressor
- β₂
- the true coefficient on the omitted variable
- δ
- the slope from regressing the omitted variable on the included one
Bias appears only if δ is non-zero; its sign is the sign of β₂ × δ. δ can be found as correlation × σ of the omitted variable ÷ σ of the included one.
A practice question
What is the effect on OLS of omitting a relevant variable that is correlated with an included regressor?
- The included coefficient is unbiased but has a larger variance.Why not: That is the effect of including an irrelevant variable, not of omitting a relevant one.
- The included regressor's coefficient is biased and inconsistent. - correct
- The model cannot be estimated because the regressors are collinear.Why not: Omission leaves fewer regressors and does not create linear dependence; estimation is straightforward.
- The coefficient is unbiased, and only the R² declines.Why not: The correlation with the omitted variable moves part of its effect into the included coefficient, so bias arises.
The included variable picks up part of the omitted variable's effect through their correlation, so its coefficient is biased and the bias does not vanish in large samples.
A harder one
The true model is y = a + 0.5 x1 + 0.8 x2 + e. The analyst omits x2. The standard deviations of x1 and x2 are 2 and 3, and their correlation is 0.5. What is the expected value of the slope on x1 in the misspecified regression?
- 0.50Why not: Assumes the omission is harmless; the slope is biased because x1 and x2 are correlated.
- 0.90Why not: Uses the correlation as delta; delta is the slope of x2 on x1, which is correlation x sd(x2) / sd(x1).
- 0.60Why not: Reports only the bias term; the expected slope is the true coefficient plus the bias.
- 1.10 - correct
delta = corr x sd(x2)/sd(x1) = 0.5 x 3/2 = 0.75. Expected slope = beta1 + beta2 x delta = 0.5 + 0.8 x 0.75 = 1.10.
The hardest tier
The true coefficient on leverage in a credit-spread model is 1.2. An omitted variable has a true coefficient of -0.4 in the model, and a regression of the omitted variable on leverage has a slope of +0.5. What is the expected leverage coefficient when the variable is left out?
- 1.4, an upward bias.Why not: Gets the sign of the bias wrong: the bias is beta2 x delta, which is negative here.
- 1.2, no bias.Why not: Ignores that the omitted variable is correlated with leverage, so the estimate absorbs part of its effect.
- 0.8, a downward bias.Why not: Adds the omitted coefficient in full and ignores delta, which scales how much of its effect the included variable absorbs.
- 1.0, a downward bias. - correct
Bias = beta2 x delta = -0.4 x 0.5 = -0.2. The expected estimate is 1.2 + (-0.2) = 1.0, below the true 1.2: the negative effect of the omitted variable, positively related to leverage, drags the estimate down.
In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.
Start free →More in Regression Diagnostics
- Describe how to check whether heteroskedasticity affects a regression.
- Outline the ways of working with heteroskedastic data.
- Describe multicollinearity and set it apart from perfect collinearity.
- Explain two procedures for selecting a model and how each connects with the bias-variance trade-off.
- Outline the different ways to visualise residuals and the strengths of each relative to the others.
- Apply methods that identify outliers and describe the effect those outliers have.
- Identify when OLS qualifies as the best linear unbiased estimator by stating the required conditions.