Charta is on Google Play: practise on the go. Get the app
Charta

Apply methods that identify outliers and describe the effect those outliers have.

FRM® Part I · Quantitative Analysis · Regression Diagnostics · FRM1.QA.9.g

The idea

An outlier is an observation that sits far from the pattern of the rest of the data, either in y (a large residual) or in the regressors (high leverage). Leverage hi is the diagonal element of the hat matrix. It lies between 1/n and 1, and the leverages sum to k, the number of estimated parameters including the intercept, so the average is k ÷ n. A studentised residual puts residuals on a common scale. Studentised residual = ei ÷ (s × √(1 − hi)). Cook's distance combines size and leverage. Di = ei² ÷ (k × s²) × hi ÷ (1 − hi)². A value above about 1 marks a point whose removal would change the fit materially. Influential outliers pull the fitted line, distort coefficients and inflate the residual variance. A high-leverage point with a small residual can be the most misleading. Investigate before deleting, and remove only data errors.

Formulas

Cook's distance for observation i

D = [e² ÷ (k × s²)] × [h ÷ (1 − h)²]

e
the observation's residual
k
the number of estimated parameters, intercept included
s²
the estimated error variance
h
the observation's leverage

Above about 1 flags an influential point. Both a large residual and high leverage push it up; neither alone is enough.

Residual standardised for leverage

studentised residual = e ÷ (s × √(1 − h))

e
the observation's residual
s
the standard error of the regression
h
the observation's leverage

Puts residuals on a common scale; an absolute value above 3 is a common outlier flag. Ignoring leverage understates high-leverage residuals.

A practice question

From the question bank, with the reason each wrong option is wrong.

What does Cook's distance measure for an observation in a regression?

  1. The distance of its regressor value from the mean of that regressor, regardless of the residual.Why not: This describes leverage, which is only one component of Cook's distance.
  2. The size of its residual divided by the regression standard error, regardless of leverage.Why not: This describes a standardised residual, which ignores how unusual the regressor values are.
  3. The correlation between that observation's residual and the residual of the previous observation.Why not: This describes serial correlation of residuals; Cook's distance has nothing to do with the order of observations.
  4. How much the fitted values change when that observation is deleted. - correct

Cook's distance combines a large residual with high leverage to measure influence: a value above roughly 1 flags an observation whose deletion would change the fitted regression substantially.

A harder one

A regression with an intercept and 3 slope coefficients is estimated on 80 observations. What is the average leverage (hat value) across the observations?

  1. 0.0125Why not: Uses 1/n, the minimum leverage in a model with only an intercept, ignoring the number of estimated parameters.
  2. 0.0500 - correct
  3. 0.0375Why not: Counts only the 3 slope coefficients; the sum of the leverages equals the number of parameters including the intercept.
  4. 0.2000Why not: Squares the number of parameters; the leverages sum to k, so their average is k / n.

The leverages sum to the number of estimated parameters k = 4 (intercept plus 3 slopes), so the average is k / n = 4 / 80 = 0.0500. An observation with leverage well above this average has an unusual regressor combination.

The hardest tier

Four observations from a one-regressor model are examined. A: x far from the mean, point lies close to the line fitted to the others. B: x near the mean, y far from the line. C: x far from the mean, y far from the line fitted to the others. D: x near the mean, y near the line. Which observation has the largest Cook's distance, and why?

  1. A, because high leverage alone is enough to make a point the most influential.Why not: A high-leverage point that lies on the pattern of the other data has little effect on the fitted line; influence needs a large residual as well.
  2. B, because it has the largest residual and so the largest effect on the slope.Why not: A point near the mean of x moves mainly the intercept; it has low leverage and little pull on the slope.
  3. C, because it combines high leverage with a large residual and so most changes the fitted slope. - correct
  4. D, because it lies close to the centre of the data and therefore anchors the line.Why not: A point near the mean of x with a small residual has low leverage and a small residual, so its Cook's distance is smallest.

Cook's distance rises with both the residual and the leverage. C is far from the mean of x and far from the line, so deleting it would substantially change the slope; A is consistent with the trend, B has low leverage, and D is unremarkable on both counts.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Regression Diagnostics