Charta is on Google Play: practise on the go. Get the app
Charta

Summarise what OLS estimators are like, covering their sampling distributions as well.

FRM® Part I · Quantitative Analysis · Linear Regression · FRM1.QA.7.d

The idea

Under the OLS assumptions the estimators have attractive properties. They are unbiased, so the average of the slope estimates over repeated samples equals the true slope. They are also consistent, so they converge to the truth as the sample grows. When errors are also homoskedastic, the Gauss-Markov theorem says OLS is the best linear unbiased estimator, meaning it has the smallest variance among linear unbiased estimators. By the central limit theorem the sampling distribution of the slope is approximately normal in large samples, whatever the error distribution. Its standard error is roughly this, where σu is the error standard deviation and σX that of X: SE = σu ÷ (√n × σX). It falls as n grows, falls when X is more spread out, and rises when the errors are noisier. Do not confuse unbiasedness, which needs no normality, with exact t distributions, which do.

Formulas

Approximate standard error of an OLS slope

SE(b) = σε ÷ (√n × σX)

σε
the standard deviation of the regression error
n
the number of observations
σX
the standard deviation of the explanatory variable

An approximation for large samples with one regressor. A wider spread in X helps as much as more data; observations bunched at the mean of X add little.

A practice question

From the question bank, with the reason each wrong option is wrong.

An OLS slope estimator is described as unbiased. This means that:

  1. the slope estimate from any single sample equals the true slope, with no sampling errorWhy not: Unbiasedness is about the average across samples; any single estimate can differ from the truth.
  2. the slope estimate converges to the true slope as the sample size grows without boundWhy not: This describes consistency, a large-sample property distinct from unbiasedness.
  3. the slope estimates average out to the true slope over repeated samples - correct
  4. the slope estimate has the smallest variance of all possible estimatorsWhy not: This describes efficiency, which is a separate property and also requires further conditions.

Unbiased means E(slope estimate) = true slope, with the expectation taken over the sampling distribution. Consistency and efficiency are different properties.

A harder one

Study A has n = 100, an error standard deviation of 3 and a standard deviation of the explanatory variable of 2. Study B has n = 225, an error standard deviation of 4 and an explanatory-variable standard deviation of 1. Using SE of slope = (error SD) / (sqrt(n) x SD of X), the ratio of Study B's standard error to Study A's is closest to:

  1. 1.78 - correct
  2. 0.56Why not: Inverts the ratio; B divided by A is above 1.
  3. 1.19Why not: Divides by n instead of the square root of n when computing the standard errors.
  4. 0.89Why not: Ignores the standard deviation of X; the spread of the explanatory variable matters.

SE(A) = 3 / (10 x 2) = 0.150; SE(B) = 4 / (15 x 1) = 0.267; ratio = 1.78. Larger n lowers the standard error, but noisier errors and a smaller spread in X raise it.

The hardest tier

A regression with one explanatory variable has residuals whose variance increases with X (heteroskedasticity). The other OLS assumptions hold. Which statement about the OLS slope estimator is correct?

  1. It becomes biased, though it remains the minimum-variance linear estimatorWhy not: Heteroskedasticity does not create bias, and minimum variance is the property that is lost.
  2. It stays unbiased and consistent, but is no longer the best linear unbiased estimator - correct
  3. It remains the best linear unbiased estimator, and only the intercept is affectedWhy not: The Gauss-Markov result needs homoskedasticity; the slope loses efficiency when it fails.
  4. It becomes inconsistent, because the variance of the sampling distribution does not shrinkWhy not: The variance still shrinks as n grows; consistency is retained under heteroskedasticity.

Unbiasedness and consistency rely on zero conditional mean, not constant variance. Gauss-Markov efficiency needs homoskedasticity, and the usual standard errors are also wrong.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Linear Regression