Charta is on Google Play: practise on the go. Get the app
Charta

Explain how confidence intervals and hypothesis tests are connected.

FRM® Part I · Quantitative Analysis · Hypothesis Testing · FRM1.QA.6.d

The idea

A confidence interval and a hypothesis test use the same ingredients: the estimate, its standard error and a critical value. A (1 − α) two-sided confidence interval is the estimate ± critical value × standard error. The critical value is taken at α ÷ 2 in each tail. A two-sided test of H0: μ = μ0 at significance level α rejects exactly when μ0 lies outside that interval, and fails to reject when μ0 lies inside. So a 95 percent interval matches a 5 percent test and a 99 percent interval matches a 1 percent test. The interval has the advantage of showing every value that would be rejected. The usual error is to read a 95 percent interval as a 95 percent probability that the fixed true value is inside this one interval.

Formulas

Two-sided confidence interval for a parameter

Confidence interval = estimate ± c × SE

estimate
the sample estimate, such as the sample mean
c
the critical value at α ÷ 2 in each tail, 1.96 for 95 percent
SE
the standard error of the estimate

Two-sided interval at confidence 1 − α. It rejects H0 at level α exactly when the hypothesised value lies outside it; the trap is reading 95 percent as the chance that this one interval holds the true value.

A practice question

From the question bank, with the reason each wrong option is wrong.

A 95 percent two-sided confidence interval for a strategy's mean return runs from 0.4 percent to 1.8 percent. For a two-sided test of H0: mu = 0 at the 5 percent significance level, which conclusion is appropriate?

  1. Reject H0, because zero lies outside the interval. - correct
  2. Fail to reject H0, because the interval contains positive values.Why not: Looks at the sign of the values rather than whether the hypothesized value mu0 = 0 lies inside the interval.
  3. Fail to reject H0, because a 95 percent interval corresponds to a 95 percent significance level.Why not: Confuses confidence level with significance; a 95 percent interval matches a 5 percent significance level, and zero is outside it.
  4. No conclusion is possible, because an interval cannot be used to test a hypothesis.Why not: Ignores the duality between two-sided intervals and two-sided tests.

A two-sided test at significance alpha rejects exactly when the hypothesized value lies outside the (1 - alpha) interval. Zero is outside (0.4, 1.8), so H0 is rejected at 5 percent.

A harder one

A sample mean excess return is 0.9 percent with a standard error of 0.4 percent. Which statement about the 95 percent confidence interval and the two-sided 5 percent test of H0: mu = 0 is correct?

  1. The interval is about -0.67 to 2.47 percent; zero is inside, so H0 is not rejected.Why not: Doubles the margin (1.96 x 0.4 x 2) as if the two-sided adjustment doubled the width.
  2. The interval is about 0.12 to 1.68 percent; zero is outside, so H0 is rejected. - correct
  3. The interval is about -1.06 to 2.86 percent; zero is inside, so H0 is not rejected.Why not: Uses 1.96 as the whole margin and forgets to multiply by the standard error of 0.4.
  4. The interval is about 0.24 to 1.56 percent; zero is outside, so H0 is rejected at 5 percent.Why not: Uses 1.645, the 90 percent two-sided value, so the interval is too narrow for 95 percent confidence.

Margin = 1.96 x 0.4 = 0.784, so the interval is 0.9 +/- 0.784 = (0.116, 1.684). Zero is not inside, so a two-sided 5 percent test rejects.

The hardest tier

A 95 percent two-sided confidence interval for a fund's mean excess return is -0.2 percent to 2.6 percent, with point estimate 1.2 percent. Which statement is correct?

  1. A two-sided 5 percent test would not reject any hypothesized mean between -0.2 and 2.6 percent, including zero. - correct
  2. Zero is rejected at the 5 percent level because the point estimate of 1.2 percent is positive.Why not: Judges by the sign of the estimate; the test depends on whether the hypothesized value lies inside the interval.
  3. In repeated sampling, 95 percent of sample means would fall inside this particular interval.Why not: Misreads the interval; the 95 percent refers to the share of intervals constructed this way that cover the true mean.
  4. The two-sided p-value for H0: mu = 0 is below 5 percent because the interval is wide.Why not: A wide interval that contains zero means the p-value is above 5 percent, not below.

Any value inside a (1 - alpha) interval is not rejected by the matching two-sided test. Zero is inside (-0.2, 2.6), so its p-value exceeds 5 percent.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Hypothesis Testing