Charta is on Google Play: practise on the go. Get the app
Charta

Describe what a p-value measures and compute it from a test statistic.

FRM® Part I · Quantitative Analysis · Hypothesis Testing · FRM1.QA.6.e

The idea

The p-value is the probability, calculated assuming the null hypothesis is true, of obtaining a test statistic at least as extreme as the one observed. It is also the smallest significance level at which the null could be rejected. The decision rule is to reject H0 if the p-value is below the chosen significance level α. For a normal test statistic z, a one-sided p-value is the tail area beyond z in the direction of H1. A two-sided p-value is twice the tail area beyond the absolute value of z, which gives this formula: P-value = 2 × (1 − Φ(|z|)). Here Φ is the standard normal cumulative distribution function. A p-value is not the probability that the null is true, and not the probability that the result arose by chance alone. A small p-value shows statistical significance, which is not the same as economic importance. The usual exam traps are forgetting to double for a two-sided test and using the wrong tail.

Formulas

Two-sided p-value from a normal test statistic

p-value = 2 × (1 − Φ(|z|))

Φ
the standard normal cumulative distribution function
z
the observed test statistic

A two-sided test with a normal statistic. For a one-sided test take the single tail in the direction of H1; the trap is comparing an undoubled tail with α in a two-sided test.

A practice question

From the question bank, with the reason each wrong option is wrong.

Which statement correctly defines a p-value?

  1. The probability that the null hypothesis is true, given the observed data.Why not: Reverses the conditioning; the p-value is computed assuming the null is true, not the probability of the null.
  2. The probability, under the null, of a statistic at least as extreme as the one observed. - correct
  3. The probability of making a Type II error given the observed sample.Why not: Confuses the p-value with beta; it is a tail probability under the null.
  4. One minus the significance level, the probability of correctly retaining the null.Why not: Describes the confidence level 1 minus alpha, which is chosen in advance and not computed from the data.

The p-value is a tail probability computed under H0. Small values mean the observed statistic would be rare if the null were true.

A harder one

A sample of 36 monthly returns has a mean of 1.34 percent and a standard deviation of 2.40 percent. Using the normal approximation, the two-sided p-value for H0: mu = 0.5 percent is closest to:

  1. 0.726Why not: Standardizes by the standard deviation 2.40 instead of the standard error 2.40/6, giving z = 0.35.
  2. 0.018Why not: Reports a single tail area and does not double it for a two-sided test.
  3. 0.036 - correct
  4. 0.982Why not: Reports the cumulative probability below z = 2.10 rather than the tail area beyond it.

SE = 2.40/sqrt(36) = 0.40, z = (1.34 - 0.50)/0.40 = 2.10. Two-sided p = 2 x P(Z > 2.10) = 2 x 0.0179 = 0.036.

The hardest tier

A software package reports a two-sided p-value of 0.06 for a test of H0: mu = 0 on a portfolio's mean return; the sample mean is +1.4 percent. The team specified H1: mu > 0 in advance. Which conclusion is correct?

  1. The p-value of 0.06 exceeds 5 percent, so fail to reject at both levels.Why not: Ignores that the directional H1 and a positive estimate mean the one-sided p-value is half the two-sided one.
  2. The one-sided p-value is 0.03, so reject at 5 percent but not at 1 percent. - correct
  3. The one-sided p-value is 0.03, so reject at 1 percent but not at 5 percent.Why not: Reverses the decision rule: 0.03 is below 5 percent, not below 1 percent.
  4. The one-sided p-value is 0.12, so fail to reject at both levels.Why not: Doubles the two-sided p-value instead of halving it.

With the sample in the direction of H1, the one-sided p-value is 0.06/2 = 0.03. It is below 0.05 (reject at 5 percent) and above 0.01 (no rejection at 1 percent).

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Hypothesis Testing