Charta is on Google Play: practise on the go. Get the app
Charta

Show why backtesting should take conditional coverage into account.

FRM® Part II · Market Risk Measurement and Management · Backtesting VaR · FRM2.MR.4.e

The idea

A model can have the right number of exceptions and still be poor if they arrive in clusters. Unconditional coverage (the Kupiec test) looks only at the frequency of exceptions. Conditional coverage adds independence, so the chance of an exception tomorrow should not depend on whether one occurred today. Christoffersen's approach estimates π11, the probability of an exception after an exception, and π01, the probability after a non-exception. For a good model both equal p. The independence statistic LRind is chi-square with 1 degree of freedom. The joint statistic for conditional coverage is the sum of the frequency and independence components, so that LRcc = LRuc + LRind. It is chi-square with 2 degrees of freedom, with 5 percent critical value 5.99. Clustering usually means the model reacts too slowly to changing volatility. A model can pass the unconditional test and fail the conditional one, and two components that each pass can still fail in sum.

Formulas

Conditional coverage statistic

LRcc = LRuc + LRind

LRuc
the Kupiec unconditional coverage statistic
LRind
the independence statistic

Chi-square with 2 degrees of freedom, 5% critical value 5.99. Two components each below 3.84 can still fail in sum.

Probability of an exception after an exception

π11 = n11 ÷ (n10 + n11)

n11
days with an exception that follow an exception day
n10
days without an exception that follow an exception day

Should be close to p for a good model. Dividing by all days instead of the days that follow an exception hides clustering.

Probability of an exception after a non-exception

π01 = n01 ÷ (n00 + n01)

n01
days with an exception that follow a day without one
n00
days without an exception that follow a day without one

Compared with π11: under independence both are about p, and a π11 far above π01 shows clustering.

A practice question

From the question bank, with the reason each wrong option is wrong.

What does a conditional coverage test add to the unconditional coverage (Kupiec) test?

  1. It requires that the loss on an exception day exceed VaR by no more than a fixed percentage.Why not: The size of exceedance is not what conditional coverage tests; it concerns the timing of exceptions.
  2. It also requires exceptions to be independent over time, so an exception today should not make one tomorrow more likely. - correct
  3. It tests only whether the number of exceptions equals the expected number.Why not: That describes unconditional coverage, which ignores timing.
  4. It checks that VaR estimates at different confidence levels are consistent with each other.Why not: This is not part of the conditional coverage test, which uses a single VaR series.

Conditional coverage = correct frequency plus independence. The Christoffersen test checks whether the probability of an exception depends on whether there was one the previous day.

A harder one

A backtest of a VaR model gives LR_uc = 3.0 and LR_ind = 3.4. The 5% chi-square critical values are 3.84 with 1 degree of freedom and 5.99 with 2 degrees of freedom. Which conclusion about conditional coverage is correct?

  1. Do not reject: both components are below 3.84.Why not: Tests the parts separately; the conditional coverage test uses their sum against the 2-degree-of-freedom value.
  2. Do not reject: LR_cc = 6.4 is below 6.63.Why not: Uses the 1 percent critical value for 1 degree of freedom instead of 5.99.
  3. Reject: LR_cc = 3.0 x 3.4 = 10.2 exceeds 5.99.Why not: Multiplies the components instead of adding them.
  4. Reject: LR_cc = 6.4 exceeds 5.99, though neither part alone exceeds 3.84. - correct

LR_cc = LR_uc + LR_ind = 3.0 + 3.4 = 6.4, chi-square with 2 degrees of freedom. 6.4 > 5.99, so reject, although each component alone passes.

The hardest tier

A backtest reports LR_cc = 7.2 (5% critical value 5.99 with 2 degrees of freedom) and LR_uc = 2.1 (5% critical value 3.84 with 1 degree of freedom). Which reading is best?

  1. LR_ind = 9.3: the model fails because of clustering and excessive frequency alike.Why not: Adds LR_uc to LR_cc instead of subtracting it.
  2. LR_ind = 3.6, below 3.84: independence holds and the rejection comes from frequency.Why not: Divides LR_cc by its 2 degrees of freedom, which is not how the statistic decomposes.
  3. LR_ind cannot be inferred, because the independence statistic is computed separately from the transition counts.Why not: Under the Christoffersen decomposition LR_cc = LR_uc + LR_ind, so LR_ind is the difference.
  4. LR_ind = 5.1 exceeds 3.84: exceptions are not independent, though their frequency is acceptable. - correct

LR_ind = LR_cc - LR_uc = 7.2 - 2.1 = 5.1 > 3.84. The frequency passes (2.1 < 3.84) but independence fails, so the conditional coverage rejection is driven by clustering.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Backtesting VaR