Charta is on Google Play: practise on the go. Get the app
Charta

Talk through the difficulties of benchmarking VaR models and the remedies suggested for them.

FRM® Part II · Market Risk Measurement and Management · Validating Bank Holding Companies’ Value-at-Risk Models for Market Risk · FRM2.MR.6.d

The idea

Benchmarking compares the VaR model, and its inputs and components, with an independent alternative, such as a different methodology, a vendor model or a simpler model, to see whether results are broadly consistent. The difficulties are several. No true benchmark exists, so a gap does not say which model is wrong. The models may differ in confidence level, horizon, data, risk factors and valuation approach. The benchmark may share the same flaws or may itself be unvalidated. Agreement shows consistency, not accuracy. The suggested remedies are to align confidence level, horizon and portfolio first, to test on simple hypothetical portfolios, to decompose differences by risk factor, volatility and correlation, and to use more than one benchmark. Benchmarking supplements backtesting and does not replace it.

Formulas

Rescaling a normal VaR to another confidence level

VaR₂ = VaR₁ × z₂ ÷ z₁

VaR₁
the VaR at the original confidence level
z₁
the normal quantile at the original level
z₂
the normal quantile at the target level

Normal returns with zero mean. Scale by the ratio of quantiles, not of confidence levels, before comparing with a benchmark.

A practice question

From the question bank, with the reason each wrong option is wrong.

What is benchmarking in the validation of a VaR model?

  1. Comparing the model's daily VaR with realised trading P&L to count exceptions.Why not: Describes backtesting, which compares VaR with realised outcomes.
  2. Comparing the model's VaR and its components with those of an independent alternative model or approach to check that results are broadly consistent. - correct
  3. Re-running the same model with changed inputs to see how VaR responds.Why not: Describes sensitivity analysis, which varies inputs of a single model.
  4. Applying an extreme historical scenario to the portfolio and measuring the loss.Why not: Describes scenario-based stress testing, not a comparison between models.

Benchmarking uses an alternative model (another methodology, a vendor system or a simplified version) as a point of comparison. The other options describe backtesting, sensitivity analysis and stress testing.

A harder one

A bank's internal 99% one-day VaR is USD 8.0 million. A vendor benchmark reports USD 6.4 million at a 97.5% confidence level, one day. Assuming normal returns (z = 2.326 at 99%, 1.960 at 97.5%), how much of the gap remains after aligning the confidence levels?

  1. About USD 0.4 million: the vendor figure scales to about USD 7.6 million at 99%. - correct
  2. About USD 1.5 million: the vendor figure scales by 99/97.5 to about USD 6.5 million.Why not: Scales by the ratio of confidence levels rather than the ratio of normal quantiles.
  3. About USD 2.6 million: the vendor figure scales by 1.960/2.326 to about USD 5.4 million.Why not: Inverts the quantile ratio, scaling the 97.5% figure in the wrong direction.
  4. About USD 1.6 million: no alignment is needed because both are one-day VaRs.Why not: Ignores the difference in confidence level, which must be aligned before comparing.

6.4 x 2.326/1.960 = 7.60 million. The remaining gap is 8.0 - 7.6 = 0.4 million, about 5%, which can then be traced to data, methodology or position differences.

The hardest tier

An internal model reports a one-day 99% VaR of USD 5.0 million. A benchmark model reports a ten-day 99% VaR of USD 14.8 million. Assume the square-root-of-time rule applies. Which interpretation is correct?

  1. The internal ten-day VaR is USD 50.0 million, far above the benchmark, signalling a severe discrepancy.Why not: Scales linearly by 10 instead of by the square root of 10.
  2. The internal ten-day VaR is about USD 1.6 million, far below the benchmark.Why not: Divides by the square root of 10 instead of multiplying to extend the horizon.
  3. The internal ten-day VaR is about USD 15.8 million, roughly 7% above the benchmark, a modest gap to investigate by decomposition. - correct
  4. The internal VaR is about 66% below the benchmark, so the two models strongly disagree.Why not: Compares 5.0 with 14.8 directly, without aligning the holding period.

5.0 x sqrt(10) = 15.81 million. 15.81 / 14.8 - 1 = 6.8%. Only after aligning horizon is the comparison meaningful, and the residual 7% is modest enough to investigate rather than reject either model.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Validating Bank Holding Companies’ Value-at-Risk Models for Market Risk