Charta is on Google Play: practise on the go. Get the app
Charta

Explain how the shape of the PIT distribution signals how good a VaR model is.

FRM® Part II · Market Risk Measurement and Management · Forecasting Distribution Using Probability Integral Transform · FRM2.MR.7.c

The idea

The histogram of PITs shows how the model fails, since a correct model gives a flat histogram with every bin holding about the same share. Use the convention that a low PIT is a bad P&L outcome. A U shape, with too many PITs near 0 and 1, means realised outcomes land in the tails more often than forecast. The model then understates risk, typically because volatility or tail thickness is too low. A hump in the middle, with few PITs in the tails, means the forecast distribution is too wide and the model overstates risk. A tilt towards 1 means outcomes keep beating the forecast, so the forecast is biased low. A tilt towards 0 means the opposite. A flat histogram is necessary but not sufficient, because the PITs must also be independent, so check for serial dependence as well.

A practice question

From the question bank, with the reason each wrong option is wrong.

A validator draws the histogram of PITs from a long backtest of a VaR model. Which shape indicates a well-calibrated model?

  1. Roughly flat, with all bins about equally full. - correct
  2. Bell-shaped and centred at 0.5, because P&L is approximately normal.Why not: Confuses the PIT with the P&L; the transform removes the shape of the forecast distribution, leaving a flat histogram if the model is right.
  3. Concentrated below 0.05, since VaR measures the left tail of the distribution.Why not: Assumes a good model puts the outcomes in the loss tail; the PIT of a good model is spread evenly.
  4. U-shaped, with PITs piled near 0 and 1, because both tails carry the risk.Why not: Mistakes a U shape for realism; it signals too many outcomes in both tails relative to the forecast.

If forecast distributions are right, PITs are uniform on 0 to 1, so the histogram is flat. Any systematic departure shows the way the model is wrong.

A harder one

Over a backtest, the PITs of a VaR model (a low PIT meaning a bad P&L outcome) bunch strongly near 1, with few below 0.5. What does this suggest?

  1. The model overstates volatility, so too few outcomes fall in the tails.Why not: Describes a hump in the centre, not a one-sided tilt; overstated volatility removes PITs from both tails.
  2. The model understates the likelihood of large losses, so too many exceedances are expected.Why not: Reads the PITs near 1 as bad outcomes; with a low PIT meaning bad, large losses would show up near 0.
  3. The PITs are independent, so the model is fine and the tilt is purely random.Why not: Mistakes the tilt for independence; a long record showing such a pattern signals a bias and is not noise.
  4. Realised P&L is systematically better than forecast, so the forecast is biased low. - correct

A PIT near 1 means the outcome was in the top of the forecast distribution. A persistent tilt towards 1 indicates a downward bias in the forecast mean or a skew the model does not capture.

The hardest tier

A model's volatility forecast is 1.5 times the true volatility, while true and forecast P&L are both normal with zero mean. Which share of PITs is closest to the proportion that fall between 0.2 and 0.8?

  1. 43%Why not: Divides by 1.5 instead of multiplying, which models understated volatility.
  2. 60%Why not: Assumes calibrated PITs, but a wrong volatility forecast distorts the shares.
  3. 79% - correct
  4. 94%Why not: Applies the variance ratio 1.5² where the volatility ratio 1.5 belongs.

u is between 0.2 and 0.8 when |x / sigma_model| < 0.8416. The true z = x / sigma_true must satisfy |z| < 0.8416 x 1.5 = 1.262, with probability 2 x Phi(1.262) - 1 = 0.79, against 60% if calibrated. The excess in the middle is the hump.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Forecasting Distribution Using Probability Integral Transform