Charta is on Google Play: practise on the go. Get the app
Charta

Describe how a validator computes probability integral transforms (PITs) from a VaR model's forecasts.

FRM® Part II · Market Risk Measurement and Management · Forecasting Distribution Using Probability Integral Transform · FRM2.MR.7.b

The idea

The probability integral transform converts each day's outcome into a probability. For day t, the model issues a forecast distribution of P&L before the outcome is known, with cumulative distribution function Ft. Once the realised P&L xt is observed, the validator computes a number between 0 and 1, ut = Ft(xt). For a normal forecast with mean μt and standard deviation σt, the standardised outcome is zt = (xt − μt) ÷ σt. The PIT is then found with Φ, the standard normal cumulative distribution function, as ut = Φ(zt). For historical simulation or Monte Carlo, ut is the fraction of the simulated scenario P&Ls that are less than or equal to xt. The parameters must be the ones in force before the outcome, not re-estimated afterwards, and the P&L must be the one the model was forecasting. A low PIT means a bad outcome in the forecast tail. If the model is right, the PITs are independent and uniform between 0 and 1.

Formulas

PIT of a realised P&L under a normal forecast

uₜ = Φ((xₜ − μₜ) ÷ σₜ)

Φ
the standard normal cumulative distribution function
xₜ
the realised P&L on day t
μₜ
the forecast mean for day t, set before the outcome
σₜ
the forecast standard deviation for day t, set before the outcome

Use the parameters in force before the outcome, never ones re-estimated afterwards. The z-score alone is not the PIT: only after Φ is applied does it lie between 0 and 1.

A practice question

From the question bank, with the reason each wrong option is wrong.

In PIT backtesting, what is the probability integral transform of a day's realised P&L?

  1. The realised P&L divided by the day's VaR forecast.Why not: Builds a ratio of loss to VaR; the PIT is a probability read from the forecast distribution function.
  2. The estimated probability of an exceedance tomorrow, the exceedance count divided by the sample size.Why not: Describes the average exceedance frequency, a single number for the sample rather than a daily transform.
  3. The standardised outcome, P&L minus the forecast mean divided by the forecast sigma, which can take any real value.Why not: Describes the z-score; it becomes a PIT only after applying the standard normal distribution function, and then lies in 0 to 1.
  4. The forecast cumulative distribution function evaluated at the realised P&L, a value between 0 and 1. - correct

The PIT is u_t = F_t(x_t), the forecast distribution function F_t made before the outcome, evaluated at the realised P&L x_t. If the forecast is correct, u_t is uniform on 0 to 1.

A harder one

A model forecasts tomorrow's P&L as normal with mean 0 and standard deviation USD 2.0 million. The realised P&L is a loss of USD 3.5 million. Which value is closest to the PIT?

  1. -1.75Why not: Reports the z-score, (-3.5 - 0) / 2.0, without applying the normal distribution function.
  2. 0.04 - correct
  3. 0.08Why not: Doubles the lower tail probability as if it were a two-sided p-value; a PIT is a one-sided cumulative probability.
  4. 0.96Why not: Takes the upper tail, 1 - 0.04, which uses the loss rather than the P&L convention.

z = (-3.5 - 0) / 2.0 = -1.75, and Phi(-1.75) = 0.040. The outcome lies in the worst 4% of the forecast distribution, so the PIT is about 0.04.

The hardest tier

Five days of forecasts for daily P&L are normal, with mean, standard deviation and realised P&L (all in USD million) of: day 1 (0.4, 1.2, +1.2); day 2 (0.6, 1.2, -1.5); day 3 (0, 1.2, -1.8); day 4 (-0.2, 0.8, +1.2); day 5 (0.4, 1.0, -1.5). On how many days is the PIT below 0.05?

  1. 0Why not: Ignores the forecast means and so standardises by sigma only, finding no day below 0.05.
  2. 1Why not: Compares each realised P&L with -1.645 directly, which ignores each day's mean and sigma.
  3. 2 - correct
  4. 3Why not: Subtracts the mean but ignores each day's sigma, treating P&L minus mean as a z-score.

The PITs are: day 1 0.75, day 2 Phi(-1.75) = 0.040, day 3 Phi(-1.5) = 0.067, day 4 0.96, day 5 Phi(-1.9) = 0.029. Only days 2 and 5 are below 0.05, so the answer is 2.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Forecasting Distribution Using Probability Integral Transform