Charta is on Google Play: practise on the go. Get the app
Charta

Walk through how credit risk scoring and rating models get built.

FRM® Part II · Credit Risk Measurement and Management · Credit Scoring and Rating · FRM2.CR.6.c

The idea

Building a scoring or rating model follows a sequence. First define default, the dependent variable, and the observation window. Then assemble historical data on borrowers with known outcomes. Screen and transform candidate variables, for example binning and weight-of-evidence coding. Then estimate the model, most often a logistic regression. In it the log-odds of default, ln(PD ÷ (1 − PD)), are a linear function z of the variables. Solving for PD turns the score z back into a probability between zero and one. PD = 1 ÷ (1 + e^(−z)). If defaults were oversampled, correct the odds back to the true population default rate. Test discrimination on a holdout sample using the ROC curve and its area, AUC. Gini = 2 × AUC − 1. Check calibration against observed default rates, map scores to grades, and set cut-offs by comparing profit on good accounts with loss on bad ones. The usual error is judging a model on the sample that built it.

Formulas

PD from a logistic regression score

PD = 1 ÷ (1 + e^(−z))

z
the score, the log odds of default: the intercept plus each coefficient × its variable

Logistic models only. e^z is the odds, not the probability, and flipping the sign of z gives the probability of no default.

Gini coefficient from the ROC area

Gini = 2 × AUC − 1

AUC
the area under the ROC curve

A random model has AUC 0.5 and Gini 0; a perfect one has Gini 1. Measure it on a holdout sample, not the build sample.

A practice question

From the question bank, with the reason each wrong option is wrong.

What is the main purpose of a holdout (out-of-sample) sample in scorecard development?

  1. To test whether the model discriminates on borrowers not used in estimation, exposing overfitting. - correct
  2. To add more defaults to the estimation data so that the coefficients are estimated more precisely.Why not: The holdout is excluded from estimation; it is not used to fit coefficients.
  3. To convert scores into rating grades on the master scale.Why not: Mapping to a master scale is a separate calibration step and does not need a holdout.
  4. To define which borrowers count as defaulted.Why not: Default definition is fixed at the start of model building and has no connection with the holdout.

A model that fits its development data well may still fail on new borrowers. Testing on data not used in estimation measures real performance and detects overfitting.

A harder one

A validation team reports that a new scorecard has an area under the ROC curve (AUC) of 0.78. The Gini coefficient (accuracy ratio) is closest to:

  1. 0.28Why not: Subtracts 0.5 from the AUC instead of applying 2 x AUC - 1.
  2. 0.44Why not: Uses 2 x (1 - AUC), the reverse of the Gini formula.
  3. 0.78Why not: Treats the Gini coefficient as equal to the AUC.
  4. 0.56 - correct

Gini = 2 x AUC - 1 = 2 x 0.78 - 1 = 0.56. A random model has AUC 0.5 and Gini 0.

The hardest tier

A developer builds a PD model on a sample containing 20% defaulters and 80% non-defaulters, because defaults are rare. The true default rate of the portfolio is 2%. For one borrower the model outputs 0.40. After correcting for the oversampling by adjusting the odds, the PD is closest to:

  1. 5.2% - correct
  2. 4.00%Why not: Scales the probability by the ratio of default rates (0.02 / 0.20), whereas the correction applies to the odds.
  3. 40%Why not: Uses the uncorrected output, which reflects the oversampled 20% default rate.
  4. 3.3%Why not: Multiplies the probability, rather than the odds, by the odds-ratio correction factor of 0.0816.

Sample odds = 0.40 / 0.60 = 0.667. Correction = (0.02 / 0.98) / (0.20 / 0.80) = 0.0816. Corrected odds = 0.0544, PD = 0.0544 / 1.0544 = 5.2%.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Credit Scoring and Rating