Outline the methods for encoding categorical variables and apply them.
The idea
Models need numbers, so categorical variables must be encoded. One-hot encoding creates one 0/1 column per category and suits nominal variables with no order, such as sector or country. In a regression with an intercept, keep only k − 1 columns for k categories. The dropped one is the reference. Including all k makes the columns sum to the intercept, the dummy variable trap of perfect multicollinearity. Each dummy coefficient is then measured against the reference category. Ordinal (integer) encoding assigns 1, 2, 3 and suits genuinely ordered categories such as rating buckets. Used on nominal data it imposes a false order and spacing. For categories with very many levels, target encoding replaces a level by the mean outcome for that level. It must be computed out-of-fold or smoothed, or the label leaks into the feature. Frequency and binary encoding are compact alternatives.
A practice question
Which of the following variables is best suited to ordinal (integer) encoding?
- Credit rating bucket with levels AAA, AA, A, BBB, BB and B. - correct
- Country of incorporation.Why not: A nominal variable; integer codes would imply that one country ranks above another.
- Industry sector of the borrower.Why not: A nominal variable; coding sectors 1 to 8 would suggest an arbitrary distance between them.
- Currency of the loan.Why not: A nominal variable; an integer code would make one currency appear as a multiple of another.
Ordinal encoding is justified only when categories have a natural order. Ratings are ordered; countries, sectors and currencies are not.
A harder one
A regression with an intercept uses Sector, which has 6 categories, as a predictor. To avoid the dummy variable trap, the number of dummy columns to include is:
- 6Why not: Includes a column for every category, so the dummies sum to the intercept column and create perfect multicollinearity.
- 1Why not: Uses a single integer-coded column, which is label encoding rather than dummy coding.
- 5 - correct
- 7Why not: Adds an extra column for the intercept to the six sector dummies, which compounds the redundancy.
With k categories and an intercept, use k - 1 = 5 dummies. The omitted sector is the reference category.
The hardest tier
A credit model has three numeric features and three categorical variables: Sector (8 nominal levels), Region (5 nominal levels) and Rating bucket (7 ordered levels). The model includes an intercept. Nominal variables are dummy-encoded with one reference level dropped, and Rating is integer-encoded in a single column. The number of input columns, excluding the intercept, is:
- 17Why not: Keeps all levels of Sector and Region as columns (8 + 5) instead of dropping one reference level from each.
- 15 - correct
- 20Why not: Dummy-encodes Rating with 6 columns even though it is an ordered variable with a single integer column.
- 23Why not: Combines both errors: no reference level dropped and Rating fully one-hot encoded.
3 numeric + 7 (Sector, 8 - 1) + 4 (Region, 5 - 1) + 1 (Rating) = 15 columns.
In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.
Start free →More in Machine Learning and Prediction
- Say how forecasting a quantity can draw on a linear regression and on a logistic regression, and what each does there.
- Assess how well logistic regression models predict.
- Explain the usefulness of regularization and set ridge regression alongside LASSO for comparison.
- Show how a decision tree is built and how it should be read.
- Explain how a set of learners is combined into an ensemble.
- Describe the underlying intuition and the working steps of two classification methods, K nearest neighbors and support vector machines.
- Describe how neural networks are put together and how the values of their weights are arrived at.
- Use a confusion matrix to compare the logistic regression and neural network approaches to classification.