Charta is on Google Play: practise on the go. Get the app
Charta

Compare the two methods used to rescale variables when preparing data, and apply them.

FRM® Part I · Quantitative Analysis · Machine-Learning Methods · FRM1.QA.14.b

The idea

Features measured on different scales must be rescaled before many algorithms, such as K-means, regularised regression and neural networks, will treat them fairly. There are two standard methods. Min-max normalisation rescales each value as follows. Normalised value = (x − minimum) ÷ (maximum − minimum). This places every value between 0 and 1, and it is sensitive to outliers because the extremes define the range. Standardisation rescales each value as follows. Standardised value = (x − mean) ÷ standard deviation. This gives a variable with mean 0 and standard deviation 1, and its values are not bounded. The classic error is to compute the minimum, maximum, mean or standard deviation on the whole sample. Those parameters must come from the training set only and then be applied unchanged to validation and test data. A test value can therefore legitimately fall outside 0 to 1.

Formulas

Min-max normalisation

x′ = (x − min) ÷ (max − min)

x
the raw feature value
min
the training-set minimum
max
the training-set maximum

Training values fall between 0 and 1, but test values scaled with training parameters can fall outside. Sensitive to outliers, which set the range.

Standardising a feature

z = (x − mean) ÷ standard deviation

x
the raw feature value
mean
the training-set mean
standard deviation
the training-set standard deviation

Gives mean 0 and standard deviation 1, with no fixed bounds. Compute the mean and standard deviation on the training data only.

A practice question

From the question bank, with the reason each wrong option is wrong.

A feature is rescaled with the formula (x - minimum) / (maximum - minimum). What is the range of the rescaled values in the data used to compute the minimum and maximum?

  1. A mean of 0 and a standard deviation of 1, with no fixed bounds.Why not: Describes standardisation, not min-max normalisation.
  2. From 0 to 1. - correct
  3. From -1 to 1.Why not: Min-max normalisation as defined here has a lower bound of 0, not -1.
  4. From the minimum to the maximum of the original data.Why not: Confuses the rescaled values with the original units.

Min-max normalisation maps the minimum to 0 and the maximum to 1. Standardisation, by contrast, gives mean 0 and standard deviation 1.

A harder one

A feature has 99 values between 40 and 60 and one data-entry error of 4,000. A modeller applies min-max normalisation and standardisation separately. What is the most likely effect?

  1. Min-max compresses the 99 normal values into a tiny interval near zero because the outlier sets the maximum. - correct
  2. Min-max leaves the 99 normal values spread evenly between 0 and 1.Why not: Ignores that the maximum is the outlier, which squeezes the rest.
  3. Standardisation is unaffected because the mean and standard deviation ignore extreme values.Why not: The outlier inflates both the mean and the standard deviation.
  4. Both methods always set the outlier exactly to 1.Why not: Standardisation does not bound values at 1.

Min-max divides by a range of about 3,960, so values near 50 map to about 0.003 to 0.005. Outliers should be examined before rescaling.

The hardest tier

A model is trained on a feature whose training-sample mean is 5.0 and standard deviation is 2.0. The full sample, including test data, has mean 6.0 and standard deviation 2.5. The test sample alone has mean 7.0 and standard deviation 1.25. For a test observation of 9.0, which standardised value should be fed to the model?

  1. 1.20Why not: Uses full-sample statistics, which leaks test information into the scaling.
  2. 2.00 - correct
  3. 1.60Why not: Uses the test sample's own mean and standard deviation instead of the training parameters.
  4. 1.00Why not: Divides by the training variance of 4.0 instead of the standard deviation.

Scaling parameters come from the training data only: (9 - 5) / 2 = 2.00.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Machine-Learning Methods