Out-of-sample
Using data that were not used to estimate the model. Out-of-sample tests show how well a model forecasts; a close fit in sample can hide overfitting.
Where you will meet it
Out-of-sample appears in 5 learning outcomes on the CFA syllabus. A few of them:
- Set out and contrast the stages and procedures involved when a strategy is backtested
“This mimics real-time investing and gives out-of-sample results.”
- Distinguish forecasts made within the sample from those made outside it, and rank time-series models by forecast accuracy using root mean squared error
“In-sample forecast errors measure how well a model fits the data used to estimate it; out-of-sample errors measure how well it predicts data it has not seen.”
- Spot flaws in the backtest of a strategy
“Remedies include point-in-time data, reporting lags, out-of-sample tests and cost models.”
- Choose a suitable time-series model for a given investment problem and give reasons for the choice
“Finally compare models on out-of-sample RMSE.”
See also
Practise it
Practise out-of-sample until it is automatic: questions that explain every wrong option.
Take the free diagnostic →