Charta is on Google Play: practise on the go. Get the app
Charta

Explain what is meant by machine learning, artificial intelligence and big data

CFA® Level I · Quantitative Methods · Introduction to Financial Data Science · QM.11.b

The idea

Big data is usually characterised by volume, velocity and variety, to which veracity is often added because the first three say nothing about whether the data is any good. Its sources are traditional financial and company records, data generated by individuals such as posts and reviews, data thrown off by business processes such as card transactions, and sensor data from connected devices. Much of it is unstructured — text, images, audio — and that is the part that is genuinely new. Artificial intelligence is the broad ambition of machines performing tasks that would otherwise need human judgement. Machine learning is the practical subset that infers a rule from data rather than being given one: supervised learning trains on labelled examples, unsupervised learning looks for structure in unlabelled data, and deep learning stacks layers to capture relationships too complex to specify in advance. The recurring failure is overfitting, where a model learns the training data's noise as though it were signal, which is why a model is judged on data it has never seen.

A practice question

From the question bank, with the reason each wrong option is wrong.

Which three features are most often used to characterise big data?

  1. Volume, velocity and variety — correct
  2. Volume, value and validityWhy not: Plausible but not the conventional set. The fourth that is sometimes added is veracity, not validity.
  3. Accuracy, brevity and clarityWhy not: Qualities of good writing rather than properties of a dataset.

Volume is how much there is, velocity how fast it arrives, and variety how many forms it takes. Veracity is often added as a fourth, because the first three say nothing about whether any of it can be believed.

A harder one

A model is trained on labelled examples of loans that did and did not default. Which kind of learning is this?

  1. Supervised learning — correct
  2. Unsupervised learningWhy not: That works on unlabelled data, looking for structure rather than predicting an answer already known for the training cases.
  3. Reinforcement learningWhy not: That learns from rewards earned by acting in an environment, not from a fixed set of labelled examples.

The labels are the supervision: each training case carries the answer, and the model is scored on how close it comes. Credit scoring, fraud detection and most return prediction are set up this way.

The hardest tier

A model fits its training data almost perfectly but performs poorly on new data. What has happened?

  1. It has overfitted — correct
  2. It has been underfittedWhy not: Underfitting shows as poor performance on the training data as well, not only on data the model has not seen.
  3. The new data must be faultyWhy not: Blaming the held-out data is the reflex that holding data back exists to prevent.

A model flexible enough to fit every wrinkle in its training set has fitted the noise along with the signal. The noise does not repeat, so the fit does not survive contact with anything new.

In the app: 9 practice questions on this outcome, a narrated walkthrough with diagrams, spaced review that brings it back before you forget it.

Take the free diagnostic →

More in Introduction to Financial Data Science