Charta is on Google Play: practise on the go. Get the app
Charta

Describe how natural language processing works and what it is used for.

FRM® Part I · Quantitative Analysis · Machine-Learning Methods · FRM1.QA.14.g

The idea

Natural language processing turns unstructured text into numbers a model can use. Typical preparation lower-cases the text, splits it into tokens, removes stop words such as the and of, and reduces words to a stem or lemma. The cleaned text is then converted to features. One example is a bag-of-words count of each term. Another is a TF-IDF weight, which scales a term frequency by how rare the term is across documents. Models then classify or score the documents. Uses in finance include sentiment scoring of news and earnings calls, reading central bank statements, extracting risk disclosures from filings, and chatbots. Standard weaknesses are negation, sarcasm, and word lists written for general language that misread financial vocabulary. Domain-specific dictionaries and context-aware models help.

Formulas

TF-IDF weight of a term in a document

TF-IDF = TF × ln(N ÷ df)

TF
the term's count divided by the number of words in the document
N
the number of documents in the corpus
df
the number of documents containing the term

A term that appears in every document gets a weight of zero, which is the point: common words carry no information.

A practice question

From the question bank, with the reason each wrong option is wrong.

What is the central task of natural language processing in finance?

  1. To forecast interest rates directly from regression of rates on their lags.Why not: Describes a time-series regression, not text processing.
  2. To standardise numerical data to mean 0 and standard deviation 1.Why not: Describes feature scaling, a data-preparation step rather than NLP.
  3. To convert unstructured text into numeric features that models can analyse. - correct
  4. To group numerical observations into K clusters.Why not: This is K-means clustering.

NLP preprocesses and encodes text (tokens, counts, TF-IDF weights, embeddings) so classification, sentiment or other analyses can be run.

A harder one

The sentence 'profit rose as profit margins improved and costs fell' is lower-cased, tokenised and cleaned by removing the stop words 'as' and 'and'. How many tokens remain?

  1. 9Why not: Counts the original tokens without removing stop words.
  2. 6Why not: Counts distinct words (profit appears twice) rather than tokens.
  3. 8Why not: Removes only one of the stop words.
  4. 7 - correct

Original tokens: profit, rose, as, profit, margins, improved, and, costs, fell = 9. Removing 'as' and 'and' leaves 7 tokens.

The hardest tier

A corpus has 1,000 filings. The term default appears in 50 of them. In one filing of 400 words it appears 8 times. Define term frequency as count divided by words in the filing, inverse document frequency as the natural log of (documents / documents containing the term), and TF-IDF as their product. What is the TF-IDF of default in that filing?

  1. 0.020Why not: Reports the term frequency only, ignoring the IDF weight.
  2. 0.026Why not: Uses a base-10 logarithm for IDF instead of the natural log.
  3. 0.060 - correct
  4. 23.97Why not: Multiplies the raw count of 8 by IDF instead of the term frequency.

TF = 8 / 400 = 0.02; IDF = ln(1000 / 50) = ln(20) = 2.996; TF-IDF = 0.02 x 2.996 = 0.060.

In the app: 6 practice questions on this outcome, spaced review that brings it back before you forget it.

Start free →

More in Machine-Learning Methods