Set out the aims, techniques and examples of exploring data
The idea
Data exploration has three parts. Exploratory data analysis summarises and visualises the data: histograms, bar charts and box plots for one feature, scatterplots and correlation matrices for pairs, and word clouds for the most frequent text tokens. Feature selection keeps only features that help explain the target, reducing noise and overfitting. Feature engineering creates new features from existing ones, for example ratios, logs, or one-hot encoding a category into indicator variables. The aim is to understand the data and give the model the most useful inputs before training starts. The common confusion is between selection, choosing among existing features, and engineering, creating new ones from them.
In the app: 2 practice questions on this outcome, a narrated walkthrough with diagrams, spaced review that brings it back before you forget it.
Take the free diagnostic →More in Big Data Projects
- Recognise the stages of a data analysis project and explain each one
- Set out the aims, stages and examples of data preparation and wrangling
- Judge how well a machine learning algorithm fits
- Explain techniques for extracting, choosing and engineering features from text data
- Set out the aims, stages and techniques of training a model
- Explain how text-based data is prepared, wrangled and explored for use in financial forecasting