EMET8002 Chap.12 Machine Learning for Economic Prediction
Machine Learning for Economic Prediction
Prediction rather than explanation
The additional lecture contrasts econometrics, which favours models that explain, with machine learning, which favours models that predict.
It covers supervised and unsupervised learning, the bias-variance trade-off between underfitting and overfitting, and cross validation with training, validation and test sets.
Trees and ensembles
Decision trees split the data by minimising squared residuals, are easy to read but prone to overfitting, and are controlled by hyper-parameters such as depth and leaf size.
Ensembles combine many trees, and the lecture's case compares them with linear regression for predicting income.
Reporting a prediction exercise
Prediction questions reward clarity about what is being optimised.
A strong answer separates training error from test error, explains overfitting in terms of the bias-variance trade-off, names the complexity controls for a tree, and reports averaged cross-validated measures such as root mean squared error, mean absolute error and R-squared. It also keeps prediction apart from causal claims, which still need an identification strategy.
What this chapter covers
- 01
Explanation versus prediction
- 02
Supervised and unsupervised learning
- 03
The bias-variance trade-off
- 04
Training, validation and test sets
- 05
Cross validation in k folds
- 06
Regression trees and their splits
- 07
Hyper-parameters and pruning
- 08
Random forests and boosting
Worked example · free
Choosing a model by its test error
- 1Judge models on test error, since training error always rewards complexity.
- 1The shallow tree has the lowest test error, 7.0.
- 1The deep tree's gap between 1.2 and 11.8 is the signature of overfitting.
Key terms
- Supervised Learning
- Learning from labelled data with known features and outcomes in order to predict outcomes.
- Overfitting
- Fitting the training data so closely that predictions for new data become poor.
- Cross Validation
- Out-of-sample testing used to compare models and estimate how they perform on new data.
- Decision Tree
- A non-parametric model that predicts an outcome through a sequence of simple splitting rules.
- Hyper-Parameter
- A setting that controls model complexity, such as tree depth, tuned during cross validation.
Machine Learning for Economic Prediction FAQ
What is the bias-variance trade-off?
Simple, high-bias models miss real patterns and underfit, while complex, high-variance models fit the training data closely but generalise badly. The best predictive model balances the two, which is why it is chosen by its error on data it has not seen.
How does k-fold cross validation work?
The data are divided into k folds. Each fold serves once as the test set while the others train the model, and the performance measures are averaged over the k rounds. The lecture notes that researchers commonly use ten folds.
How does a regression tree choose its splits?
For every feature and candidate split point, it divides the data in two, takes the mean outcome in each part and adds up the squared residuals. It keeps the split with the smallest total and repeats the search inside each new region.
What is the difference between supervised and unsupervised learning?
Supervised learning uses labelled data with known features and outcomes to predict outcomes, through regression for continuous outcomes and classification for categorical ones. Unsupervised learning finds patterns in data without a known outcome, through clustering, density estimation or decompositions such as principal components.
What are ensemble methods?
They combine many decision trees into one more robust predictor. Averaging methods such as random forests and extra trees combine trees grown independently, while boosting methods such as gradient boosted trees and AdaBoost grow trees in sequence, each correcting the errors of the last.
Assessment move
Compute one tree split by hand on a small dataset, then explain in a sentence why the next split always lowers training error. Use that sentence to justify choosing tree depth by cross validation. Finally, explain in two sentences why a model that predicts wages well is not thereby evidence about what causes wages.
Working through Machine Learning for Economic Prediction in EMET8002? Sia is AskSia’s AI Economics tutor — ask any EMET8002 Machine Learning for Economic Prediction question and get a clear, step-by-step explanation grounded in how EMET8002 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
STAT7038 Regression Modelling · STAT7055 Introductory Statistics for Business and Finance · ECON7074 Macroeconomic Analysis