INFS3822 Chap.7 Trees, Ensembles and Model Evaluation
Trees, Ensembles and Model Evaluation
Decision trees learn readable rules but can be unstable and prone to overfitting. Random forests reduce variance by combining bootstrap trees, while boosting learns sequentially from earlier errors. This chapter then computes accuracy, precision, recall and F1 from a confusion matrix, explains ROC/AUC, and connects metric choice to false-positive and false-negative consequences rather than treating one score as a verdict.
What this chapter covers
- 01
Decision-tree rules
- 02
Tree instability and overfitting
- 03
Random-forest bagging
- 04
Sequential boosting
- 05
Training and unseen testing
- 06
Confusion-matrix cells
- 07
Precision, recall and F1
- 08
ROC curves and AUC
Worked example · free
Recompute four metrics
- +1Accuracy = (36 + 63) / 120 = 99 / 120 = 0.825, or 82.5%.
- +1Precision = 36 / (36 + 12) = 36 / 48 = 0.75, or 75.0%.
- +1Recall = 36 / (36 + 9) = 36 / 45 = 0.80, or 80.0%.
- +1F1 = 2 x 0.75 x 0.80 / (0.75 + 0.80) = 1.20 / 1.55 = 0.774, or about 77.4%.
Key terms
- Decision tree
- A predictive model that applies learned feature splits from a root to terminal leaves.
- Random forest
- A bagged ensemble that combines many diverse decision trees.
- Boosting
- Sequential ensemble learning in which later models focus on errors left by earlier models.
- ROC curve
- A plot of classifier performance across thresholds using true-positive and false-positive rates.
Trees, Ensembles and Model Evaluation FAQ
Why can accuracy be misleading?
When one class dominates, a model can predict only the majority class and still report high accuracy. Inspect the confusion matrix and class-specific precision, recall and F1. Compare with a simple majority baseline and connect the missed cases and false alarms to operational consequences.
How does a random forest differ from one tree?
A single tree exposes one set of splits but can change sharply with the training sample. A random forest trains many trees on bootstrap samples and combines their predictions. The ensemble often generalises better, although its combined reasoning is harder to explain.
Does a higher AUC choose the deployment threshold?
No. AUC summarises ranking across thresholds and can support model comparison. Deployment still needs a specific threshold chosen from false-positive and false-negative costs, class prevalence, calibration and operational capacity. Inspect the confusion matrix at the proposed operating point.
Why should metric rates be converted to case counts?
Operations handle cases rather than percentages. Apply the proposed threshold to expected volume and prevalence, then estimate true interventions, false alarms and missed positives. Compare those counts with review capacity and error harm. This can reject a seemingly strong operating point whose workload is infeasible or whose missed cases remain unacceptable.
Assessment move
Redraw a confusion matrix from memory and calculate each metric from named cells before using a formula. Pair every score with the question its denominator answers. When comparing trees and ensembles, write one benefit and one cost, then check whether the performance difference is reliable enough to justify the cost. Trace a small decision tree from root to leaf and describe each split in plain language.
Then perturb one training case and ask why the learned structure may change. Use that instability to motivate bagging rather than treating a random forest as simply a larger tree. Explain that bootstrap samples and varied candidate features create diverse trees whose errors can average out. Contrast boosting as a sequential process that focuses later learners on earlier errors and can chase noise if complexity is uncontrolled.
Keep construction and evaluation separate. A readable tree can overfit; an opaque ensemble can generalise; neither property follows from the diagram alone. Build metric drills from counts. Start with total cases and prevalence, fill true and false cells, compute rates, and convert the rates back into expected weekly volumes. Move the threshold and predict the direction of recall, precision and workload before calculating.
When comparing two models, use the same test evidence, target and operating constraint. Inspect variation across validation folds and class-specific errors rather than declaring victory from a small aggregate difference. Add a governance column with explanation needs, maintenance and subgroup behaviour.
Practise three recommendation endings: choose the tree because its performance is close and its rules enable meaningful review; choose the forest because its improvement is stable and explanation controls are sufficient; or choose neither because the available threshold cannot meet both harm and capacity constraints. For ROC/AUC, remember that ranking across many thresholds does not select the deployed point.
Pair the curve with a confusion matrix at the proposed cut-off and the class prevalence in the use population. This makes evaluation a decision argument rather than a scoreboard.
Working through Trees, Ensembles and Model Evaluation in INFS3822? Sia is AskSia’s AI Computer Science tutor — ask any INFS3822 Trees, Ensembles and Model Evaluation question and get a clear, step-by-step explanation grounded in how INFS3822 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
