MAIE5421 Chap.3 Classification Metrics and the Confusion Matrix
Classification Metrics and the Confusion Matrix
Three shapes of classification problem
Classification learns from input-output pairs whose outputs are categories.
The lecture distinguishes binary problems with exactly two classes, such as deciding whether an email is spam or verifying that a face matches an enrolled identity; multi-class problems, in which each sample belongs to exactly one of more than two classes, such as recognising which person or which object is in an image; and multi-label problems, in which a sample can carry several labels at the same time, such as tagging a document or a photograph with every topic it contains.
The confusion matrix as the source of every score
For a binary problem, each prediction falls into one of four outcomes.
True positives and true negatives are correct. A false positive is a false alarm, a negative sample wrongly flagged; a false negative is a missed detection, a positive sample wrongly passed.
Laying the four counts out as a two-by-two table, with the prediction on one axis and the ground truth on the other, gives the confusion matrix, and every metric in the chapter is a ratio of its cells.
Four metrics and what each protects against
Accuracy is the share of all samples classified correctly, and the lecture warns that it is unreliable on imbalanced datasets, where always predicting the majority class already scores well.
Precision is the share of predicted positives that are real, so it matters when false positives are costly. Recall is the share of real positives that are detected, so it matters when false negatives are costly, as in medical screening.
The F1 score combines precision and recall into one number that is low unless both are reasonably high, which the lecture recommends when the class distribution is uneven.
Beyond two classes
For a multi-class problem the confusion matrix grows to one row and one column per class.
Overall accuracy is still the diagonal divided by the total, and precision, recall and F1 can be computed for each class by treating that class as positive and every other class as negative. For a multi-label problem, a separate two-by-two matrix is built for each label, and each label receives its own accuracy, precision, recall and F1.
What this chapter covers
- 01
Binary, multi-class and multi-label classification
- 02
True positives, false positives, false negatives and true negatives
- 03
Reading the axes of a confusion matrix
- 04
Accuracy and why it misleads on imbalanced data
- 05
Precision when false alarms are costly
- 06
Recall when missed detections are costly
- 07
The F1 score as a combined measure
- 08
Per-class metrics for multi-class and multi-label problems
Worked example · free
Choose between two pedestrian detectors with the same F1
- 2Detector A: precision 180 / 200 = 0.900, recall 180 / 240 = 0.750.
- 2Detector B: precision 220 / 300 = 0.733, recall 220 / 240 = 0.917.
- 2F1: A = 2 × 0.900 × 0.750 / 1.650 = 0.818; B = 2 × 0.733 × 0.917 / 1.650 = 0.815.
- 1Recommend B for braking, because a missed pedestrian is the costlier error.
Key terms
- Confusion Matrix
- A table counting predictions against ground truth, with one cell for each combination of predicted and actual class.
- True Positive
- A positive sample that the model correctly predicts as positive.
- False Positive
- A negative sample that the model wrongly predicts as positive, also called a false alarm.
- False Negative
- A positive sample that the model wrongly predicts as negative, also called a missed detection.
- Accuracy
- The share of all samples that the model classifies correctly, the diagonal of the confusion matrix over the total.
- F1 Score
- The harmonic mean of precision and recall, which stays low unless both are reasonably high.
- Multi-label Classification
- Classification in which one sample can belong to several classes at the same time.
- Imbalanced Dataset
- A dataset in which one class is much rarer than another, so that accuracy alone flatters weak models.
Classification Metrics and the Confusion Matrix FAQ
Why is accuracy misleading on imbalanced data?
Because a model can reach a high accuracy by always predicting the majority class. If 95 of every 100 images are clean, a model that never flags anything is 95% accurate and finds no defects at all. Precision, recall and F1 look at the positive class directly, so they expose a model that ignores it, which is why the lecture prefers them when classes are uneven.
What is the difference between precision and recall?
Precision asks, of everything the model flagged as positive, how much really was positive. Recall asks, of everything that really was positive, how much the model flagged. Precision falls when there are many false alarms and recall falls when there are many misses. Which one to prioritise depends on which error is more expensive in the application.
How do I compute precision for one class in a multi-class problem?
Treat that class as positive and every other class as negative. Its true positives are the diagonal entry for that class. Its false positives are the other samples the model assigned to that class, so precision is the diagonal entry divided by the total number of predictions of that class. Recall divides the same diagonal entry by the number of samples that truly belong to the class.
Assessment move
Draw a blank two-by-two matrix and label the axes before every metric question, then fill the four cells from the wording of the problem. Most errors in this topic come from putting a count in the wrong cell, not from the formulas, and the blank matrix forces you to decide which error is the false alarm and which is the miss before you divide anything.
Working through Classification Metrics and the Confusion Matrix in MAIE5421? Sia is AskSia’s AI Computer Science tutor — ask any MAIE5421 Classification Metrics and the Confusion Matrix question and get a clear, step-by-step explanation grounded in how MAIE5421 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.