BMS5010 Chap.2 Supervised Learning: Classification and Regression
Supervised Learning: Classification and Regression
What the label buys and what it costs
Supervised learning is defined in the course as the category of machine learning that trains algorithms on labelled datasets so that they predict outcomes and recognise patterns, and the strategy is given in three moves: collect a dataset of labelled training examples, teach a model to produce accurate predictions on it, and rely on the model staying accurate on new data of the same kind.
The whole contrast with unsupervised work is the label and nothing else. What is easy to forget is that the label is a record of a decision somebody already made, under conditions that were not designed with a model in mind, so a model can only ever learn to reproduce that decision.
Two questions fix the task before any library is opened
Is every row labelled, and is the target a category or a quantity.
Those two answers determine the task type, and the task type determines which error measures are even available, which is why the evaluation chapters are separate. Classification places inputs into discrete categories, and the course separates binary from multiclass with its own clinical examples.
Binary: whether a patient responds to immune therapy from genetic data, whether a tumour in an image is malignant or benign, whether a DNA sequence belongs to a healthy person or to someone carrying a genetic disorder.
Multiclass: which disease from clinical data, which cell type from stained tissue, which stage of cancer from molecular markers.
Why the count of categories is not a detail
A binary problem has a single decision threshold that can be moved and one pair of error types to trade against each other, and that trade is the whole content of the confusion matrix chapter.
A multiclass problem has neither, so the summary figures you may quote change and a claim about sensitivity has to name the class it refers to. There is a third shape worth recognising: an output that returns a separate score for each of several findings, as in the chest radiograph example, where one image can carry several findings at once.
That is not one class from a list but a set of independent binary answers, and reading it as multiclass changes every denominator.
Regression, and the unit that has to travel with the number
Regression uses the same labelled setup to predict a continuous output, and the examples the course gives are resource and outcome questions rather than diagnostic ones: inpatient mortality and long length of stay from medical text, the number of emergency department arrivals in a period along with the staff and beds that implies, and survival from health conditions and age.
Two habits protect a regression answer. Carry the unit through every sentence, because an error of four is meaningless and an error of four days is a decision.
And say what the model is conditioned on, because a prediction made at admission and one made on the third day are different models with different uses.
The four models the assignment compares
A linear model predicts an outcome as a weighted sum of the inputs plus an intercept, and fitting means choosing the weights that make the predictions closest to the recorded outcomes under some definition of closest.
The assignment replaces the plain fit with three alternatives from the same library and asks you to compare all four by their squared error. Two of them add a penalty on the size of the coefficients and choose its strength by internal resampling; the third reaches the same objective by repeated small steps rather than by solving it directly.
Four models, one dataset, and a table whose value is showing which differences are real.
What each penalty actually buys
Penalising the sum of absolute coefficient values drives some of them exactly to zero, so the model selects variables as a side effect.
Penalising the sum of squared values shrinks them all toward zero without eliminating any, so correlated variables share the weight instead of one taking it all. The course names this correspondence when it introduces the two error measures, and it is worth memorising in that direction because a question about regularisation is usually a question about loss.
One consequence matters for the demonstration: the penalised estimators already use resampling internally, so quoting their training error as evidence of anything is a mistake.
What this chapter covers
- 01
Labelled data, and the decision a label actually records
- 02
Binary against multiclass, and the multi-label shape that is neither
- 03
Continuous targets, units, and what a model is conditioned on
- 04
The linear prediction written out, and what fitting means
- 05
Absolute-value against squared penalties on the coefficients
- 06
Why a penalised estimator's training error is not evidence
- 07
Annotation as the real cost of every supervised application
Choose between a better fit and a better estimate of the future
- 2Say which of the four numbers estimates future performance.
- 3Name the pattern the pair of pairs demonstrates.
- 2State what the penalty bought and what it cost.
Key terms
- Label
- The recorded correct answer attached to a training example, which encodes a decision made earlier by a person under conditions the model never sees.
- Binary Classification
- A supervised task with exactly two categories, which is the only shape that has a single movable decision threshold and one pair of error types.
- Multiclass Classification
- A supervised task assigning each instance to one of three or more categories, where a claim about sensitivity has to name which category it concerns.
- Intercept
- The constant term in a linear prediction, representing the predicted outcome when every input sits at zero.
- Sparse Model
- A fitted model in which many coefficients are exactly zero, produced by penalising the sum of absolute coefficient values.
Supervised Learning: Classification and Regression FAQ
How do I decide whether an ordered outcome is a category or a quantity?
Ask whether the true underlying values are recoverable. Categories such as no event, one event and more than one are ordered, so treating them as a quantity keeps the information that two is further from zero than one is. But the top category has no upper bound, so any number a regression produces above one is an invention.
Predict the count only when the true counts exist in the data; otherwise keep the ordered categories and say in the methods that the ordering is not being used, which is honest and costs you nothing.
Which of the four assignment models should I expect to win?
There is no expected winner, and a report that predicts one has misunderstood the exercise. Which estimator generalises best depends on how many of the variables carry signal and how correlated they are, and the comparison exists to find that out on this dataset.
What the report should predict is the shape of the answer: if most variables are irrelevant the absolute-value penalty should do well, and if they are correlated and all plausible the squared penalty should. Saying which pattern you expected and whether the table agreed is worth more than the table.
Does swapping the estimator change how I prepare the data?
It changes how much the preparation matters. Penalties act on coefficient size, and a coefficient's size depends on the scale of its variable, so a penalised fit on unstandardised inputs penalises the variables measured in small units far more heavily than the others. Standardise inside the training portion and apply the same constants outward.
The unpenalised fit is indifferent to this, which is precisely why a comparison run without standardisation can make the penalised models look worse than they are.
Assessment move
Write the two task-type questions at the top of every notebook section before importing anything: what is labelled, and is the target a category or a quantity. Then add one line saying what a single error would mean to the person affected. Those three lines fix the metric you are allowed to quote and are also the opening of your demonstration, which makes them the cheapest preparation available.
Working through Supervised Learning: Classification and Regression in BMS5010? Sia is AskSia’s AI Artificial Intelligence tutor — ask any BMS5010 Supervised Learning: Classification and Regression question and get a clear, step-by-step explanation grounded in how BMS5010 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
