COMP5318 Machine Learning and Data Mining

How COMP5318 is assessed

What COMP5318 covers

Eleven chapters trace COMP5318 from foundations and distance measures through regression, Naïve Bayes, trees, SVMs and PCA, neural and deep networks, and on to clustering, Markov models and reinforcement learning.

This free study guide re-derives every core method and works each calculation by hand, matched to how COMP5318 is assessed at the University of Sydney: a 60% closed-book final exam alongside weekly Canvas quizzes and two group assignments. It turns the Machine Learning and Data Mining syllabus into reliable, method-by-method exam marks.

Worked example · free

Decision-tree information gain of a split

Key terms

Supervised learning

Learning from labelled input–output pairs to predict a target: classification when the target is a category, regression when it is a number.

Unsupervised learning

Finding structure in unlabelled data, e.g. clustering points into groups or reducing dimensions with PCA.

Overfitting

When a model fits the training noise so training error keeps falling while true (test) error rises; the best model sits at the lowest test error, not lowest train.

Euclidean distance

Straight-line distance √(Σ(aᵢ−bᵢ)²) between two feature vectors; features must be scaled first so no large-range feature dominates.

k-Nearest Neighbour (kNN)

A lazy classifier that labels a query by the majority class of its k closest training points; small k overfits, large k over-smooths.

Naïve Bayes

A probabilistic classifier picking the class that maximises P(class)·∏P(featureⱼ|class), assuming features are conditionally independent given the class.

Entropy

H(S) = −Σpᵢlog₂pᵢ, the impurity of a node in bits: 0 when pure, 1 for a two-class 50/50 split.

Information gain

The drop in entropy from a split, H(S) − Σ(|Sₖ|/|S|)H(Sₖ); a decision tree splits on the attribute with the highest gain.

Support vector

A training point lying exactly on an SVM's ±1 margin plane; only the support vectors determine the maximum-margin boundary.

Principal Component Analysis (PCA)

An unsupervised method that finds orthogonal directions of maximum variance for dimensionality reduction and compression — it ignores labels.

Backpropagation

Training a neural net by propagating error back: output δ = (t−o)o(1−o) for sigmoid, and each weight updates by Δw = η·δ·(source output).

Regularization

A penalty on the weights to curb overfitting: Ridge (L2) shrinks weights toward zero, while Lasso (L1) drives some exactly to zero for feature selection.

Confusion matrix

The 2×2 table of TP, FP, FN, TN from which accuracy, precision, recall and F1 are read; on imbalanced data prefer precision/recall over accuracy.

Q-learning

A reinforcement-learning update Q(s,a) ← Q + α[r + γ·maxₐ′Q(s′,a′) − Q] that learns action values from rewards, not labelled examples.

COMP5318 FAQ

Questions students ask in each COMP5318 chapter

Is machine learning examined by calculation or by explanation in COMP5318?

Both. The final is paper-based, 2 hours, closed book with only a non-programmable calculator, and mixes short-answer and True/False concept questions with small by-hand calculations. Foundations appear as quick items such as classifying a task as supervised or unsupervised, normalising a feature, or computing a Euclidean or Manhattan distance, so you are marked on method plus the correct final number plus a one-line reason. Chapter 01 · Foundations of Machine Learning & Data Mining

Do I have to know both Euclidean and Manhattan distance for the exam?

Yes. Short distance calculations appear in the k-NN style of question, and the paper may specify either metric, so practise both. Euclidean squares each coordinate difference then takes a square root; Manhattan sums the absolute differences. Because the exam is closed book with only a non-programmable calculator, the numbers are kept small enough to work by hand — always show the working, since method marks are awarded even if one slip changes the final vote. Chapter 02 · k-Nearest Neighbour & Rule-Based Classifiers

Is linear regression fitted the same way as logistic regression?

No — and the exam tests the difference. Linear regression fits a numeric-target line by least squares (minimising SSE), which has a closed form. Logistic regression is a two-class classifier: it runs the linear score through a sigmoid and chooses the weights that make the observed 0/1 labels most probable — maximum likelihood, solved iteratively. Writing 'logistic regression minimises SSE' is a common false statement. Chapter 03 · Linear & Logistic Regression, Regularization

Why is Naïve Bayes called "naïve", and does the assumption break it?

It is naïve because it assumes the features are independent of each other once you know the class, which is rarely true of real data. In practice the classifier still performs well and is very cheap: one pass to count the training data, then a product of likelihoods to predict. The assumption is what lets you multiply the per-feature likelihoods instead of estimating a full joint distribution, so it is a deliberate simplification rather than a flaw. Chapter 04 · Naïve Bayes & Model Evaluation

How do I decide which attribute a decision tree splits on?

Compute the information gain of every candidate attribute at that node and split on the one with the highest gain. Information gain is the parent entropy minus the weighted average of the child entropies, where each child is weighted by its share of the examples (|Sₖ|/|S|), not a plain average. Because a pure child has entropy 0, an attribute that produces a pure branch tends to win. Watch the bias: information gain favours many-valued, ID-like attributes that split data into near-unique singletons — those look perfect but only memorise the training set, which is why gain ratio or dropping ID columns is used. Chapter 05 · Decision Trees & Ensembles

How do I tell which points are support vectors, and why do only they matter?

Solve for the boundary first, then test each point against the supporting hyperplanes w·x + b = ±1. A point is a support vector only if it lies exactly on its class's line, i.e. yᵢ(w·xᵢ + b) = 1. Every other correctly classified point sits strictly beyond its margin (yᵢ(w·xᵢ + b) > 1). Only the support vectors touch the margin, so they alone pin the boundary in place — deleting a non-support-vector changes nothing, while deleting a support vector can move the boundary. That is why an SVM is defined by a handful of points, not the whole dataset. Chapter 06 · Support Vector Machines, Kernels & PCA

Is neural networks examined by calculation or by explanation in COMP5318?

Both, and usually by calculation. The final is paper-based, 2 hours, closed book with only a non-programmable calculator, and neural networks appear as short numeric items: run a forward pass through a small sigmoid network, take one backpropagation step to update a weight, count trainable parameters of a Dense network, or explain the vanishing-gradient problem in a sentence. Marks are awarded for the method and the correct final number, so always show each substitution. Chapter 07 · Neural Networks & Backpropagation

Why is a CNN better than an ordinary (dense) neural network for images?

A fully-connected net gives every pixel its own weight to every neuron — an enormous parameter count with no sense of locality and no reuse. A CNN builds in image structure: local receptive fields (a pixel matters most to its neighbours), weight sharing (one small filter reused everywhere, so far fewer parameters and less overfitting), and translation invariance (a feature is detected wherever it appears). That is why the standard answer to 'classify MNIST handwritten digits' is a CNN rather than an MLP or k-NN. Chapter 08 · Deep Learning: CNN, RNN & Transformer

How is clustering different from classification?

Classification is supervised — it is given class labels and learns to predict them. Clustering is unsupervised — there are no labels, so it can only group similar examples together, never 'predict a class'. That is why clustering quality is judged by internal measures such as cohesion and separation (or the silhouette coefficient), not by accuracy against labels, unless a separate ground-truth is explicitly provided for an external measure. Chapter 09 · Clustering

What is the difference between the Forward and Viterbi algorithms?

They share the same trellis and the same initialisation, but they answer different questions and use a different operator. Forward SUMS over all incoming paths and returns a probability P(observation sequence) - it evaluates how likely the observations are. Viterbi takes the MAX incoming path (and stores back-pointers) and returns a sequence of states - it decodes the single most likely hidden path. A quick check in the exam: if the question asks for a probability, use Forward's sum; if it asks which states, use Viterbi's max and trace the pointers back. Chapter 10 · Markov Models & HMMs

What is the main difference between reinforcement learning and supervised learning?

In supervised learning the model is trained on labelled examples and tries to match a given target for each input. In reinforcement learning there are no labels: the agent receives only a scalar reward signal from the environment, and its objective is to maximise expected cumulative (discounted) future reward. There are two further contrasts the exam rewards naming: the agent's own actions change the data (states) it sees next, and rewards can be delayed rather than immediate. This is a common 2-mark short-answer, so list two or three of these differences explicitly. Chapter 11 · Reinforcement Learning

How to study for the exam

Treat COMP5318 Machine Learning and Data Mining at the University of Sydney as a breadth unit assessed mostly by a 60% closed-book final: keep pace with the weekly Canvas quizzes, start the two group assignments early, and above all rehearse the by-hand calculations that recur on the paper — a distance, a kNN vote, a Naïve Bayes product, entropy and information gain, precision/recall/F1, a PCA judgement, a backpropagation step, an HMM Forward pass and a Q-learning update.

For each, name the method, do one clean calculation, and write the one-line reason; because the exam carries a 40% hurdle, practise touching every topic rather than perfecting one, and use Sia to explain any step you get stuck on so it lifts your WAM without doing the work for you.

Your AI Computer Science tutor for COMP5318

Stuck on a hard COMP5318 question? Sia is AskSia’s AI Computer Science tutor — ask any COMP5318 Machine Learning and Data Mining question and get a clear, step-by-step explanation grounded in how the course is actually taught and assessed. Read this whole study guide free, then take your hardest questions to Sia.