COMP4318 Machine Learning and Data Mining
The Complete Exam Bible · S1 2027COMP4318 Overview
- University of Sydney
- S1 2027
- 11 chapters
- Computer Science
COMP4318/COMP5318 Machine Learning and Data Mining is the University of Sydney's co-badged undergraduate/postgraduate machine-learning unit, taught in the School of Computer Science over weekly two-hour lectures and one-hour tutorials (standard 6 credit points — confirm on the unit outline).
- Assessed by Weekly Homework Quizzes 5% · Assignment 1 15% (confirm the current weight on Canvas) + 2 more
- Key terms Supervised learning, Unsupervised learning, Overfitting, Regularization
- How to prepare Treat COMP4318 as a set of algorithms you must be able to run by hand, not just import.
- Most asked Where can I find past exam papers/practice for COMP4318?
How COMP4318 is assessed
| Component | Weight | Format |
|---|---|---|
| Weekly Homework Quizzes | 5% | 10 weekly online quizzes (weeks 2–11), direct application of that week's lecture; no late submissions |
| Assignment 1 | 15% (confirm the current weight on Canvas) | Group programming task + short report (out ~Week 3, due ~Week 7) |
| Assignment 2 | 20% (confirm the current weight on Canvas) | Group programming task + report discussing the results (out ~Week 8, due ~Week 11) |
| Final Exam | 60% | Paper-based, 2 hours, closed book; non-programmable calculator permitted (confirm any permitted A4 note sheet on Canvas) |
What COMP4318 covers
COMP4318/COMP5318 is assessed on Canvas through ten weekly homework quizzes (5%), two group programming assignments (roughly 15% and 20% — confirm the current weights on the unit outline) and one comprehensive paper-based final exam worth 60%, which carries a hard 40%-on-the-exam hurdle.
This eleven-chapter map follows the weekly lectures from the machine-learning pipeline through to reinforcement learning, and every chapter is written so you can run the algorithm by hand and explain it — which is exactly what the closed-book exam asks. Use it to see how each week's method builds toward that final.
Introduction to Machine Learning & Data Mining
The ML & data-mining pipeline · supervised vs unsupervised · training/validation/test · the workflow (Week 1)Week 1 sets up the whole unit: what machine learning and data mining are, the end-to-end pipeline (data → preprocessing → model → evaluation → deployment), and the three learning paradigms — supervised, unsupervised and reinforcement.
- What machine learning is (building models that generalise from data) and how data mining relates to it
- The ML pipeline: data → preprocessing/normalisation → model → evaluation → deployment
- Supervised (labelled → predict) vs unsupervised (unlabelled → group) vs reinforcement learning
- Attribute types: categorical/nominal vs numeric/continuous
- Training vs test sets; accuracy = proportion of test examples correctly classified
- Generalisation, features and labels — the vocabulary the unit reuses
Nearest Neighbour & Rule-Based Algorithms
k-NN · distance metrics · choosing k · 1R & PRISM rule learners (Week 2)Week 2 introduces instance-based learning — k-nearest-neighbours with its distance metrics and the effect of k — and rule-based learners (1R and PRISM) that build IF–THEN rules.
- k-NN: store all training examples, classify a new point by majority vote of its k nearest neighbours
- Distance metrics: Euclidean (L2, √ of sum of squares), Manhattan (L1, sum of absolute differences), Minkowski
- Feature scaling / min-max normalisation so large-scale features don't dominate the distance
- Choosing k: rule of thumb k ≤ √(#training examples); larger k = more robust to noise, k=1 fits the training set perfectly
- Weighted k-NN (weight ∝ 1/d²) and regression k-NN (average the neighbours' values)
- Complexity: training O(1) (just storing), classification O(mn); speed up with KD-trees / ball trees
Linear & Logistic Regression; Overfitting & Regularization
Linear & logistic regression · the sigmoid · overfitting · Ridge (L2) & Lasso (L1) (Week 3)Week 3 covers linear regression by least squares, logistic regression and the sigmoid for classification, and the overfitting problem with its regularization fixes (Ridge/L2 and Lasso/L1). It is the unit's first real dose of the bias–variance trade-off.
- Linear regression line ŷ = b0 + b1·x; least-squares slope b1 and intercept b0 from the sums
- Error sums SSE, SST, SSR and the identity SST = SSR + SSE; R² = SSR/SST in [0,1]
- Error metrics MAE, MSE, RMSE for train or test predictions
- Logistic regression for 2-class problems; the sigmoid/logistic curve P = e^(z)/(1 + e^(z)), z = b0 + b1·x
- Log-odds (logit) form ln(p/(1−p)) = b0 + b1·x; fit by maximum likelihood; threshold at 0.5
- Overfitting (low train, high test error) vs underfitting; the complexity–generalisation trade-off
Naïve Bayes & Evaluating ML Methods
Bayes' theorem · naïve Bayes · confusion matrix · precision/recall/F1 · cross-validation (Week 4)Week 4 builds the naïve Bayes classifier from Bayes' theorem and the conditional-independence assumption, then covers how to evaluate any classifier: the confusion matrix, accuracy/precision/recall/F1, and cross-validation.
- Bayes' theorem: P(H|E) = P(E|H)·P(H) / P(E), with H = class and E = evidence
- Naïve Bayes assumptions: attributes conditionally independent given the class, and all equally important
- Classification: P(H|E) ∝ P(E1|H)·…·P(Ek|H)·P(H); pick the class with the largest product
- Zero-frequency problem and the Laplace / m-estimate smoothing fix
- Gaussian naïve Bayes for numeric attributes via the normal PDF f(x) = 1/(σ√(2π))·e^(−(x−μ)²/(2σ²))
- Confusion matrix; accuracy, precision = tp/(tp+fp), recall = tp/(tp+fn), F1 = 2PR/(P+R)
Decision Trees & Ensembles
Entropy & information gain · Gini · bagging · boosting (AdaBoost/gradient) · random forests (Week 5)Week 5 covers decision-tree induction with entropy and information gain (and Gini as an alternative), pruning, then ensembles — bagging, boosting (AdaBoost and gradient boosting) and random forests.
- Top-down recursive divide-and-conquer tree induction; a node becomes a leaf when it is pure
- Entropy H(S) = −Σ pᵢ·log2(pᵢ) in bits (with log2(0) = 0); pure = 0, 50/50 = 1
- Information gain Gain(S,A) = H(S) − Σ (|Sᵢ|/|S|)·H(Sᵢ); pick the attribute with the highest gain
- Highly-branching-attribute bias and the gain-ratio fix; Gini index as CART's alternative purity measure
- Pruning: pre- vs post-pruning (post preferred), validated on a hold-out set
- Ensembles work when the base classifiers are accurate (error < 0.5 for binary) AND diverse
Support Vector Machines & Dimensionality Reduction
The maximum-margin classifier · the kernel trick · PCA · SVD (Week 6)Week 6 covers support vector machines — the maximum-margin hyperplane, soft margins and the kernel trick — then dimensionality reduction with PCA and SVD.
- Linear decision boundary w·x + b = 0; classify a new point z by sign(w·z + b)
- Support vectors = the closest points, on H1: w·x+b=1 and H2: w·x+b=−1; SVM picks the maximum-margin hyperplane
- Margin size d = 2/‖w‖; maximising the margin ⇔ minimising ½‖w‖² subject to yᵢ(w·xᵢ+b) ≥ 1
- Soft margin: the parameter C trades margin width against training error (large C → fit the training data harder)
- Kernel trick: K(u,v) = Φ(u)·Φ(v) computes high-dimensional dot products without ever forming Φ
- Common kernels (Mercer's theorem): polynomial (x·y+1)^p, RBF e^(−‖x−y‖²/(2σ²)), tanh
Neural Networks: Perceptrons & Backpropagation
The perceptron · activation functions · gradient descent · backpropagation (Week 7)Week 7 builds neural networks from the perceptron up: activation functions, the multi-layer perceptron, gradient descent and the backpropagation algorithm that trains the weights.
- Neuron output a = f(w·x + b), where f is the activation/transfer function
- Perceptron: step activation, binary output; learning rule e = t − a, w ← w + e·x, b ← b + e
- Perceptron limits: solves AND/OR/NAND but NOT XOR (needs a 2-layer net); converges iff data are linearly separable
- Multi-layer perceptron: feedforward, fully connected, differentiable activation (sigmoid f(z) = 1/(1+e^(−z)))
- Sigmoid derivative f'(x) = f(x)(1 − f(x)); the vanishing-gradient problem; ReLU y = max(0, x) as a fix
- Backpropagation: forward pass, then propagate error backward by gradient descent; Δw = η·δ·o
Deep Neural Networks: CNNs, RNNs & Transformers
Convolutional nets · recurrent nets · transformers & attention · what depth adds (Weeks 8–9)Weeks 8–9 cover deep learning: convolutional neural networks (CNNs) for images, recurrent networks (RNNs/LSTM/GRU) for sequences, and the transformer/attention architecture, along with softmax and cross-entropy.
- CNNs for images: CONV + ReLU + POOL + FC layers; parameter sharing and local receptive fields beat a plain dense net
- Max-pooling vs average-pooling and their drawbacks
- Counting trainable parameters of a Dense layer: inputs·units + units biases (Flatten/pooling add none)
- RNNs for sequences and their vanishing gradients over time
- LSTM (forget/input/output gates plus a cell state) vs GRU (reset/update gates)
- Transformer: multi-head self-attention with Q/K/V matrices, concatenate then W_O
Clustering
k-means · Gaussian mixtures · hierarchical clustering · DBSCAN (Week 10)Week 10 covers unsupervised clustering: k-means (and choosing k), Gaussian mixture models with EM, hierarchical/agglomerative clustering and the dendrogram, and density-based clustering (DBSCAN).
- Clustering = unsupervised grouping: high intra-cluster cohesion, low inter-cluster similarity
- k-means: assign each point to the nearest centroid, recompute centroids, repeat until stable; SSE compares clusterings
- Choosing k and the sensitivity to initial centroids; k-means++ spreads the initial centroids out
- GMM with EM: soft assignment in the E-step, re-estimate μ and σ in the M-step; allows elliptical clusters
- Hierarchical: agglomerative (merge closest) vs divisive; single/complete/average/Ward link; the dendrogram
- DBSCAN: parameters Eps and MinPts; core / border / noise points; finds arbitrary shapes and resists noise
Markov Models & Hidden Markov Models
Markov chains · hidden Markov models · the Forward algorithm · Viterbi (Week 11)Week 11 covers sequence models: Markov chains and the Markov property, hidden Markov models (states, emissions, transitions), and the inference algorithms — the Forward algorithm for an observation's probability and Viterbi for the most likely state sequence.
- Markov property: P(πₜ | π₁…πₜ₋₁) = P(πₜ | πₜ₋₁); transition matrix A and initial distribution A0
- Sequence probability P(π1,…,πK) = P(π1)·Π P(πᵢ₊₁ | πᵢ)
- Hidden Markov Model λ = (states, A, A0, emission matrix E); the states are hidden, only observations are seen
- Problem 1 (Evaluation): probability of an observation sequence via the Forward algorithm
- Forward recurrence: fₖ(1) = A0(k)·eₖ(x1); fₖ(i) = eₖ(xᵢ)·Σⱼ fⱼ(i−1)·aⱼₖ; P(X) = Σₖ fₖ(m)
- Problem 2 (Decoding): most likely hidden-state sequence via Viterbi (max instead of sum, plus back-pointers)
Reinforcement Learning
Markov decision processes · value & policy · Q-learning · deep Q-learning (Week 12)Week 12 introduces reinforcement learning: the agent–environment loop, Markov decision processes, value functions and optimal policies, Q-learning, and deep Q-learning with experience replay — learning from reward rather than labels.
- The agent–environment loop: states, actions, rewards R[s, a, s'] and the discount factor γ
- The Markov decision process (MDP) as the reinforcement-learning setting
- The value function V*(s) = cumulative (discounted) reward under the optimal policy
- The optimal policy: the action in each state that maximises long-run reward
- Q-learning: learning action-values Q(s, a) from experience
- Deep Q-learning: a neural network approximates Q; experience replay stabilises training
It is a practical, technical introduction to machine learning and data mining: the end-to-end pipeline (data → preprocessing → model → evaluation), then the algorithm families in sequence — nearest neighbour and rule-based learners, linear and logistic regression with regularization, naïve Bayes and model evaluation, decision trees and ensembles, support vector machines and dimensionality reduction, neural networks and deep learning, clustering, Markov and hidden-Markov models, and reinforcement learning.
The unit is assessed on Canvas through ten weekly homework quizzes (5%), two group programming assignments (roughly 15% and 20% — confirm the current weights on the unit outline) and a comprehensive final exam worth 60%.
That final is paper-based, closed book and two hours (plus reading time), with a non-programmable calculator permitted, and it carries a hard hurdle: you must score at least 40% on the exam itself to pass, regardless of your semester marks, and you still need an overall final mark of 50 or more.
Because the exam rewards running each algorithm by hand — computing an entropy, a distance, a Bayes posterior, a backprop update, a Viterbi path — steady weekly practice beats a last-minute STUVAC cram, and the COMP4318 result feeds the Weighted Average Mark (WAM) that later Sydney computer-science units build on.
Worked example · free
Confusion matrix: accuracy, precision, recall and F1 of a spam filter
- +1Build the confusion matrix with spam as the positive class. TP = 150 (spam correctly flagged); FP = 30 (ham wrongly flagged as spam); FN = 200 − 150 = 50 (spam missed); TN = 300 − 30 = 270 (ham correctly left alone).
- +1Accuracy = (TP + TN)/total = (150 + 270)/500 = 420/500 = 0.84.
- +1Precision = TP/(TP + FP) = 150/(150 + 30) = 150/180 = 0.833.
- +1Recall = TP/(TP + FN) = 150/(150 + 50) = 150/200 = 0.75.
- +1F1 = 2·P·R/(P + R) = 2·(0.833·0.75)/(0.833 + 0.75) = 1.25/1.583 = 0.79.
Key terms
- Supervised learning
- Learning a mapping from labelled examples so you can predict the label (class or value) of new, unseen examples. Classification and regression are the two supervised tasks in the unit.
- Unsupervised learning
- Learning structure from unlabelled data — grouping similar examples (clustering) or reducing dimensions — with no target to predict.
- Overfitting
- When a model has low training error but high test error because it has memorised noise rather than generalised; the opposite is underfitting (too simple, poor on both). Controlled by the complexity–generalisation trade-off.
- Regularization
- Explicitly restricting a model to avoid overfitting: Ridge (L2) adds α·Σwᵢ² to the cost and shrinks coefficients; Lasso (L1) adds α·Σ|wᵢ| and can drive some coefficients to exactly 0 (feature selection).
- Cross-validation
- Splitting the training data into k folds, training on k−1 and testing on the held-out fold in turn, then averaging the k accuracies. Stratified 10-fold is the ML standard; the test set is never used to tune hyperparameters.
- Confusion matrix
- A table of true/false positives and negatives from which accuracy = (tp+tn)/total, precision = tp/(tp+fp), recall = tp/(tp+fn) and F1 = 2PR/(P+R) are computed.
COMP4318 FAQ
Is COMP4318 hard?
It is demanding but tractable if you keep up. COMP4318/COMP5318 is broad — a new algorithm family almost every week, from k-NN and naïve Bayes to SVMs, deep networks, HMMs and reinforcement learning — and the closed-book final expects you to run each method by hand with a non-programmable calculator, not just import it.
The maths is mostly algebra, probability and a little linear algebra, so the real challenge is breadth and consistency. Students who do the weekly homework quiz on time and work one small numeric example of each week's method tend to find it manageable; leaving it to STUVAC is what makes it feel hard.
A High Distinction (HD, 85+) rewards being able to start every topic cleanly, which is what the 40% exam hurdle is really testing.
Can AI help me with COMP4318?
Yes — Sia is an AI tutor trained on how COMP4318/COMP5318 is actually taught and assessed, so it can walk you through a worked exam question step by step (an entropy split, a naïve Bayes posterior, a backprop update or a Viterbi path) and check your reasoning as you go. Use it to understand the method and rehearse under exam-style conditions.
It does not do graded assessment for you, and University of Sydney academic-integrity rules apply — any AI-tool use in your assignments must be acknowledged, and unapproved or unacknowledged use is a breach.
Where can I find past exam papers/practice for COMP4318?
Start on Canvas, where the unit posts its lecture slides, tutorial and homework-quiz solutions and any released sample papers, and check the University of Sydney Library exam-paper collection for past papers. The tutorials and weekly quizzes are the closest match to the exam's style.
This guide also includes a re-authored practice exam that mirrors the final's shape — distances, Bayes, information gain, SVM margins, a forward pass, a Viterbi path — with fresh numbers, and you can ask Sia to generate extra questions in the same style and explain each step. Treat any third-party 'model answers' with caution and confirm what is officially provided on Canvas.
Does COMP4318 have a hurdle?
Yes. The final exam is worth 60% of the unit and is a hurdle set by the School of Computer Science: you must score at least 40% on the exam itself (40 of 100 marks) to pass, regardless of your semester marks. A student who misses the hurdle is capped at a maximum final mark of 45, and to pass overall you also need a final mark of 50 or more.
That is why it pays to cover every topic through the semester rather than banking on a strong coursework total — confirm the exact rule for your offering on the Canvas Assessments page and the unit outline.
Is the COMP4318 final open- or closed-book?
The available unit materials describe it as paper-based, two hours, closed book with a non-programmable calculator permitted, and no materials supplied. However, some prior offerings of the co-badged unit allowed a single self-made A4 note sheet, so do not assume you can bring nothing — 'no materials supplied' means no formula sheet is given, which is not the same as a note sheet being banned.
Confirm the exact permitted-materials rule on Canvas and the exam-paper front page, along with the date, time and room on the University of Sydney exam timetable, before you rely on either interpretation.
Questions students ask in each COMP4318 chapter
- What's the difference between machine learning and data mining?
- They overlap heavily. Machine learning is about algorithms that build models which generalise from data; data mining is the broader activity of exploring and analysing large datasets to find useful patterns, and it uses machine-learning methods to do so. In this unit they are taught together — you learn the algorithms (machine learning) and how to apply them to discover patterns, classify, and predict (data mining). Chapter 01 · Introduction to Machine Learning & Data Mining
- How do I choose k in k-NN?
- A common rule of thumb is k ≤ √(number of training examples), tuned with a validation set. Small k (down to k=1) fits the training data tightly but is sensitive to noise; larger k is smoother and more robust but can blur genuine class boundaries. In the exam, use the k the question states and show the votes; if asked to choose, pick the k that minimises validation error. Chapter 02 · Nearest Neighbour & Rule-Based Algorithms
- What's the difference between Ridge and Lasso?
- Both add a penalty on the coefficient sizes to the least-squares cost to fight overfitting. Ridge (L2) penalises Σwᵢ² and shrinks all coefficients smoothly toward zero without eliminating them. Lasso (L1) penalises Σ|wᵢ| and can drive some coefficients to exactly zero, so it also selects features. Larger α means stronger penalty and simpler models in both. Chapter 03 · Linear & Logistic Regression; Overfitting & Regularization
- Why is naïve Bayes called 'naïve'?
- Because it makes the simplifying (often unrealistic) assumption that the attributes are conditionally independent given the class, so their probabilities can be multiplied. It also treats all attributes as equally important. Despite the assumption it works surprisingly well, is fast (one scan of the data) and is robust to isolated noise; strongly correlated attributes are what hurt it. Chapter 04 · Naïve Bayes & Evaluating ML Methods
- Entropy or Gini — does it matter which I use?
- Both measure node impurity and usually pick similar splits; entropy (with information gain) is the ID3/C4.5 default and Gini is CART's measure. In this unit the information-gain calculation with entropy is the examinable routine, so master that; know Gini exists as CART's alternative and quote it if asked. Chapter 05 · Decision Trees & Ensembles
- What exactly is the 'kernel trick'?
- It lets an SVM draw a non-linear boundary without paying the cost of an explicit high-dimensional mapping. The optimisation only ever needs dot products between points, so you replace each dot product with a kernel K(u,v) = Φ(u)·Φ(v) that computes the mapped dot product directly — for example K(u,v) = (u·v)² corresponds to a quadratic feature map. You never form Φ, which may be huge or infinite-dimensional. Chapter 06 · Support Vector Machines & Dimensionality Reduction
- Why can't a single perceptron learn XOR?
- Because a perceptron draws a single straight decision boundary, and XOR is not linearly separable — no straight line separates its two classes. It can learn AND, OR and NAND, which are linearly separable, but XOR needs a two-layer network (a hidden layer) to bend the boundary. Chapter 07 · Neural Networks: Perceptrons & Backpropagation
- Why are CNNs better than plain dense networks for images?
- Because they exploit image structure. Convolutional filters share weights across the image (far fewer parameters than a fully connected layer) and each neuron sees only a local receptive field, so the network learns translation-tolerant local features. That makes CNNs more accurate and more efficient than a dense net on images like MNIST. Chapter 08 · Deep Neural Networks: CNNs, RNNs & Transformers
- How do I choose the number of clusters k?
- There is no single answer; common approaches are the elbow method on the SSE-versus-k curve (look for the bend), or a validation measure like the silhouette coefficient. Remember every algorithm will return k clusters even in random data, so validate that the structure is real rather than imposed. Chapter 09 · Clustering
- What makes a model 'Markov'?
- The Markov property: the future depends only on the present state, not on how you got there. So the probability of the next state is conditioned only on the current one, which is what lets you multiply one-step transition probabilities to score a whole sequence. Chapter 10 · Markov Models & Hidden Markov Models
- How is reinforcement learning different from supervised learning?
- Supervised learning trains on labelled examples with known correct outputs. Reinforcement learning has no labels — the agent interacts with an environment and learns from a scalar reward signal, discovering which actions are good by trial and error over time. The objective is a policy that maximises long-run reward, not matching given labels. Chapter 11 · Reinforcement Learning
How to study for the exam
Treat COMP4318 as a set of algorithms you must be able to run by hand, not just import. Each week, take that week's method and work one small numeric example end to end — an entropy and information-gain split, a k-NN vote, a Bayes posterior with a Gaussian density, a gradient-descent or backprop update, a Forward/Viterbi pass — because the closed-book final asks exactly that, with a non-programmable calculator.
Do the weekly homework quiz on time (no late submissions, no extensions) and use the two group assignments to lock in the pipeline and evaluation habits. Keep a one-page summary per algorithm: what it assumes, its core formula, its complexity, and the trap that costs marks (which speed goes into an average, which class is positive, log base 2 for entropy, minimise ½‖w‖² for the SVM).
For the 60% final, prioritise breadth — the 40% hurdle rewards being able to start every topic — then deepen the ones you find hardest, and keep old topics warm through STUVAC. When a step won't click, ask Sia to explain that single step a different way and set you a fresh practice question in the same style; it teaches the method and checks your reasoning, and it never does your graded assessment for you.
Confirm the exact exam date, room and permitted-materials rule on Canvas and the exam timetable.
Your AI Computer Science tutor for COMP4318
Stuck on a hard COMP4318 question? Sia is AskSia’s AI Computer Science tutor — ask any COMP4318 Machine Learning and Data Mining question and get a clear, step-by-step explanation grounded in how the course is actually taught and assessed. Read this whole study guide free, then take your hardest questions to Sia.
