COMP5318 Cheatsheet · Machine Learning and Data Mining

University of Sydney · memory aid planner for the final exam. Semester 2 formal examination period, from 16 November 2026 (the Exams Office publishes the exact date). Full COMP5318 study guide →

A4 landscape · 6 columns · memory aid planner · build your own notes 2 sides · COMP5318 · USyd · S2 2026
COMP5318
Machine Learning and Data Mining
University of Sydney · Postgraduate coursework · Semester 2, 2026
COMP5318 cheatsheet QR code – open the full COMP5318 study guide
Full guide + ask Sia about COMP5318
SIDE 1/2   WHAT EARNS SPACE · rules → formulas → assumptions → decision cues MEMORY AID PLANNER Final Exam 60% · paper-based · 2 hours

0 · Exam Mapcourse data · L13

5A1 15A2 20FINAL EXAM 60%
AssessmentWtFormat
Weekly Homework Quizzes ×105%Canvas, W2–11
Assignment 115%group program + report
Assignment 220%group program + report
Final Exam60%paper, 2 h; calculator

Permitted materials during the exam: One sheet of your own notes - double-sided, A4 size, handwritten or typed (COMP5318 lecture 13 slides)

Hurdle: score ≥40% in the final or the final course mark is capped at 45; ≥50% overall is required to pass. (course data)

Use this as a selection plan: copy only what you can explain, prioritising procedures, assumptions, denominators and traps.

> LABELS? yesno numeric → reg. group/compress class → labelNB/tree/SVM/NN cluster → Kmeancompress → PCA
Task fork: target type selects regression/classification; no labels selects clustering or PCA. Ch 1 topic map

1 · Scale, Distance, kNNCh 1–2

scaling + similaritymin–max: x′=(x−min)/(max−min)
z=(x−x̄)/s
D₂(A,B)=√Σ(aᵢ−bᵢ)² · D₁=Σ|aᵢ−bᵢ|
cos(A,B)=A·B/(‖A‖‖B‖)

Scale before distance: otherwise the largest-range feature dominates. Euclidean = straight-line; Manhattan = axis steps; cosine compares direction rather than magnitude.

kNN: distance → sort → take k → majority vote (classification) or mean (regression). Compare squared Euclidean distances to rank. Choose k by validation error; small k = low bias/high variance, large k = high bias/low variance; k=n predicts the majority.

Exam bite → Lazy learning stores all rows; one query over m rows × n features costs about m×n distance operations. Prefer an eager model for real-time prediction.

1b · Risk + Rule BaselinesCh 1–2

ItemDecision cue
empirical riskaverage loss on training data
true riskexpected unseen-data loss; estimate on held-out data
error train pick min validation overfitoversmooth
Pick k at the validation-error minimum, never at the training-error minimum. Source: knn-rule-based fig 2

2 · Linear RegressionCh 3

least squaresŷ=b₀+b₁x
b₁=[Σxy−ΣxΣy/n]/[Σx²−(Σx)²/n]
b₀=ȳ−b₁x̄ · eᵢ=yᵢ−ŷᵢ · SSE=Σeᵢ²

fit scoreSST=Σ(yᵢ−ȳ)² = SSR+SSE
R²=SSR/SST=1−SSE/SST · r=sign(b₁)√R²

Where it bites: show Σx, Σy, Σxy, Σx², then substitute; predict only after fitting. Least squares minimises squared vertical residuals, not their raw sum. Test-set R² can be negative when predictions are worse than using ȳ.

3 · Logistic + PenaltiesCh 3

classification probabilityz=w·x+b · p(y=1|x)=σ(z)=1/(1+e⁻ᶻ)
p≥0.5 ⇔ z≥0 → class 1

MethodObjective/effectUse
Linearleast-squares SSEnumeric target
Logisticmaximum likelihoodbinary target
Ridge L2+αΣwⱼ²; shrinkretain all features
Lasso L1+αΣ|wⱼ|; zerosfeature selection

Larger α → heavier penalty → simpler model. Never claim logistic regression minimises SSE.

9 · PCACh 6

Procedure: centre → covariance → eigenvectors/eigenvalues (or SVD X=UΣVᵀ) → order λ descending → project onto first k components.

variance retainedfirst k PCs = Σᵢ₌₁ᵏλᵢ / Σⱼλⱼ

PCA is unsupervised, ignores labels and yields orthogonal PCs. Kernel SVM maps upward to separate; PCA projects downward to retain variance.

9b · Regression ChecksCh 3

Two fast traps → R² is a fraction of variation explained, not the slope. Logistic p is nonlinear in x, but log-odds are linear; z=0 is the default boundary.

p=1.5z=0class 1class 0
Sigmoid threshold: p=0.5 exactly at z=0; log-odds are linear although p is not. Source: regression fig 2

4 · Naïve BayesCh 4

factor, score, argmaxP(H|E)=P(E|H)P(H)/P(E)
score(c)=P(c)∏ⱼP(xⱼ|c) · choose argmax c

Nominal feature: likelihood = class-conditional count/class count. Zero frequency: Laplace add-one smoothing stops one unseen value making the whole product zero. Numeric: use the Gaussian density with each class’s mean and sample SD.

Gaussian likelihoodp(x|c)=1/(√(2π)σc) · exp[−(x−μc)²/(2σc²)]

Assumption: features are conditionally independent given the class. Compare unnormalised class scores; P(E) is common.

5 · Honest EvaluationCh 4

ChoiceRule
Holdoutfast; train/test once
k-fold CVeach row tested once; average k scores
Stratifiedpreserve class proportions
LOOCVk=n; costly, low bias

Train weights; validation/CV tunes k, α, depth; test estimates final generalisation. Never tune on test.

metricsaccuracy=(TP+TN)/N
precision=TP/(TP+FP) · recall=TP/(TP+FN)
F1=2PR/(P+R)=2TP/(2TP+FP+FN)

Exam bite → Imbalanced classes: accuracy can hide failure. Precision answers “flagged: how many right?” Recall answers “actual positives: how many caught?”

11 · Deep Architecture ChooserCh 8

ProblemPickReason
imageCNNlocal + shared weights
short sequenceRNNhidden state
long dependencyLSTMcell + 3 gates
whole sequenceTransformerself-attention

GRU is the lighter two-gate alternative. Pooling: max keeps the strongest response; average keeps the mean.

PREDICTEDTRUTH TPFNFPTN positivenegative
Rows are truth, columns prediction; diagonal cells are correct. Build counts before metrics. Source: Naïve Bayes/evaluation fig 3

6 · Trees: Split + StopCh 5

impurityH(S)=−Σᵢpᵢlog₂pᵢ · 0log₂0:=0
Gini(S)=1−Σᵢpᵢ²

information gainGain(S,A)=H(S)−Σₖ(|Sₖ|/|S|)H(Sₖ)
choose the largest gain

Entropy: pure=0; binary 50/50=1 bit. Numeric feature: sort values, try midpoint thresholds where labels change, score each. Weight every child by its row share.

High-cardinality trap: an ID-like attribute makes singleton pure leaves and memorises training data. Drop it or use gain ratio. Pre-pruning stops early; post-pruning grows then cuts using validation performance.

7 · EnsemblesCh 5

Bagging / RFBoosting
Trainparallel, independentsequential
Databootstrap rowsreweight mistakes
Combineequal voteweighted vote
Main gain↓ variance↓ bias
Noisemore robustmore sensitive

Random forest = bagging + random feature subset at each split, decorrelating trees. Bootstrap means sample with replacement.

7b · Tree Decision ChecksCh 5

SituationResponse
numeric featuretry midpoints where class changes
deep tree overfitspre-stop or post-prune using validation
unstable high variancebag independent bootstrap trees
correlated treesrandom feature subset per RF split
persistent biassequential boosting

Extra checks: Gini is 0 at a pure node and 0.5 at binary 50/50. Gain ratio counters information gain’s bias toward many-valued splits. Bagging combines by majority vote or averaging; boosting combines weighted learners and is more noise-sensitive.

8 · SVM + KernelsCh 6

hard-margin geometryboundary: w·x+b=0
supports: w·x+b=±1
constraint: yᵢ(w·xᵢ+b)≥1
margin=2/‖w‖ ⇔ minimise ½‖w‖²

Support vector iff yᵢ(w·xᵢ+b)=1; a point strictly beyond the margin is not one. Solve the ±1 support equations: direction alone does not fix ‖w‖ or margin.

C: larger C penalises training errors more; smaller C tolerates errors for a wider/softer margin. Kernel trick: K(x,y)=Φ(x)·Φ(y) computes a feature-space inner product without constructing Φ.

common kernelspolynomial: K(x,y)=(x·y+1)ᵈ
RBF: γ controls boundary smoothness

> x₁x₂h₃h₄o₅forward → · ← backward
A 2→2→1 sigmoid net: forward activations, then backward error signals. Source: neural networks fig 2

10 · Neurons + BackpropCh 7

forwardn=Σwᵢxᵢ+b · o=σ(n)=1/(1+e⁻ⁿ)
σ′(n)=o(1−o)

backward + updateδout=(t−o)o(1−o)
δh=oh(1−oh)Σout(δout·who)
Δw=η·δreceiver·osource · Δb=ηδ

Perceptron: step output; e=t−a; w←w+ex, b←b+e. It converges only for linearly separable data; one unit cannot learn XOR.

Dense parameter count: nᵢₙnₒᵤₜ+nₒᵤₜ (one bias per receiving neuron). Sigmoid saturation gives tiny derivatives → vanishing gradients; course fixes include ReLU, better initialisation and dropout.

10b · Boundary / Update ChecksCh 6–7

ClaimCheck
SVM supportyᵢ(w·xᵢ+b)=1 exactly
outside marginstrictly >1; not a support vector
perceptron correcte=0 ⇒ no weight moves
input xᵢ=0its perceptron weight does not move
backprop edgeδ belongs to receiving unit

Show direction and scale: solving the support equations fixes w and b; guessing only the boundary direction can give the wrong margin.

K-meanshard / roundGMMsoft assignDBSCANshape/noise hierarchy → cut tree
Choose by geometry and output: hard means, soft memberships, density/noise or hierarchy. Source: clustering figs 1–2

12 · ClusteringCh 9

K-means: choose k centroids → assign nearest → replace each centroid by member mean → repeat until unchanged. Minimises within-cluster SSE; sensitive to start, scale and outliers; prefers compact spherical clusters.

GMM/EM: soft probabilistic generalisation: E-step assigns responsibilities; M-step updates parameters. Agglomerative: start singleton, merge closest clusters; single=min pair distance, complete=max, average=mean.

DBSCAN: with Eps and MinPts, core has enough neighbours, border is reachable from a core but not core, noise is neither. Finds arbitrary shapes/noise; varying densities are hard.

qualitySSE=ΣclustersΣx∈C‖x−μC‖²
silhouette s=(b−a)/max(a,b)

13 · HMM + Reinforcement LearningCh 10–11

Markov / HMMP(π₁…πK)=P(π₁)∏ᵢP(πᵢ₊₁|πᵢ)
fₖ(1)=A₀(k)eₖ(x₁)
fₖ(t)=eₖ(xₜ)Σⱼ[fⱼ(t−1)aⱼₖ]
P(X)=Σₖfₖ(T)

Viterbi: same trellis but replace Σ with max, store back-pointers, trace argmax. Rows of A are from, columns to; each row sums to 1.

return + Bellman/QGₜ=Σₖ≥0γᵏrₜ₊ₖ₊₁
V*(s)=maxₐΣs′P(s′|s,a)[r+γV*(s′)]
Q←Q+α[r+γmaxₐ′Q(s′,a′)−Q]

ε-greedy: explore with ε; otherwise exploit argmax Q. Terminal next state: max term=0. Experience replay stores (s,a,r,s′), samples random minibatches, breaks sequential correlation and reuses experience.

Exam split: Forward evaluates P(observations) by summing every hidden path; Viterbi decodes one most likely path by max plus back-pointers; Baum–Welch/EM learns A, E and A₀. In an MDP, (S,A,R,P,γ) names states, actions, reward, transitions and discount.

Your exam allows your own A4 notes only: build your own sheet from this plan; do not bring this printout.
asksia.aiForward this to your tutorial group · Shared by ________
https://www.asksia.ai/cheatsheet/usyd-comp5318 · Full COMP5318 study guide →
v2 · 2026-10-16 · Semester 2 formal examination period, from 16 November 2026 (the Exams Office publishes the exact date).
COMP5318
Machine Learning and Data Mining · Worked Drills
Reproduce each pipeline unaided · then diagnose the first wrong line
COMP5318 cheatsheet QR code – open the full COMP5318 study guide
Full guide + ask Sia about COMP5318
SIDE 2/2   FIVE WORKED DRILLS · traps · fast recall · method chooser DO → CHECK → EXPLAIN Keep the guide’s numbers · carry every calculation to the answer
e=y−ŷminimise Σe²
Least squares minimises squared vertical residuals; show the residual column before R². Source: regression fig 1

Drill 1 · Fit + ScoreCh 3 worked example

Given: (x,y)=(1,1),(2,3),(3,2),(4,4); n=4. Fit ŷ=b₀+b₁x, predict x=5, compute R².

  1. Σx=10, Σy=10 ⇒ x̄=ȳ=2.5; Σxy=29; Σx²=30.
  2. b₁=[29−(10·10)/4]/[30−100/4]=4/5=0.8.
  3. b₀=2.5−0.8(2.5)=0.5; ŷ=0.5+0.8x.
  4. x=5 ⇒ ŷ=4.5.
  5. ŷ: 1.3,2.1,2.9,3.7; e: −0.3,0.9,−0.9,0.3; SSE=.09+.81+.81+.09=1.80.
  6. SST=(−1.5)²+.5²+(−.5)²+1.5²=5.00; R²=1−1.8/5=0.64. Thus r=+√.64=+0.8.

Check → SSR=5−1.8=3.2; SSR/SST=.64. A bare answer loses the auditable method trail.

Fast Recall · Distance + kNNCh 1–2

PromptWrite first
normalise(x−min)/(max−min), not x/max
rank Euclideancompare squared distances
classifyscale → distance → sort → k → vote
select kvalidation minimum; k≤√n is a rule of thumb

Foundation ChecksCh 1–2

StatementVerdict + reason
lowest train error is bestfalse; estimate true risk on unseen data
unlabelled grouping predicts a classfalse; clustering discovers structure
cosine tracks magnitudefalse; it compares vector direction
k=1 is always bestfalse; low bias, high variance
kNN has costly trainingfalse; lazy training, costly prediction

1R builds one rule from the single best attribute. PRISM learns a covering set of IF–THEN rules. They are baselines, not distance methods.

6+ / 4− · H=.971 Region G=.125Member G=.610 4+/1−2+/3−5+/0−1+/4−
Member wins: its pure child drives weighted entropy down and gain up. Source: Ch 5 worked example

Drill 2 · Entropy + GainCh 5 worked example

Given: parent 6 Buy/4 No-buy. Region → North 4+/1−, South 2+/3−. Member → yes 5+/0−, no 1+/4−.

  1. H(parent)=−.6log₂.6−.4log₂.4=.6(.737)+.4(1.322)=.971 bits.
  2. H(4/1)=−.8log₂.8−.2log₂.2=.722; H(2/3)=H(.4/.6)=.971.
  3. Weighted Region=.5(.722)+.5(.971)=.846; Gain=.971−.846=.125.
  4. Member yes is pure ⇒ H=0; Member no (1/4) ⇒ H=.722.
  5. Weighted Member=.5(0)+.5(.722)=.361; Gain=.971−.361=.610.
  6. Choose Member: .610>.125.

Check → Memorise anchors: pure=0; 50/50=1; 80/20≈.722; 60/40≈.971. The child average must be weighted.

Tree / Ensemble TrapsCh 5 FAQ

Wrong moveCorrection
plain mean of child Hweight by |Sₖ|/|S|
ID gives perfect gainmemorisation; drop/use gain ratio
RF = boostingRF is bagging + feature subsets
bag without replacementbootstrap is with replacement

Split / Prune RecallCh 5 topic map

Numeric split: sort values; place candidate thresholds at midpoints where the class label changes; compute impurity reduction for each. A feature may be reused deeper with another threshold.

Pre-pruning halts growth by a limit such as depth or node size. Post-pruning grows the tree, then removes branches whose removal improves validation behaviour.

Sanity checks: gain cannot exceed parent entropy; pure children contribute zero; a split with unchanged class proportions has zero gain.

> q·k: 8,4÷2: 4,2softmax.88,.12 .88(1,0)+.12(0,1) z₁=(.88,.12)
Attention’s four lines: dot → scale → softmax → weighted values. Source: deep learning fig 4

Drill 3 · Self-AttentionCh 8 worked example

Given: q₁=(2,0,2,0); k₁=(2,1,2,1), k₂=(1,1,1,1); v₁=(1,0), v₂=(0,1); dₖ=4.

  1. Scores: q₁·k₁=2·2+0·1+2·2+0·1=8; q₁·k₂=2·1+0·1+2·1+0·1=4.
  2. Scale by √dₖ=√4=2 ⇒ scores 4, 2.
  3. Softmax: e⁴=54.60, e²=7.39; sum=61.99 ⇒ weights .88, .12.
  4. z₁=.88(1,0)+.12(0,1)=(.88,.12).

Interpret: token 1 contributes 88% because its key matches q₁ more strongly. Weights sum to 1; output dimension equals the value dimension.

Check → Scale before softmax. Softmax on raw 8,4 is the classic lost line.

Neural / Deep Fast RecallCh 7–8

PromptMark-winning phrase
why CNN?local receptive fields + shared weights + translation invariance
RNN weaknessproducts of derivatives → vanishing/exploding gradient
why LSTM?cell state; forget/input/output gates preserve long context
parametersincoming weights + one bias per receiving unit

Backprop / Convolution ChecksCh 7–8

backprop orderforward hidden → forward output → δout → update hidden→output → δhidden → update input→hidden

For sigmoid, derivative is o(1−o), using the neuron’s already-computed output. Each weight change is η × receiver delta × source output; bias uses source 1.

Convolution: multiply each image patch elementwise by the shared filter and sum. With a 3×3 input, 2×2 filter, stride 1 and no padding, the guide’s map is 2×2 [[2,4],[4,2]]; max pool gives 4, average pool gives 3.

Transformer check: positional encoding adds order because attention is order-agnostic. Multi-head attention uses distinct Q/K/V projections of one embedding, then concatenates the outputs.

C₁ μ=(1.33,1.33)C₂ μ=(6.5,5)SSE=1.83
Means need not be data points; recompute, reassign, stop only when membership is unchanged. Source: clustering fig 1

Drill 4 · K-MeansCh 9 worked example

Given: P₁(1,1), P₂(1,2), P₃(6,5), P₄(7,5), P₅(2,1); k=2; initial c₁=(1,1), c₂=(6,5); squared Euclidean.

  1. Distance pairs d²(c₁),d²(c₂): P₁ 0,41; P₂ 1,34; P₃ 41,0; P₄ 52,1; P₅ 1,32.
  2. C₁={P₁,P₂,P₅}; C₂={P₃,P₄}.
  3. Means: c₁=((1+1+2)/3,(1+2+1)/3)=(4/3,4/3)≈(1.33,1.33); c₂=((6+7)/2,(5+5)/2)=(6.5,5).
  4. Epoch 2: no reassignment. Example P₂: d²(c₁)=.11+.44=.56 vs d²(c₂)=30.25+9=39.25 ⇒ converged.
  5. SSE C₁=.22+.56+.56=1.33; C₂=.25+.25=.50 ⇒ SSE=1.83.

Check → Compare squared distances for assignment; take square roots only when an actual distance is requested.

Clustering TrapsCh 9

TrapFix
centroid = real rowcentroid is mean; medoid is real point
K-means finds noiseDBSCAN labels noise
single linkageminimum pair distance
complete linkagemaximum pair distance
silhouette near 1good cohesion/separation

Cluster Family ChooserCh 9 topic map

Need / geometryMethodTrade-off
known k, compact groupsK-meansfast; start/outlier sensitive
soft membershipGMM + EMprobabilistic, more parameters
nested groupingagglomerativedendrogram; linkage matters
arbitrary shape + noiseDBSCANEps/MinPts; varied density hard

Silhouette: a = mean distance within own cluster; b = best alternative-cluster mean distance. Near +1 is well placed, near 0 is on a boundary, negative suggests a better neighbouring cluster.

max Q′=10target=14error=12step=6 Qnew=2+.5(12)=8 α=.5 · γ=.9 · r=5 · Qold=2
Keep target, TD error and α-scaled step as separate lines. Source: Ch 11 worked example

Drill 5 · Q-LearningCh 11 worked example

Given: α=.5, γ=.9, Q(s,a)=2, r=5; next-state Q values are 4,10,6.

  1. Write Q←Q+α[r+γmaxₐ′Q(s′,a′)−Q].
  2. Best next value=max(4,10,6)=10.
  3. TD target=5+.9(10)=14.
  4. TD error=14−2=12.
  5. Qnew=2+.5(12)=8.

Interpret: α=.5 moves the estimate halfway from 2 toward 14. If s′ is terminal, the max-next term is 0 and target=r.

Check → γ multiplies the maximum next Q, not a sum/mean. α multiplies the entire TD error, not the target alone.

HMM Recall DrillCh 10 · source fig 2

Forward trellis values for Write, Browse: t=1 gives F=.48, D=.12; t=2 gives F=.0768, D=.1512; therefore P(X)=.228.

Question asksOperatorFinish
P(observations)Forward: Σ incomingΣ last column
most likely statesViterbi: max incomingargmax + backtrack
sequence probabilityinitial × transitionsK states ⇒ K−1 transitions

Do not reverse aⱼₖ: row=from, column=to. Multiply the current observation’s emission once, outside the incoming sum/max.

Sequence / RL ChecksCh 10–11

Complexity: Forward/Viterbi dynamic programming is O(N²T), versus enumerating O(Nᵀ) paths. HMM parts are hidden states, transition A, emission E and initial distribution A₀.

Return: γ near 0 is myopic; γ near 1 values distant rewards. ε-greedy: random action with probability ε, otherwise current argmax. The action changes the next state, unlike a fixed supervised dataset.

Deep Q: a network approximates Q; replay memory stores (s,a,r,s′) and random minibatches break successive-transition correlation while reusing experience.

> F .48D .12F .0768D .1512 WriteBrowse
Forward sums all incoming paths; Viterbi selects one and records its predecessor. Source: HMM fig 2

Final-Pass Method Chooserall topic maps

If the prompt says…Reach for…First line
closest labelled pointskNNscale + distance
numeric predictionlinear regressionŷ=b₀+b₁x
class probabilitylogisticσ(w·x+b)
priors + likelihoodsNaïve BayesP(c)∏P(xⱼ|c)
best splittreeparent H−weighted child H
widest boundarySVMw·x+b=0; supports ±1
variance compressionPCAcentre → covariance/SVD
small neural updatebackpropforward → δout → δhidden
imageCNNfilter → ReLU → pool
token interactionsattentionQKᵀ/√dₖ → softmax → V
unlabelled groupsclustergeometry/noise selects family
P(observation seq.)ForwardΣ incoming × emission
hidden pathViterbimax incoming + pointers
rewarded action updateQ-learningtarget → error → α step

Common Error RadarFAQs + study strategies

  • Data leakage: tuning on test or scaling with information from test.
  • Wrong denominator: precision uses predicted positives; recall uses actual positives.
  • Wrong objective: logistic ≠ least squares; PCA ≠ classifier; clustering has no labels.
  • Wrong operator: NB multiplies likelihoods; Forward sums paths; Viterbi and Q-learning take maxima.
  • Missing assumption: scale for distance, conditional independence for NB, linearly separable for hard-margin SVM/perceptron convergence.
  • Missing reason: after the number, write one sentence interpreting the model choice or failure mode.

Last check → Formula → substituted values → result → one-line reason. That four-part trace makes partial credit visible.

Your exam allows your own A4 notes only: build your own sheet from this plan; do not bring this printout.
asksia.aiForward this to your tutorial group · Shared by ________
https://www.asksia.ai/cheatsheet/usyd-comp5318 · Full COMP5318 study guide →
v2 · 2026-10-16 · Semester 2 formal examination period, from 16 November 2026 (the Exams Office publishes the exact date).

Want one for YOUR exact syllabus?

Sia is your free desktop study agent. Drop your University of Sydney COMP5318 slides — Sia builds a sheet tailored to YOUR exam. Better than this library because it knows YOUR materials.

↓ Download Sia · Free