MAIE5421 Computer Vision
MAIE5421 Overview
- HKUST
- Fall 2026-27
- Postgraduate
- Computer Science
- Coursework and team project
What this course asks you to be able to do
MAIE5421 Computer Vision is taught at HKUST together with ARIN5201 Machine Learning for Computer Vision, and the two codes share one set of lectures. The course opens with a block called Machine Learning and Computer Vision Basics and then moves to Advanced Topics in Computer Vision.
- Assessed by Attendance, written assignments and a three-part team project made of a proposal, a presentation and a technical report.
- Hardest step Carrying a calculation such as entropy, a least-squares fit or a parameter count through every line without a slip.
- How to prepare Work each method by hand on a small example, then reproduce it in code with the tensor shapes written beside every line.
- Where to confirm Weights, deadlines and submission settings are controlled by the course Canvas page.
How MAIE5421 is assessed
| Component | Weight | Format |
|---|---|---|
| Attendance | 10% | Attendance at the scheduled weekly class |
| Assignments | 30% | Written assignments; Assignment 1 is individual work submitted as one PDF file, with LaTeX recommended |
| Final Project Proposal | 10% | One proposal per team of five to six students, submitted by a designated member |
| Final Project Presentation | 20% | Presentation of the final project; the lecture topic table places presentations in weeks 12 and 13 |
| Final Project Technical Report | 30% | Written technical report on the final project |
The course information page lists these five components and they total 100%. The final project accounts for 60% across three parts, assignments for 30% and attendance for 10%. The materials available here do not say how the 30% is divided between assignments; the first assignment page shows it is individual work submitted as a single PDF. The proposal page treats extensions as exceptional, requires the instructor's agreement to be secured before the deadline, and warns that late work handed in without that agreement may be refused. No hurdle, pass mark or grading scale is published in the course materials available here, so the hurdle column records that the condition is not stated. Check the course Canvas page for anything that governs whether you pass, and for the current split and deadlines.
Assessment structure
Segment widths reproduce the five published percentage weights, which total 100%. The proposal, presentation and technical report together make up the 60% final project.
Current MAIE5421 dates
| Date | Item | Control |
|---|---|---|
| 17 October 2026 | Final Project Proposal due | Due 11:59 pm, one submission per team, shown on the proposal page and the course summary for this offering. |
| 18 October 2026 | Assignment 1 due | Due 11:30 pm as a single PDF file, shown on the assignment page and the course summary for this offering. |
| Weeks 12 and 13 | Final Project Presentation | The topic table in the lecture slides places the final project presentations in these weeks; confirm dates on Canvas. |
Dates are as published in the assignment pages, the course summary and the lecture topic table for this offering. Confirm exact deadlines and submission settings in the live LMS.
What MAIE5421 covers
Ten chapters follow the machine learning elements and deep learning lectures, from the learning workflow and regression through classification metrics, classic classifiers, clustering and dimensionality reduction to neural network training, deep architectures and the PyTorch pipeline.
Machine Learning Foundations and the Learning Workflow
rule-based against data-driven systems · AI, machine learning and deep learning · four learning types · samples and the training, validation and test split · overfitting and underfittingTwo ways to make a machine behave intelligently
The course frames its subject with a choice.
- Rule-based systems against data-driven learning
- How artificial intelligence, machine learning and deep learning nest
- A brief history from the perceptron to foundation models
- Regression and classification as the two core tasks
- Data samples, datasets and the training, validation and test split
- Supervised, unsupervised, semi-supervised and reinforcement learning
Linear, Polynomial and Regularised Regression
simple linear regression by ordinary least squares · the matrix form for several features · polynomial degree and overfitting · ridge and lasso penalties and the tuning parameterFrom one input to many
The regression lecture works through a family of models in increasing flexibility.
- Regression with single and multi-dimensional inputs and outputs
- Simple linear regression and ordinary least squares
- Multiple linear regression in matrix form
- Polynomial regression and the choice of degree
- Overfitting and the role of large coefficients
- Ridge regression and the L2 penalty
Classification Metrics and the Confusion Matrix
true and false positives and negatives · accuracy, precision, recall and F1 · imbalanced classes · per-class metrics for multi-class and multi-label problemsThree shapes of classification problem
Classification learns from input-output pairs whose outputs are categories.
- Binary, multi-class and multi-label classification
- True positives, false positives, false negatives and true negatives
- Reading the axes of a confusion matrix
- Accuracy and why it misleads on imbalanced data
- Precision when false alarms are costly
- Recall when missed detections are costly
Logistic Regression, k-NN and Naive Bayes
the logistic curve and its parameters · nearest neighbours, the choice of k and distance measures · Bayes' theorem with prior, likelihood and posterior · the naive independence assumptionLogistic regression: a linear score made into a probability
Logistic regression is the lecture's first classifier for a binary target.
- Five typical supervised classifiers
- The logistic function, its location and scale
- Maximum likelihood rather than least squares
- The k-NN algorithm and majority voting
- Choosing k and the effect of smoothing
- Euclidean, Manhattan and cosine measures
Decision Trees and Support Vector Machines
entropy and information gain in ID3 · gain ratio and continuous thresholds in C4.5 · the maximum-margin hyperplane and support vectors · slack variables, feature mapping and multi-class strategiesDecision trees learn the questions to ask
The lecture introduces decision trees by comparison with hand-written if-else code: a tree performs classification through a hierarchy of decisions, but it builds those decisions from data.
- Decision trees as learned if-else rules
- Root nodes, branches and leaves
- Entropy as a measure of disorder
- Information gain and the ID3 algorithm
- Gain ratio and split information in C4.5
- Thresholds for continuous attributes
Clustering and Association Rule Learning
compactness, separation, inertia and the Dunn index · K-Means iterations and the elbow curve · agglomerative and divisive hierarchies · normalisation · support, confidence, lift and AprioriLearning without labels
Unsupervised learning finds patterns in data that carries no labels.
- Clustering, association rules and dimensionality reduction
- Compactness, separation, inertia and the Dunn index
- The K-Means assignment and update loop
- Stopping criteria and the elbow curve for choosing k
- Agglomerative and divisive hierarchical clustering
- Single, complete, average and centroid linkage
Dimensionality Reduction with PCA and LDA
redundant features and variance by direction · centring, covariance and eigenvalue decomposition · choosing the number of components · within-class and between-class scatter in LDAWhy reduce dimensions at all
Feature sets built by hand usually contain redundancy.
- Redundant and correlated features
- Variance by direction as a measure of expressiveness
- The five ideas behind PCA
- Centring, covariance and eigenvalue decomposition
- Choosing K from accumulated eigenvalues
- Projecting data onto principal components
Training a Multilayer Perceptron
perceptron weights, bias and activation · sigmoid, ReLU and softmax · forward propagation · mean squared error and cross-entropy · batches, iterations and epochs · gradient descent and dropoutA journey through one simple model
The deep learning lecture teaches training through a single task, classifying MNIST handwritten digits, and a single model family, the multilayer perceptron.
- The digit classification task and its input and output
- Perceptron weights, bias and activation
- Building a multilayer perceptron from fully connected layers
- Sigmoid, ReLU and softmax activations
- Forward propagation and model structure design
- Mean squared error, cross-entropy and binary cross-entropy
Deep Architectures for Images and Sequences
gradient vanishing and the residual block · convolution kernels, stride, channels and pooling · recurrent state and LSTM gates · attention in the Transformer · pretraining and fine-tuningWhy images need more than fully connected layers
The lecture motivates convolutional networks with two weaknesses of the multilayer perceptron on images.
- Why flat, fully connected networks suit images poorly
- Convolution kernels, kernel size, stride and channels
- Max and average pooling and the receptive field
- The ImageNet error record from 2010 to 2015
- Gradient vanishing in very deep networks
- Residual blocks and skip connections
Building a PyTorch Training Pipeline
Dataset and DataLoader classes · Sequential and Functional model definitions · layer input and output shapes · loss functions and optimisers · the training loop, validation and predictionSix components of a PyTorch pipeline
The lecture's PyTorch walkthrough uses MNIST digit recognition to build a complete pipeline in six parts: load the data, define the model, train it, evaluate it, tie the parts together and make predictions.
- The six components of a PyTorch pipeline
- Custom Dataset classes with init, len and getitem
- DataLoader batching, shuffling and random_split
- Sequential and Functional model definitions
- Input and output shapes of common layers
- Ready-made and custom loss functions
The foundations block is where the quantitative groundwork is laid: how a model is fitted to data, how a classifier is judged, how unlabelled data is grouped and compressed, and how a deep network is trained with gradients. The later block applies that groundwork to image classification, detection, segmentation, attention, generative models and multi-modal learning.
Assessment is entirely coursework, so the habits that matter are setting a problem up correctly and showing each step.
Machine Learning Foundations and the Learning Workflow
The opening chapter separates systems that follow hand-written rules from systems that learn their behaviour from labelled examples, and places deep learning as the innermost of three nested ideas.
It sets out the four learning types by what the data carries, writes a sample as an attribute vector paired with a target, and explains why a dataset is cut into three parts with different jobs. The diagnosis of overfitting and underfitting from a pair of error rates closes it.
Linear, Polynomial and Regularised Regression
Regression predicts a continuous quantity.
The chapter fits a straight line by ordinary least squares with every sum written out, generalises it to several features through the normal equations, and shows how adding powers of the input lets the fitted curve bend at the risk of chasing noise.
Ridge and lasso then add a penalty on the size of the coefficients, and the chapter shows numerically why the first shrinks every weight while the second can set some of them exactly to zero.
Classification Metrics and the Confusion Matrix
A classifier makes two kinds of mistake, and the confusion matrix counts each one separately.
From its four cells the chapter builds accuracy, precision, recall and the F1 score, then shows on a rare-defect example how a model that never raises an alarm can still post an impressive accuracy. Per-class matrices extend the same reasoning to problems with many classes or many labels per image.
Logistic Regression, k-NN and Naive Bayes
Three classifiers that reason in very different ways are set side by side.
Logistic regression squeezes a linear score into a probability with an S-shaped curve. Nearest neighbours stores the training set and votes among the closest samples, so the distance measure and the value of k decide everything.
Naive Bayes turns a prior and a likelihood into a posterior, and the chapter works a screening example in which a sensitive detector still produces many false alarms.
Decision Trees and Support Vector Machines
Trees choose questions. The chapter computes entropy and information gain by hand to choose the first split, normalises it into a gain ratio, and finds a threshold for a continuous attribute from midpoints.
Support vector machines choose a boundary instead, the hyperplane with the widest margin, and the chapter covers support vectors, slack for points inside the margin, mapping to a higher-dimensional space and the two ways of handling many classes.
Clustering and Association Rule Learning
Without labels, structure has to be found.
The chapter measures cluster quality by compactness and separation, runs K-Means to convergence on a small set of points, builds a dendrogram by merging the closest clusters, and explains why features on different scales must be normalised before any distance is computed.
Association rules close the chapter, with support, confidence and lift computed on a set of image-tag transactions and the Apriori pruning rule applied level by level.
Dimensionality Reduction with PCA and LDA
Many features carry overlapping information.
Principal component analysis finds orthogonal directions ordered by how much variance the data has along each, and the chapter carries a two-feature example through centring, the covariance matrix, its eigenvalues and the projection.
Linear discriminant analysis uses class labels to choose directions that separate classes rather than directions that merely spread the data.
Training a Multilayer Perceptron
The deep learning lecture builds a digit classifier one decision at a time, and so does this chapter: the perceptron and its weights, the activation that makes stacking layers worthwhile, the forward pass, the loss that scores a prediction, and the gradient step that improves the weights.
Batches, iterations and epochs are counted precisely, and dropout appears as the network's own defence against overfitting.
Deep Architectures for Images and Sequences
Fully connected networks ignore the layout of an image and grow enormous.
The chapter shows how convolution shares a small kernel across positions, how pooling shrinks feature maps, why very deep networks need the shortcut paths of a residual block, how recurrent networks carry a state along a sequence, what attention adds, and how a model pretrained on a large dataset gives a smaller task a head start.
Building a PyTorch Training Pipeline
The final chapter turns the theory into code in the order the lecture uses: define a dataset class with its three required methods, wrap it in a loader that delivers shuffled batches, define the model in either of two styles, pick a loss and an optimiser, and write the loop that zeroes gradients, steps the weights and keeps the best model by validation loss.
Tensor shapes are tracked at every stage.
How to use this guide
Read the front matter for the assessment structure, then take the chapters in order, because each later method reuses an idea from an earlier one. Write your own answer to every practice item before reading the model answer.
The mixed problem set near the end draws questions from every chapter, and the closing chapter is a checklist for the days you submit assignments and project work.
Worked example · free
Read a defect detector's confusion matrix before trusting its accuracy
- 2Fill the four cells: TP = 40, FP = 20, FN = 10, TN = 930.
- 3Accuracy = (40 + 930) / 1,000 = 0.97; precision = 40 / 60 = 0.667; recall = 40 / 50 = 0.80.
- 2F1 = 2PR / (P + R) = 2 × 0.667 × 0.80 / 1.467 = 0.727.
- 1A detector that never flags scores (0 + 950) / 1,000 = 0.95 accuracy with zero recall.
Key terms
- Supervised Learning
- Learning from training data in which every input is paired with its label, so that the model can predict labels for new, unseen inputs.
- Validation Set
- The part of a dataset used to compare designs and tune settings during development, kept separate from both the data the model is fitted on and the final test data.
- Overfitting
- The condition in which a model performs very well on its training data but poorly on new data, usually because it is too complex or the training set is too small.
- Ordinary Least Squares
- The fitting method that chooses regression coefficients to minimise the sum of squared differences between observed and predicted values.
- Lasso Regression
- Linear regression with a penalty on the sum of absolute coefficient values, which tends to set the weights of unhelpful features exactly to zero.
- Precision
- The share of samples predicted positive that really are positive, the metric to watch when false alarms are costly.
- Recall
- The share of truly positive samples that the model detects, the metric to watch when a missed case is costly.
- Information Gain
- The fall in entropy produced by splitting a dataset on one attribute, used by the ID3 algorithm to choose which question a tree asks first.
- Support Vector
- A training point lying closest to a support vector machine's separating hyperplane; these points alone determine where the maximum-margin boundary sits.
- Principal Component
- A direction in feature space, found as an eigenvector of the covariance matrix, along which the data varies; components are ordered by the variance they capture.
- Cross-Entropy Loss
- The classification loss that equals the negative logarithm of the probability the model gives the true class, so a confident wrong prediction is penalised heavily.
- Residual Block
- A network block whose output adds its input back through a skip connection, so the layers learn only the difference and gradients reach early layers in very deep networks.
- Transfer Learning
- Pretraining a model on a large, generic dataset and then fine-tuning it on a smaller dataset for the specific task, so the new task starts from useful weights.
MAIE5421 FAQ
How is this course assessed?
Through five components that the course information page says total 100%. Attendance is worth 10%, assignments 30%, the final project proposal 10%, the final project presentation 20% and the final project technical report 30%. Three of the five belong to one team project, so a single piece of work carries 60% of the grade and is built up over several weeks rather than in one sitting.
Why do the lecture slides carry two course codes?
They are co-listed. The course site describes the class as MAIE5421 Computer Vision listed together with ARIN5201 Machine Learning for Computer Vision, and the lecture decks carry both codes on their title slides. Students registered under either code attend the same lectures and see the same materials, so a classmate's notes from the other code describe the same content you are taught.
What does Assignment 1 require?
Answers to the questions in the assignment instructions, submitted as a single PDF file through the course site. The teaching team recommends preparing the submission in LaTeX, which tells you the answers are expected to contain typeset mathematics. The page states that it is an individual assignment and that each student must submit their own work, so share understanding with classmates but write every line yourself.
How do teams work for the final project?
Students form teams of five to six through the groups area of the course site, using self sign-up, and everyone in a team must join the same group. Only one proposal is submitted per team, by a member the team designates. The proposal follows the requirements of the chosen topic in the project guidelines, which spell out what goes in and comes out, the core functions, the results expected and how the work will be judged.
Can I get an extension on a project deadline?
Only in exceptional cases, according to the proposal page. A team that cannot meet a deadline for a serious reason has to reach the instructor and secure agreement before the deadline passes, and work handed in late without that agreement may be refused. Plan around the published deadline rather than hoping for flexibility, and raise any problem before the due date rather than after it.
How much mathematics does the course use?
A fair amount of applied mathematics. The machine learning lecture derives least-squares solutions with matrix algebra, computes entropy with base-two logarithms, applies Bayes' theorem with Gaussian densities and performs an eigenvalue decomposition for PCA. The deep learning lecture adds gradients and loss functions.
Comfort with vectors, matrices, derivatives and basic probability is what lets you follow the derivations rather than memorise them.
Which programming framework does the course teach?
PyTorch. The deep learning lecture introduces it as a free, open-source Python library for building and training networks and walks through a full pipeline for handwritten digit recognition: a custom dataset class, a data loader, a model defined in either the sequential or the functional style, a loss, an optimiser and a training loop. A notebook bundle for running the code in Colab is posted with the lecture.
Questions students ask in each MAIE5421 chapter
- Is face recognition a regression or a classification problem?
- When the system must say which of a known set of people is in the picture, it is classification, because the output is one category from a finite list; the lecture lists face recognition among multi-class problems. Face verification, deciding whether two photographs show the same person, is a binary classification. Estimating a continuous attribute from a face, such as age in years, would be regression instead. Chapter 01 · Machine Learning Foundations and the Learning Workflow
- When should I use ridge rather than lasso?
- Use ridge when you expect most features to carry some signal and you want to stop any of them dominating, because it shrinks every coefficient while keeping all of them. Use lasso when you suspect several features are irrelevant, because its L1 penalty can set coefficients exactly to zero and so performs feature selection. In both cases, choose the strength of the penalty by cross-validation. Chapter 02 · Linear, Polynomial and Regularised Regression
- Why is accuracy misleading on imbalanced data?
- Because a model can reach a high accuracy by always predicting the majority class. If 95 of every 100 images are clean, a model that never flags anything is 95% accurate and finds no defects at all. Precision, recall and F1 look at the positive class directly, so they expose a model that ignores it, which is why the lecture prefers them when classes are uneven. Chapter 03 · Classification Metrics and the Confusion Matrix
- Why can logistic regression not be fitted by least squares?
- Because its output is a probability passed through a non-linear curve and its targets are class labels, so squared error is not the natural measure of fit and does not give a convenient closed-form solution. The lecture states that maximum likelihood estimation is used instead, choosing the parameters under which the observed labels are most probable, and leaves the details to advanced courses. Chapter 04 · Logistic Regression, k-NN and Naive Bayes
- What is the difference between information gain and gain ratio?
- Information gain measures how much splitting on an attribute reduces entropy. It is biased towards attributes with many distinct values, because splitting into many small subsets makes each look pure. Gain ratio, used by C4.5, divides the gain by the attribute's split information, which grows with the number of values, and so removes much of that bias. Chapter 05 · Decision Trees and Support Vector Machines
- How do I choose k for K-Means?
- Run K-Means for a range of k and plot an evaluation metric such as inertia against k. The metric keeps improving as k grows, so look for the elbow, the point after which the improvement becomes small. The lecture warns that too small a k underfits by merging real groups, while too large a k overfits by giving noise and outliers clusters of their own. Chapter 06 · Clustering and Association Rule Learning
- What is the difference between PCA and LDA?
- PCA is unsupervised and keeps the directions along which the data varies most, regardless of class. LDA is supervised and keeps the directions that best separate the classes relative to their internal spread. PCA can keep any number of components up to the original dimension, while LDA can keep at most one fewer than the number of classes. Chapter 07 · Dimensionality Reduction with PCA and LDA
- What is the difference between an iteration and an epoch?
- An iteration is one update of the model using one batch of training data. An epoch is one full pass through every batch in the training set. With 50,000 training images and a batch size of 100, one epoch contains 500 iterations, and training for 20 epochs performs 10,000 weight updates in total. Chapter 08 · Training a Multilayer Perceptron
- Why do convolutional networks need fewer parameters than fully connected ones?
- Because a convolution kernel is reused at every position of the image. A 3 by 3 kernel has nine weights per input channel however large the image is, while a fully connected layer needs a separate weight for every pair of input and output units. Pooling then shrinks the feature maps, so the final fully connected layer sees far fewer values than the raw pixels. Chapter 09 · Deep Architectures for Images and Sequences
- What do the three methods of a PyTorch Dataset do?
- The init method sets up whatever the other methods need, such as the file path, the number of samples and a list of indices. The len method returns the number of samples, which tells the loader when an epoch ends. The getitem method takes an index and returns one sample, typically the input features and the label, and it is where any per-sample conversion or normalisation happens. Chapter 10 · Building a PyTorch Training Pipeline
How to prepare for the assessments
Treat every method in this course as two exercises: one on paper and one in code. On paper, take a dataset small enough to hold in your head, five points for a regression line, eight samples for an entropy split, six transactions for Apriori, and carry the calculation to the end, writing the formula before each substitution. Then rebuild the same calculation in code and check that the numbers agree.
When they disagree, the disagreement is usually a convention, a variance divided by n rather than n minus one, a logarithm in the wrong base, a loss that applies softmax for you, and finding it teaches you more than the lecture slide did. For the deep learning chapters, write the tensor shape after every layer before you run anything; a model whose shapes you can predict is a model you understand.
Keep a running list of every metric your project could report and the error each one punishes, so that the proposal names its evaluation criterion before any result exists.
Your AI Computer Science tutor for MAIE5421
Stuck on a hard MAIE5421 question? Sia is AskSia’s AI Computer Science tutor — ask any MAIE5421 Computer Vision question and get a clear, step-by-step explanation grounded in how the course is actually taught and assessed. Read this whole study guide free, then take your hardest questions to Sia.