MATH7005 Data Analytics with Programming

How MATH7005 is assessed

The official course surface publishes these two components, and their numeric weights total 100%. The available materials say detailed marking criteria will be released later. Confirm the released brief, current submission settings and the precise late-November project schedule on the course LMS.

Current MATH7005 dates

Dates are as published in the captured official course assessment surface for the current offering. Confirm exact deadlines and submission settings in the live LMS.

What MATH7005 covers

The guide moves from classification outputs and losses through optimisation, network architecture and gradient flow to convolution, attention and efficient Transformer methods.

The captured course surface points students toward Hong Kong Government open datasets and asks them to frame a useful context, analyse data, incorporate coding into a final report and demonstrate a finished product live. The guide therefore treats every mathematical method as part of an evidence chain.

A model output matters only when the data, preprocessing, validation, threshold and resulting product action can all be explained.

How the assessment is organised

The published structure has a short assignment and a much larger project. The assignment is a three-minute voice-over video pitch designed to establish the context for the proposed data product.

The project includes a final report with coding, a ten-minute live demonstration with no slides and an approximately twenty-minute question-and-answer session. The available materials say that detailed criteria will be released later, so this guide does not invent a University rubric.

It supplies technical checks and rehearsal structures that make the work defensible when the current brief appears.

Logistic models and sigmoid scores

The first chapter separates a linear score, a sigmoid output and a threshold decision. A coefficient moves log-odds rather than probability directly, and a threshold belongs to an operational policy rather than the fitted model.

Worked calculations show how signs, intercepts and feature scale affect a prediction. Numerical stability is included because an algebraically correct sigmoid may overflow when implemented naively for an extreme score.

Cross-entropy and likelihood

Cross-entropy reads the probability assigned to the observed class.

The chapter expands the binary formula into its positive-label and negative-label cases, then joins the summed loss to maximum likelihood through the negative logarithm.

It distinguishes a sum from a mean and explains why confidence matters: two models can have the same hard-class accuracy while assigning very different probability to the outcomes that occurred.

Softmax and multiclass prediction

Softmax makes several class logits compete through a shared denominator.

The chapter uses maximum subtraction for stable computation, checks that probabilities sum to one across the class axis and connects categorical cross-entropy to the probability at the true-class index.

Tensor shapes show why normalising across the batch is a serious error even when the returned numbers still sit between zero and one.

Gradient descent and adaptive optimisers

Optimisation is presented as a sequence of local decisions. The gradient gives direction, the learning rate controls trust in that direction and the slope must be recomputed after every move.

Momentum, RMSProp and ADAM are compared through what they remember or rescale. Training curves are then read diagnostically so overfitting, oscillation and disconnected gradients are not all answered by changing the optimiser.

Feed-forward network architecture

A feed-forward network composes affine transformations with nonlinear activations.

The chapter keeps a shape ledger from input through hidden layers to output, because matrix dimensions catch architectural errors before code runs. Sigmoid, tanh and ReLU are compared by range and gradient behaviour.

Output design follows the label structure: binary, mutually exclusive multiclass, independent multi-label or regression.

Backpropagation through layers

Backpropagation is organised chain-rule bookkeeping. Forward values are stored, an upstream gradient arrives from the loss and each operation multiplies by a local derivative. Contributions add where paths branch.

A fully worked scalar graph and a finite-difference check show how an automatic-differentiation result can be audited rather than trusted as a black box.

Convolution, pooling and feature maps

Convolution reuses a local kernel across an image, creating feature maps with far fewer parameters than a fully connected image layer. The chapter separates spatial output size, channel count and trainable parameter count.

ReLU and max pooling are traced on new numeric arrays, and U-Net motivates why segmentation must recover spatial detail rather than compress everything into one label.

Scaled dot-product attention

Attention uses query-key similarity to create weights and uses those weights to mix values. Scaling by the square root of the key dimension prevents large dot products from forcing a sharply saturated softmax.

Matrix dimensions distinguish several queries from several keys and explain why the output has the value width. Masking is applied before softmax so disallowed positions receive no probability mass.

Multi-head and causal attention

Multi-head attention splits the model channel dimension into parallel subspaces while keeping the token sequence intact.

The chapter follows the combined query-key-value projection, reshaping, per-head score matrices, concatenation and final output projection. A causal mask is tested on a tiny sequence so future-token leakage is detected directly rather than inferred from a training curve.

Efficient Transformers and distillation

The final technical chapter separates methods that solve different costs.

KV caching avoids recomputing past keys and values during decoding. Rotary embeddings modify query-key geometry to encode relative position. Online attention reorganises stable softmax and value accumulation to reduce intermediate storage.

Token-level and sentence-level distillation transfer different forms of teacher information into a smaller student.

How to work with the guide

Recalculate every example before reading its answer, then change one input and predict the direction of the new result.

For implementation chapters, assert shapes and invariants in code: probabilities sum to one, masks have the intended pattern, gradients remain finite and reported outputs match a hand calculation. Keep an error log that records the first incorrect line, because repairing the earliest divergence is faster than inspecting the final number.

How to prepare the live product

The project format rewards ownership.

Build a demonstration route inside the product itself: context, one representative record, the transformation and model output, the action it supports, validation against a baseline and a failure case. Preserve the data dictionary, split logic, model settings, dependency versions and AI-use declaration.

The report and demonstrated application should point to the same frozen version.

Evidence and assessment control

Assessment labels, weights and formats follow the captured official course surface for this offering. The examples and drills here are independently authored and are not University assessment material. The published assessment surface names the assignment and project as the assessed work.

Detailed marking criteria are announced for later release, so confirm them, the submission settings and the precise project date on the course LMS.

Worked example · free

Trace a logistic product decision from input to action

Key terms

Logit

The unrestricted linear score used before a sigmoid or softmax converts scores into probabilities.

Sigmoid Function

A monotone squashing function that maps any real score to a value between zero and one.

Cross-Entropy

A loss that penalises the model according to the probability assigned to the observed class.

Softmax

A normalisation that converts a vector of logits into positive class probabilities summing to one.

Learning Rate

The factor controlling how far an optimiser moves in the direction opposite the gradient.

Momentum

An optimiser state that combines recent gradients so persistent directions accumulate and oscillating directions partly cancel.

Activation Function

A transformation applied after an affine layer that lets a network represent nonlinear relationships.

Backpropagation

An efficient reverse traversal of a computation graph that applies the chain rule to obtain parameter gradients.

Feature Map

A spatial output channel produced when one convolutional filter is applied across an input.

Attention Weight

A row-normalised relevance value derived from a query-key score and used to mix value vectors.

Causal Mask

A lower-triangular restriction that prevents an autoregressive position from attending to future positions.

KV Cache

Stored key and value tensors from earlier decoding steps that avoid recomputing unchanged history.

Knowledge Distillation

Training a smaller student model to imitate distributions or generated outputs supplied by a larger teacher.

MATH7005 FAQ

Questions students ask in each MATH7005 chapter

Why is logistic regression a classifier?

Its linear score is passed through a sigmoid to produce a positive-class probability, and a separate threshold converts that probability into a class. The word regression refers to the mathematical form, not the final task. Chapter 01 · Logistic Models and Sigmoid Scores

Why does a confident mistake cost so much?

Negative logarithm grows rapidly as the true-class probability approaches zero. The loss therefore distinguishes a hesitant error from a model that assigns almost all probability to the wrong outcome. Chapter 02 · Cross-Entropy and Likelihood

Why subtract the largest logit?

The operation prevents a large positive exponential from overflowing and leaves every softmax probability unchanged because the same multiplicative factor cancels from numerator and denominator. Chapter 03 · Softmax and Multiclass Prediction

Why must the gradient be recomputed?

The loss slope depends on the current parameter location. Once an update moves the model, the old derivative describes the previous point and may have the wrong magnitude or even direction. Chapter 04 · Gradient Descent and Adaptive Optimisers

Why are nonlinear activations necessary?

Without them, consecutive affine layers collapse into one affine transformation. Nonlinearity lets the network build curved boundaries and hierarchical representations that a single linear map cannot express. Chapter 05 · Feed-Forward Network Architecture

What does backpropagation add to the chain rule?

It organises and reuses intermediate derivatives across a computation graph, avoiding repeated calculation of the same subexpressions. The mathematics remains the chain rule, applied efficiently in reverse order. Chapter 06 · Backpropagation Through Layers

Why does convolution use fewer parameters?

The same small kernel is reused at every spatial location. Parameter count therefore depends on kernel dimensions and channel counts, rather than growing with the full image width and height. Chapter 07 · Convolution, Pooling and Feature Maps

Why are attention scores scaled?

Dot-product variance grows with key dimension, making softmax excessively peaked and leaving weak gradients at most positions. Dividing by the square root of key width keeps the score scale more stable. Chapter 08 · Scaled Dot-Product Attention

Does each head receive different tokens?

No. Each head usually sees the same sequence positions but projects their channel representations into different learned subspaces. The time axis remains intact while the model-width axis is divided. Chapter 09 · Multi-Head and Causal Attention

What work does a KV cache save?

It prevents the model from recomputing keys and values for earlier tokens at every decoding step. The new query still compares with the stored history, and the cache consumes memory that grows with sequence length. Chapter 10 · Efficient Transformers and Distillation

How to prepare for the assessments

Build one small notebook that follows the guide from start to finish. Use a tiny labelled dataset, calculate a logistic prediction and cross-entropy by hand, reproduce them in code, then extend the notebook to a multiclass output, a short network and one attention row. At every stage print shapes and retain a hand-computed test.

For the project, maintain a separate evidence index with data provenance, preprocessing, the validation split, baseline, final settings, failure cases and declared AI assistance. Rehearse the live demonstration from a cold start without slides.

After each rehearsal, write down the first point at which the audience could no longer connect the displayed result to its evidence, then repair that hand-off rather than adding more features. Confirm the released assessment criteria and current schedule on the course LMS before submission.

Your AI Computer Science tutor for MATH7005

Stuck on a hard MATH7005 question? Sia is AskSia’s AI Computer Science tutor — ask any MATH7005 Data Analytics with Programming question and get a clear, step-by-step explanation grounded in how the course is actually taught and assessed. Read this whole study guide free, then take your hardest questions to Sia.