MAT9004 · Mathematical Foundations For Data Science And Ai
Linear Algebra
Linear algebra is the algebra of data — the maths that lets you handle many numbers at once. The chapter opens with vectors: notation, arithmetic, length (norm), orthogonality, projection (the shadow of one vector on another) and linear independence. It then builds matrices and matrix multiplication, and arrives at the flagship by-hand technique of the whole unit: solving a linear system Ax = b by Gaussian elimination — row-reduce to upper-triangular form, then back-substitute, including the case of free variables and a parametrised general solution. From there come the identity, inverse and determinant (the 2×2 inverse and the 3×3 cofactor expansion), and finally eigenvalues and eigenvectors with diagonalisation A = PDP−1. Every piece is procedural and supplied no calculator, so the exam tests whether you can carry out the row operations and the eigen-solve cleanly by hand.
What this chapter covers
- 012.1 Vectors: notation and arithmetic
- 022.2 Length and orthogonality
- 032.3 Projection: the shadow of one vector on another
- 042.4 Linear independence
- 052.5 Matrices and matrix multiplication
- 062.6 Systems Ax = b solved by Gaussian elimination
- 072.7 Free variables and the general solution
- 082.8 Identity and inverse (the 2×2 inverse)
- 092.9 The determinant (2×2 and the 3×3 cofactor expansion)
- 102.10 Eigenvalues, eigenvectors and diagonalisation A = PDP⁻¹
Worked example: eigenvalues of a 2×2 matrix
- +1Set up the characteristic equation: eigenvalues solve det(A − λI) = 0.
- +1Form A − λI: [[2 − λ, 1], [1, 2 − λ]], whose determinant is (2 − λ)2 − (1)(1).
- +1Expand and solve: (2 − λ)2 − 1 = 0 ⇒ λ2 − 4λ + 3 = 0 ⇒ (λ − 1)(λ − 3) = 0.
- +1Read off the eigenvalues: λ = 1 and λ = 3.
Key terms
- Norm (length)
- The length of a vector v, written ∥v∥ = √(v·v) — the square root of the dot product of the vector with itself. It is how you measure distance and turn a vector into a unit vector.
- Orthogonal
- Two vectors are orthogonal (perpendicular) when their dot product is zero, u·v = 0. Orthogonality underlies projection, least squares and the geometry behind much of data science.
- Gaussian elimination
- The systematic row-reduction of an augmented matrix [A | b] to upper-triangular form using elementary row operations, after which back-substitution gives the solution of Ax = b. The flagship by-hand technique of the block.
- Determinant
- A single number computed from a square matrix that tells you whether it is invertible (non-zero means invertible). The 2×2 determinant is ad − bc; the 3×3 is found by cofactor expansion along a row with the +−+ sign pattern.
- Eigenvalue / eigenvector
- A scalar λ and non-zero vector v with Av = λv — the matrix only stretches v, it does not rotate it. Eigenvalues solve det(A − λI) = 0; the pair drives diagonalisation A = PDP−1.
Linear Algebra FAQ
What's the reliable way to solve Ax = b by hand?
Write the augmented matrix [A | b] and row-reduce to upper-triangular (echelon) form using elementary row operations, then back-substitute from the bottom row up. Work one column at a time, creating zeros below each pivot. Show every row operation — method marks are awarded for the reduction even if a number slips.
What does a free variable mean for the solution?
A free variable appears when a column has no pivot — the system has infinitely many solutions. You set the free variable to a parameter (say t), express the pivot variables in terms of it, and write the general solution as a particular vector plus t times a direction vector. Stating 'infinitely many solutions, parametrised by t' is the marked conclusion.
How do I find eigenvalues?
Solve the characteristic equation det(A − λI) = 0. For a 2×2 matrix this is a quadratic in λ; factor it to read off the two eigenvalues. To get each eigenvector, substitute the eigenvalue back into (A − λI)v = 0 and solve the (degenerate) system for v.
Do I get a formula sheet for the matrix formulas?
The exam is closed-book with no calculator but supplies a formula sheet. Standard identities (like the 2×2 inverse formula) may be on it, but the arithmetic — the row reduction, the cofactor expansion, the eigen-solve — you must carry out by hand. So practise the procedures, not the memorisation.
Exam move
The block rewards two chains drilled to automatic: row-reduce → back-substitute for Ax = b, and det(A − λI) = 0 → solve for λ → substitute back for eigenvectors. Practise Gaussian elimination on fresh 3×3 and 4×4 systems until your row operations are clean and you never lose a sign — this is the single most-examined technique. For free-variable systems, always finish with the parametrised general solution, not just 'no unique solution'. Keep a mental checklist for the determinant: 2×2 is ad − bc; 3×3 is cofactor expansion with the +−+ pattern. Because there is no calculator, neat by-hand arithmetic and shown working are where the marks are safest.
