BUSS6002 · Data Science In Business
Linear Algebra I: Vectors & Scientific Computing
Week 4 is where BUSS6002 turns from concepts into machinery: a dataset is a stack of vectors, and linear algebra lets you “vectorise” a statistic into one compact expression a computer evaluates fast. A vector in ℝⁿ is an ordered list of n reals (a column by default, written bold lower-case), and almost everything later in the unit is built from three operations — scaling, adding, and the inner (dot) product ⟨x,y⟩ = xᵀy, which returns a single scalar. From the inner product you get the Euclidean norm ‖x‖₂ (length), distance ‖x − y‖₂, and orthogonality (⟨x,y⟩ = 0). The payoff is that the sample mean, standard deviation and covariance are all inner products in disguise, which is why the unit teaches this before clustering and regression. The chapter is examined by the 25% mid-semester exam (Weeks 1–6) and is fair game on the final — and the course grades your notation, not just your arithmetic.
What this chapter covers
- 011. Vectors in ℝⁿ — ordered lists, columns by default, the zero 0 and ones 1 vectors
- 022. Scalar multiplication & vector addition — entry-by-entry, equal sizes only
- 033. Linear combinations & linear (in)dependence — the non-trivial-coefficients test
- 044. Inner (dot) product ⟨x,y⟩ = xᵀy — a scalar, with its algebraic and angle properties
- 055. Euclidean norm ‖x‖₂ = √⟨x,x⟩ and squared norm ‖x‖₂² (used by K-means & OLS)
- 066. Euclidean distance ‖x − y‖₂ and orthogonality ⟨x,y⟩ = 0
- 077. Statistics as inner products — sample mean, sd and covariance via the ones vector
- 088. Scientific computing in NumPy — vectorise not loop, numpy.dot at every scale
Worked example: dot product, orthogonality and norm
- +1(a) Inner product = sum of entry-wise products: ⟨a, b⟩ = (3)(2) + (1)(k) + (−2)(4) = 6 + k − 8 = k − 2.
- +1(b) Orthogonal means the inner product is zero: k − 2 = 0, so k = 2.
- +1(c) With k = 2, b = [2, 2, 4]ᵀ, so ‖b‖₂ = √(2² + 2² + 4²) = √(4 + 4 + 16) = √24.
- +1Simplify the surd: √24 = 2√6 ≈ 4.90 (3 s.f.). Do not forget the final square root.
- 0Check: a positive partial cancels a negative one to hit zero, and the norm is positive — both sanity checks pass.
Key terms
- Vector (in ℝⁿ)
- An ordered list of n real numbers, written bold lower-case and treated as a column by default; it represents one data point's features or one variable across observations.
- Scalar multiplication
- Multiplying every entry of a vector by the same number a: a·x = [ax₁, …, axₙ]ᵀ. It stretches (a > 1), shrinks (0 < a < 1) or flips (a < 0) the arrow.
- Linear combination
- A sum of scaled vectors, Σ aᵢxᵢ. The vectors are linearly dependent if some non-trivial choice of coefficients (not all zero) makes the sum equal 0.
- Inner (dot) product
- ⟨x,y⟩ := Σ xᵢyᵢ = xᵀy, the entry-wise products summed into a single scalar. It is symmetric, scales out, and distributes over addition.
- Orthogonality
- Two vectors are orthogonal (at a right angle) exactly when their inner product is zero: x ⊥ y ⟺ ⟨x,y⟩ = 0.
- Euclidean norm (2-norm)
- ‖x‖₂ := √⟨x,x⟩ = (Σ xᵢ²)^(1/2), the length of the vector. The squared norm ‖x‖₂² drops the root and equals the inner product of x with itself.
- Euclidean distance
- ‖x − y‖₂, the norm of the difference between two vectors — the gap K-means minimises when assigning a point to its nearest centre.
- Vectorisation (NumPy)
- Replacing an explicit Python loop with a whole-array operation (e.g. numpy.dot) so the arithmetic runs in fast compiled code — the practical reason for writing statistics as inner products.
Linear Algebra I: Vectors & Scientific Computing FAQ
Is the inner product a vector or a number?
A single number (a scalar). The dot product multiplies two equal-length vectors entry-by-entry and sums the results, collapsing both n-vectors into one value. If your answer is still a vector you have done element-wise (Hadamard) multiplication instead — a different operation. This is one of the most common one-mark slips in the unit.
What does 'orthogonal' mean in an exam question?
It is code for 'set the dot product to zero and solve'. Two vectors are orthogonal exactly when ⟨x,y⟩ = 0, which geometrically is a 90° angle (because cos 90° = 0). So whenever you see 'orthogonal', 'perpendicular' or 'at right angles', write the inner product, set it to zero, and solve for the unknown.
When do I take the square root for a norm or distance?
‖x‖₂ (the norm) and ‖x − y‖₂ (distance) both need the final square root of the sum of squares. The squared norm ‖x‖₂² does not — it is just the sum of squares, equal to the inner product of the vector with itself. The course deliberately tests both, and K-means uses the squared version because it is cheaper, so always read which one is written.
How do I show a set of vectors is linearly dependent?
Find a non-trivial combination (coefficients not all zero) that sums to the zero vector. The fastest route is to spot when one vector is a scalar multiple of another, e.g. if v₂ = 2v₁ then 2v₁ − v₂ + 0·v₃ = 0 with coefficients (2, −1, 0). State the explicit coefficients — that is where the mark is. Setting every coefficient to zero is the trivial solution and proves nothing.
Why does a 'business' unit spend a week on vectors?
Because everyday statistics are inner products in disguise: the sample mean is ⟨(1/n)1, x⟩, the (zero-mean) standard deviation is (1/√n)‖x‖₂, and covariance is a cross inner product. Writing them this way lets NumPy compute them on millions of rows without a Python loop, and the same ⟨·,·⟩ reappears in K-means, OLS and the log-likelihood later in the course.
Will I have to write Python by hand?
Yes. Both exams include a hand-written Python-code section, and the final has a numpy.dot question (for example a grid search that evaluates np.dot over candidate vectors). The golden rule is vectorise rather than loop: build arrays with numpy.array and compute inner and matrix products with numpy.dot rather than iterating in Python. You must be able to write runnable NumPy from memory, not just read it.
Exam move
Treat Linear Algebra I as guaranteed marks: the question types are few and stable, so drill each to reflex. Practise vector arithmetic until the sign on the last component never trips you (−2 × −2 = +4 makes the entry grow, not shrink). Memorise the chain inner product → norm → distance → orthogonality, and always ask whether a question wants ‖·‖ or ‖·‖² before you compute. For dependence, look first for one vector being a multiple of another and write the explicit non-trivial coefficients. Because this is closed-book with only a one-page (single-sided) handwritten note for the mid-sem, put the four formulas — dot product, norm, distance, orthogonality test — plus the ones-vector trick for the mean on that sheet, and rehearse a small numpy.array + numpy.dot snippet so the code section is automatic. Above all, use the course's exact notation (bold columns, transpose, ⟨·,·⟩, ‖·‖₂): marks are deducted for sloppy notation even when the arithmetic is right.
