MAT 242 Chap.12 Orthogonal Projections and Least Squares
Orthogonal Projections and Least Squares
Projection without an orthogonal basis
When W is described by an ordinary basis, put the basis vectors into the columns of A. The projection of u into W is some combination Ac, and it is the projection because the error u − Ac is perpendicular to every column. Those conditions stack into the normal equation, AᵀA c = Aᵀu.
With independent columns, AᵀA is invertible, c = (AᵀA)⁻¹Aᵀu, and the projection is p = Ac, which is also the vector in W closest to u.
Least squares
An inconsistent system Ax = b has no solution, but when the data in b are measurements, the natural question is which x makes Ax as close as possible to b. The least squares solution x̂ minimises the sum of the squared entries of Ax̂ − b, and it solves AᵀA x̂ = Aᵀb.
The formula is the projection formula without the final A: x̂ has one entry per variable, while Ax̂ has one entry per equation and approximates b.
What the paid chapter adds
It projects a vector onto a plane in R³, verifies the right angle, and then solves an inconsistent three-equation system exactly in fractions and to four decimal places, including the leftover error.
It also describes the course's own large example, rating football teams from game margins, which shows the typical least squares shape of many equations and few unknowns.
Two slips cost the most here: stopping at the coordinates c when a projection vector was asked for, and putting an extra A in front of a least squares answer.
What the Final asks
The Final Exam skills list includes the closest vector in a subspace, the orthogonal projection, and the least squares solution of a linear system.
Sample solutions penalise stopping at the coordinates when the projection vector was requested, and reporting the projection when the least squares solution was requested.
On the Final a graphing calculator is allowed, so the formulas can be evaluated directly, but A, the vector and the formula used should all be written down first.
Why the error is perpendicular
Among all points of a subspace, the closest one to u is the one whose error vector u − p meets the subspace at a right angle, just as the shortest path from a point to a line in the plane is the perpendicular one.
Translating that right-angle condition into dot products with the basis vectors is exactly what produces the normal equation, so the formula is geometry written in matrix form. A quick dot-product check of the error against each basis vector confirms any projection.
What this chapter covers
- 01
Projection with an ordinary basis
- 02
The normal equation
- 03
The closest vector in a subspace
- 04
Least squares solutions of inconsistent systems
- 05
Checking a least squares answer
Worked example · free
Least squares with one unknown
- 2A is the column (1, 1, 1) and b = (1, 2, 6), so AᵀA = 3 and Aᵀb = 9.
- 1Solve 3x̂ = 9: x̂ = 3.
- 1Interpret: the least squares answer is the mean of the three measurements.
Key terms
- Normal Equation
- The normal equation AᵀA c = Aᵀu characterises the coordinates of the projection onto the column space of A.
- Least Squares Solution
- A least squares solution is a vector x̂ that minimises the sum of the squared entries of Ax̂ − b.
- Closest Vector
- The closest vector to u in a subspace W is the orthogonal projection of u into W.
Orthogonal Projections and Least Squares FAQ
Is the least squares solution the same as the projection?
No. The least squares solution holds the weights, one per variable. Multiplying it by A gives the projection of b onto the column space, a different vector with one entry per equation.
What if the system actually has a solution?
Then b already lies in the column space, its projection is b itself, and the least squares method returns an exact solution, so nothing is lost by using it.
What is the distance from a vector to a subspace?
Project the vector onto the subspace and take the length of the difference. The difference is perpendicular to the subspace, which is why the projection is the closest point.
Exam move
Write the two formulas on one line, with and without the leading A, and say which question each answers. Then practise both on the same small matrix. Pair every least squares practice question with the matching projection question on the same matrix, so the two formulas stay distinct.
Working through Orthogonal Projections and Least Squares in MAT 242? Sia is AskSia’s AI Mathematics tutor — ask any MAT 242 Orthogonal Projections and Least Squares question and get a clear, step-by-step explanation grounded in how MAT 242 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
