MATH2089 Chap.3 Least Squares and Interpolation
Least Squares and Interpolation
Interpolation and least squares use polynomial language for different objectives. Interpolation matches every exact node, while least squares accepts residuals and minimises their squared length for an overdetermined system.
This chapter explains the projection geometry, compares normal equations with QR factorisation, uses residuals to diagnose a fit and shows how Lagrange form and Horner evaluation serve distinct construction and evaluation tasks. The first decision is whether the supplied values are exact constraints or noisy observations. Exact distinct nodes support interpolation: the polynomial is required to reproduce every node value.
Noisy or overdetermined data support least squares: discrepancies are expected, and the objective is to minimise their squared total rather than force a curve through every fluctuation. Lagrange basis polynomials make the node conditions explicit. Each basis function is one at its own node and zero at all others, so substituting the nodes provides a direct structural check before expanding any coefficients.
Other representations may be better for adding nodes or repeated evaluation. The same polynomial can look very different algebraically, and conversion should serve a purpose rather than become an automatic extra step. Horner evaluation rewrites a polynomial as a nested sequence. It reduces the multiplication pattern, makes the operation count visible and avoids separately forming high powers.
The result should still be checked at a simple value or against direct evaluation for a small case. Efficient evaluation does not cure poor interpolation behaviour, especially when a high-degree polynomial oscillates between widely spaced nodes. Least squares has a geometric interpretation.
The fitted vector is the projection of the observation vector onto the design matrix column space, and the residual is orthogonal to that space. With an intercept, residuals sum to zero; additional design columns create further orthogonality checks. Residual signs and patterns reveal structure that a coefficient list cannot show.
The normal equations express the projection condition but form A transpose A, which can square the two-norm condition number. QR factorisation reaches a triangular solve without that extra sensitivity amplification and is usually the safer computational route. A hand calculation may still use normal equations when the problem is small and the arithmetic is part of the learning objective.
Finish by matching the verification to the task: substitute every interpolation node, inspect least-squares residuals and orthogonality, or trace the nested operations used for evaluation. If a prediction lies outside the data range, label it as extrapolation because the polynomial or fitted trend is being trusted where the observations supplied no direct check.
What this chapter covers
- 01
Overdetermined systems and residuals
- 02
Orthogonal projection
- 03
Normal equations
- 04
QR factorisation and conditioning
- 05
Lagrange interpolation
- 06
Horner polynomial evaluation
Worked example · free
Fit a three point line
- +1Compute the required sums n=3, sum x=3, sum y=5, sum x squared=5 and sum xy=6.
- +1Solve 3a+3b=5 and 3a+5b=6.
- +1Obtain b=0.5 and a=7/6.
- +1Compute residuals and verify that they sum to zero because an intercept is present.
Key terms
- Least squares
- A fitting criterion that minimises the sum of squared residuals.
- Normal equations
- Equations expressing residual orthogonality through A transpose A and A transpose y.
- Lagrange basis
- Interpolation polynomials that equal one at one node and zero at the others.
Least Squares and Interpolation FAQ
Why is QR preferred to normal equations?
Forming the normal equations squares the two norm condition number of the design matrix. QR solves an equivalent projection problem without that extra sensitivity amplification.
When is interpolation appropriate?
Interpolation is appropriate when the supplied node values are treated as exact and the required polynomial must match every node. Noisy measurements usually call for fitting instead.
What does a residual pattern reveal?
A systematic curve, changing spread or influential point can show that the model form or error assumptions are inadequate. Residual inspection is part of fitting, not an optional decoration.
Exam move
Practise deciding whether a prompt asks for exact node matching or an approximate fit. Work one normal equation example by hand, then explain why QR is the stable computational method. Reconstruct a Lagrange interpolant and evaluate a power form with Horner's method so construction and evaluation remain separate. Check every interpolant by substitution at all nodes before expanding it.
For each least-squares fit, calculate fitted values and residuals, then verify an orthogonality condition rather than stopping at coefficients. Make a one-page comparison of monomial, Lagrange, Newton and nested forms that records what each representation makes easy. Include an ill-conditioned design example and describe why forming the normal equations worsens sensitivity.
Finish with an extrapolation warning whenever the requested evaluation lies beyond the data range. These habits keep the objective, representation and numerical route visible together.
Working through Least Squares and Interpolation in MATH2089? Sia is AskSia’s AI Mathematics tutor — ask any MATH2089 Least Squares and Interpolation question and get a clear, step-by-step explanation grounded in how MATH2089 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.