ECMT3120 Chap.2 Matrix OLS from Objective to Inference
Matrix OLS from Objective to Inference
Matrix notation turns least squares into a transparent sequence: construct the design matrix, define the residual vector, minimise its squared length, solve the normal equations, and verify dimensions and rank.
This chapter derives the coefficient vector, explains why fitted residuals are orthogonal to design columns, distinguishes that sample identity from population exogeneity, and links the estimation error to conditional unbiasedness and sampling variance.
What this chapter covers
- 01
Design matrix, outcome vector and coefficient dimensions
- 02
Residual vectors and sums of squared residuals
- 03
Normal equations and the least squares solution
- 04
Full column rank and perfect collinearity
- 05
Dummy variable base categories
- 06
Residual orthogonality checks
- 07
Conditional unbiasedness and OLS sampling variance
Worked example · free
Solve a three observation regression
- +1Build the design matrix with an intercept column and an x column, and stack y as a column vector.
- +2Compute X transpose X as [[3,6],[6,14]] and X transpose y as [10,23].
- +2Invert the cross product and obtain intercept one third and slope three halves.
- +1Calculate fitted values 1.833, 3.333 and 4.833 and residuals 0.167, minus 0.333 and 0.167.
- +1Verify that residuals sum to zero and their x weighted sum is zero up to rounding.
- +1Interpret the slope in outcome units per unit of x and keep causal claims conditional on exogeneity.
Key terms
- Design matrix
- The matrix whose rows are observations and whose columns are the intercept and included regressors.
- Normal equations
- The first order conditions equating the design cross product with fitted residuals to zero.
- Residual orthogonality
- The fitted least squares property that every included design column has zero sample cross product with residuals.
- Full column rank
- The condition that no design column is an exact linear combination of the others.
- Perfect collinearity
- Exact redundancy among regressors that makes the cross product singular and prevents a unique coefficient solution.
- Sampling variance
- The repeated sample variability of an estimator around its expected value under a specified model.
Matrix OLS from Objective to Inference FAQ
Why does full column rank matter for OLS?
Full column rank makes the design cross product invertible and gives a unique coefficient vector. If an intercept accompanies an indicator for every category, the design contains perfect linear dependence. Omitting a base category restores rank and defines the coefficient comparisons.
Do orthogonal fitted residuals prove exogeneity?
No. Residual orthogonality follows mechanically from minimising the sample sum of squared residuals. Exogeneity concerns the unobserved population error and the process generating regressors. A sample identity cannot by itself verify the structural assumption needed for causal interpretation.
How do dimensions help during a matrix calculation?
Dimensions show whether multiplication is conformable and whether the result has the intended shape. With n observations and k regressors, the coefficient vector must be k by one. A conformable product that returns n entries may still be the wrong map for estimating coefficients.
Exam move
Rebuild one small OLS example entirely by hand. Write every matrix with its dimensions, compute the determinant before the inverse, and keep the multiplication order visible. After obtaining coefficients, calculate fitted values and residuals and test both residual sums. Then explain why these sample checks differ from conditional mean independence. For dummy variables, identify the base category before any algebra.
Finally, derive the estimation error by substituting the model into the estimator and use it to explain where unbiasedness and the variance formula come from. This sequence makes matrix notation a diagnostic language rather than a set of symbols to memorise. Add a dimension ledger beside the calculation: X is n by k, its transpose is k by n, the cross-product is k by k, and the coefficient vector must end k by one.
When a product is conformable but returns the wrong destination, write that explicitly instead of calling it illegal multiplication. Create one rank-failure example using an intercept and exhaustive category indicators, then repair it two ways and interpret both parameterisations. For inference, trace the disturbance through the estimator before invoking unbiasedness or a variance formula.
A useful oral drill is to explain why residual orthogonality always holds after least squares even when the population error is endogenous. End each numerical exercise by substituting the coefficients back into the fitted equation, checking residual identities with exact fractions where possible, and translating the slope into the outcome and regressor units.
Working through Matrix OLS from Objective to Inference in ECMT3120? Sia is AskSia’s AI Economics tutor — ask any ECMT3120 Matrix OLS from Objective to Inference question and get a clear, step-by-step explanation grounded in how ECMT3120 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
