ECON31031 Chap.1 OLS and the Linear Model in Matrix Form
OLS and the Linear Model in Matrix Form
OLS and the Linear Model in Matrix Form restates the standard linear model as y = Xβ + ε so that every later estimator has a benchmark. With n observations and k regressors plus an intercept, X has n rows and k+1 columns.
Under the Gauss-Markov conditions, the least squares estimator (X'X)-1X'y is unbiased with conditional variance σ2(X'X)-1 and is the best linear unbiased estimator.
The projection matrix P and the residual maker M = I - P are symmetric and idempotent, and the trace of M explains why the unbiased variance estimator divides the residual sum of squares by n - k - 1. Normal errors give exact t and F tests; without normality, large-sample theory replaces them with the standard normal and chi-squared distributions, including the nR2 form used by RESET and other misspecification checks.
The chapter also shows how goodness of fit is measured: total variation splits into explained and residual parts, R-squared compares them, and adjusted R-squared uses s-squared to compensate for adding regressors. Restricted least squares can be computed directly with Lagrange multipliers, which explains why imposing restrictions that already hold in the sample leaves the estimates unchanged.
What this chapter covers
- 01
The standard linear model y = Xβ + ε and its dimensions
- 02
Gauss-Markov assumptions and the BLUE property
- 03
The OLS estimator, its mean and its conditional variance
- 04
Projection and residual-maker matrices
- 05
Trace rules and the unbiased estimator s²
- 06
Testing with t, F and restricted least squares
- 07
Large-sample chi-squared tests and auxiliary regressions
Worked example · free
Fit a two-coefficient regression with matrix algebra
- 2X'X has rows (4, 6) and (6, 14) with determinant 20, and X'y = (14, 29)'. The inverse is one twentieth of the matrix with rows (14, -6) and (-6, 4).
- 2Multiplying gives (22/20, 32/20)' = (1.1, 1.6)'. Fitted values 1.1, 2.7, 4.3 and 5.9 leave residuals -0.1, 0.3, -0.3 and 0.1, which sum to zero.
- 2e'e = 0.2 and n - k - 1 = 2, so s² = 0.1. The slope variance is 0.1 × 4/20 = 0.02, the standard error is 0.141 and t = 1.6/0.141 = 11.3.
Key terms
- Projection matrix
- The matrix X(X'X)^-1 X' that maps the response vector onto its fitted values.
- Residual maker
- The matrix I minus the projection matrix, which turns the response vector into the residual vector.
- Idempotent matrix
- A symmetric matrix that is unchanged when multiplied by itself.
OLS and the Linear Model in Matrix Form FAQ
Why does s² divide by n - k - 1?
The expected residual sum of squares equals the trace of the residual maker times the error variance, and that trace is n - k - 1. Dividing by this number makes the estimator unbiased, while dividing by n would understate the error variance.
When can I use the normal table instead of t?
Without normal errors the notes rely on large samples. As the sample grows, the t distribution approaches the standard normal, so single restrictions are judged against the normal table and joint restrictions against chi-squared once F is multiplied by the number of restrictions.
How do I impose a linear restriction?
Write the restriction as Rβ = r, substitute it into the model to remove one coefficient, and estimate the transformed equation by OLS. The restricted residual sum of squares then feeds the F statistic alongside the unrestricted one.
What does the residual-maker matrix do?
It converts the response vector into the residual vector, and because it annihilates X it also converts the error vector into the same residuals. Its symmetry and idempotency are the two properties used in almost every derivation, including the proof that s-squared is unbiased.
Why does adjusted R-squared exist?
Ordinary R-squared never falls when a regressor is added, so it rewards bigger models automatically. The adjusted version replaces sums of squares with variance estimates that account for degrees of freedom, so an irrelevant regressor can lower it.
Exam move
Rebuild the matrix results from the assumptions rather than memorising them. Start each derivation by writing the dimensions of y, X, β and the error vector, then use only named properties: MX = 0, symmetry, idempotency and the cyclic trace rule. Practise one small numerical regression by hand each week, with two coefficients and four or five observations, so that inverting a two-by-two matrix becomes routine.
Keep a separate list of test statistics with their distributions and degrees of freedom, and add the large-sample version beside each exact one. When a derivation stalls, return to the definitions: write e = My, substitute y = Xβ + ε, and use MX = 0 before taking expectations. Time yourself on one full derivation per sitting, because exam answers need both the result and the reasons.
Working through OLS and the Linear Model in Matrix Form in ECON31031? Sia is AskSia’s AI Economics tutor — ask any ECON31031 OLS and the Linear Model in Matrix Form question and get a clear, step-by-step explanation grounded in how ECON31031 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.