The University of Manchester · FACULTY OF ECONOMICS

ECON31031 Chap.1 OLS and the Linear Model in Matrix Form

- one subject, every graph, every model, every mark
7 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 1 of 9 · ECON31031

OLS and the Linear Model in Matrix Form

OLS and the Linear Model in Matrix Form restates the standard linear model as y = Xβ + ε so that every later estimator has a benchmark. With n observations and k regressors plus an intercept, X has n rows and k+1 columns.

Under the Gauss-Markov conditions, the least squares estimator (X'X)-1X'y is unbiased with conditional variance σ2(X'X)-1 and is the best linear unbiased estimator.

The projection matrix P and the residual maker M = I - P are symmetric and idempotent, and the trace of M explains why the unbiased variance estimator divides the residual sum of squares by n - k - 1. Normal errors give exact t and F tests; without normality, large-sample theory replaces them with the standard normal and chi-squared distributions, including the nR2 form used by RESET and other misspecification checks.

The chapter also shows how goodness of fit is measured: total variation splits into explained and residual parts, R-squared compares them, and adjusted R-squared uses s-squared to compensate for adding regressors. Restricted least squares can be computed directly with Lagrange multipliers, which explains why imposing restrictions that already hold in the sample leaves the estimates unchanged.

In this chapter

What this chapter covers

  • 01

    The standard linear model y = Xβ + ε and its dimensions

  • 02

    Gauss-Markov assumptions and the BLUE property

  • 03

    The OLS estimator, its mean and its conditional variance

  • 04

    Projection and residual-maker matrices

  • 05

    Trace rules and the unbiased estimator s²

  • 06

    Testing with t, F and restricted least squares

  • 07

    Large-sample chi-squared tests and auxiliary regressions

Worked example · free

Fit a two-coefficient regression with matrix algebra

Q [6 marks]. Four observations on (x, y) are (0, 1), (1, 3), (2, 4) and (3, 6). Use (X'X)^-1 X'y to estimate the intercept and slope, find s² and test whether the slope is zero. This practice weighting is assigned by AskSia and is not an official assessment scheme.
Least squares drops the observed vector y onto the space spanned by the columns of X.Least squares drops the observed vector y onto the space spanned by the columns of X. The blue segment is the fitted vector Py, and the dashed orange residual vector My meets that space at a right angle.originy (observed)fitted values Pyresiduals Mycolumn space of X
Least squares drops the observed vector y onto the space spanned by the columns of X. The blue segment is the fitted vector Py, and the dashed orange residual vector My meets that space at a right angle.
  • 2X'X has rows (4, 6) and (6, 14) with determinant 20, and X'y = (14, 29)'. The inverse is one twentieth of the matrix with rows (14, -6) and (-6, 4).
  • 2Multiplying gives (22/20, 32/20)' = (1.1, 1.6)'. Fitted values 1.1, 2.7, 4.3 and 5.9 leave residuals -0.1, 0.3, -0.3 and 0.1, which sum to zero.
  • 2e'e = 0.2 and n - k - 1 = 2, so s² = 0.1. The slope variance is 0.1 × 4/20 = 0.02, the standard error is 0.141 and t = 1.6/0.141 = 11.3.
The fitted line is y = 1.1 + 1.6x, s² is 0.1 and the slope t statistic of 11.3 on two degrees of freedom rejects a zero slope.
Sia tip — Check that the residuals sum to zero and are orthogonal to x before using them; X'e = 0 catches an arithmetic slip in the inverse.
Glossary

Key terms

Projection matrix
The matrix X(X'X)^-1 X' that maps the response vector onto its fitted values.
Residual maker
The matrix I minus the projection matrix, which turns the response vector into the residual vector.
Idempotent matrix
A symmetric matrix that is unchanged when multiplied by itself.
FAQ

OLS and the Linear Model in Matrix Form FAQ

Why does s² divide by n - k - 1?

The expected residual sum of squares equals the trace of the residual maker times the error variance, and that trace is n - k - 1. Dividing by this number makes the estimator unbiased, while dividing by n would understate the error variance.

When can I use the normal table instead of t?

Without normal errors the notes rely on large samples. As the sample grows, the t distribution approaches the standard normal, so single restrictions are judged against the normal table and joint restrictions against chi-squared once F is multiplied by the number of restrictions.

How do I impose a linear restriction?

Write the restriction as Rβ = r, substitute it into the model to remove one coefficient, and estimate the transformed equation by OLS. The restricted residual sum of squares then feeds the F statistic alongside the unrestricted one.

What does the residual-maker matrix do?

It converts the response vector into the residual vector, and because it annihilates X it also converts the error vector into the same residuals. Its symmetry and idempotency are the two properties used in almost every derivation, including the proof that s-squared is unbiased.

Why does adjusted R-squared exist?

Ordinary R-squared never falls when a regressor is added, so it rewards bigger models automatically. The adjusted version replaces sums of squares with variance estimates that account for degrees of freedom, so an irrelevant regressor can lower it.

Study strategy

Exam move

Rebuild the matrix results from the assumptions rather than memorising them. Start each derivation by writing the dimensions of y, X, β and the error vector, then use only named properties: MX = 0, symmetry, idempotency and the cyclic trace rule. Practise one small numerical regression by hand each week, with two coefficients and four or five observations, so that inverting a two-by-two matrix becomes routine.

Keep a separate list of test statistics with their distributions and degrees of freedom, and add the large-sample version beside each exact one. When a derivation stalls, return to the definitions: write e = My, substitute y = Xβ + ε, and use MX = 0 before taking expectations. Time yourself on one full derivation per sitting, because exam answers need both the result and the reasons.

Working through OLS and the Linear Model in Matrix Form in ECON31031? Sia is AskSia’s AI Economics tutor — ask any ECON31031 OLS and the Linear Model in Matrix Form question and get a clear, step-by-step explanation grounded in how ECON31031 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 41 of your subjects at The University of Manchester, plus 1,000+ more Bibles across the AskSia library.
Sia - your ECON31031 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works