The University of Manchester · FACULTY OF ECONOMICS

ECON31031 Chap.2 Maximum Likelihood Estimation for IID Data

- one subject, every graph, every model, every mark
6 Chapters4-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 2 of 9 · ECON31031

Maximum Likelihood Estimation for IID Data

Maximum Likelihood Estimation for IID Data is the method behind most of the module. For an independent sample, the likelihood is the product of individual densities, and its logarithm is a sum that can be differentiated.

Setting the score to zero gives the estimator; sometimes it has a closed form, as for a Bernoulli proportion or the slopes of a normal linear model, and sometimes it must be found numerically because the parameters sit inside a nonlinear function.

The curvature of the log-likelihood, measured by the Hessian, supplies a covariance estimator, and comparing maximised log-likelihoods gives the likelihood ratio test, read against chi-squared critical values. The same machinery later estimates probit, logit and ARMA models. The chapter connects this machinery to testing in two ways.

A likelihood ratio compares the maximised log-likelihoods with and without restrictions, and an intercept-only binary model supplies its restricted value from the sample proportion alone. Pseudo-R-squared measures reuse the same two numbers, which is why significance and fit should be reported separately.

In this chapter

What this chapter covers

  • 01

    Likelihood and log-likelihood for IID samples

  • 02

    The score and first-order conditions

  • 03

    Closed-form and numerical solutions

  • 04

    The Hessian and covariance estimation

  • 05

    Likelihood ratio tests against chi-squared values

  • 06

    Normal linear models: ML versus OLS variance estimators

Worked example · free

Estimate a Bernoulli probability and test it with a likelihood ratio

Q [6 marks]. In 80 independent trials there are 20 successes. Find the maximum likelihood estimate of the success probability and use a likelihood ratio test of p = 0.5 at the 5% level. This practice weighting is assigned by AskSia and is not an official assessment scheme.
The curve is the Bernoulli log-likelihood for six successes in twenty trials.The curve is the Bernoulli log-likelihood for six successes in twenty trials. It peaks at the sample proportion 0.30, and the dashed line marks the values from 0.13 to 0.52 that a 5% likelihood ratio test would not reject.0.10.30.50.7-12-16-20peak at 0.301.92 below peaksuccess probability plog-likelihood
The curve is the Bernoulli log-likelihood for six successes in twenty trials. It peaks at the sample proportion 0.30, and the dashed line marks the values from 0.13 to 0.52 that a 5% likelihood ratio test would not reject.
  • 2The log-likelihood is 20 ln p + 60 ln(1 - p). Setting the score 20/p - 60/(1 - p) to zero gives p = 20/80 = 0.25.
  • 2At the estimate the log-likelihood is 20 ln 0.25 + 60 ln 0.75 = -44.99; under the null it is 80 ln 0.5 = -55.45.
  • 2LR = 2(-44.99 + 55.45) = 20.93, far above the chi-squared critical value 3.841 for one restriction.
The estimate is 0.25 and the likelihood ratio statistic of 20.93 rejects p = 0.5 decisively.
Sia tip — Count restrictions, not parameters: testing one value of p uses one degree of freedom whatever the sample size.
Glossary

Key terms

Log-likelihood
The natural logarithm of the likelihood, which turns a product of densities into a sum.
Hessian matrix
The matrix of second derivatives of the log-likelihood, whose negative inverse estimates the covariance of the estimator.
Restricted log-likelihood
The maximised log-likelihood of a model with the null hypothesis imposed, used as the baseline in a likelihood ratio test.
FAQ

Maximum Likelihood Estimation for IID Data FAQ

When does a maximum likelihood estimator have a closed form?

It has one when the score equations can be solved algebraically, as for a Bernoulli proportion or the slopes of a normal linear model. When parameters enter through a nonlinear function summed over observations, such as a probit, the equations must be solved numerically.

Why is the ML variance estimate different from s²?

In the normal linear model maximum likelihood divides the residual sum of squares by n, while the unbiased OLS estimator divides by n - k - 1. Both are consistent, but only the OLS version is unbiased in finite samples.

How many degrees of freedom does a likelihood ratio test use?

It uses the number of restrictions imposed by the null hypothesis. Testing that one slope is zero uses one, and testing that three slopes are jointly zero uses three, whatever the total number of parameters.

What is the score function?

The score is the vector of first derivatives of the log-likelihood with respect to the parameters. Setting it to zero gives the first-order conditions, and for an independent sample it is a sum of contributions, one from each observation, which makes it straightforward to write down.

How do I get a standard error from a likelihood?

Take the second derivatives of the log-likelihood, change their sign, evaluate them at the estimate and invert the result. For a Bernoulli probability this gives the variance p(1 - p)/n, whose square root is the standard error used for intervals and Wald tests.

Study strategy

Exam move

Work every likelihood question through the same six steps: density, log-likelihood, score, solution, curvature and test. Write the density for one observation first and check that it respects the support of the response. Differentiate with the chain rule and keep the regressor vector visible in each term of the score.

Memorise the intercept-only shortcut for binary data, because it gives the restricted log-likelihood from the sample proportion alone. Finish each test with the statistic, the degrees of freedom, the critical value and a sentence of conclusion.

Keep a page of standard densities beside your notes, Bernoulli, normal and the exponential-type densities used in the exercise sheets, and practise writing each log-likelihood within a minute. For every test you compute, also write the null hypothesis in words, because a correct statistic with an unclear hypothesis loses its meaning.

Working through Maximum Likelihood Estimation for IID Data in ECON31031? Sia is AskSia’s AI Economics tutor — ask any ECON31031 Maximum Likelihood Estimation for IID Data question and get a clear, step-by-step explanation grounded in how ECON31031 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 41 of your subjects at The University of Manchester, plus 1,000+ more Bibles across the AskSia library.
Sia - your ECON31031 tutor, unlimited, worked the way the exam marks it
The full 4-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works