ECON31031 Chap.2 Maximum Likelihood Estimation for IID Data
Maximum Likelihood Estimation for IID Data
Maximum Likelihood Estimation for IID Data is the method behind most of the module. For an independent sample, the likelihood is the product of individual densities, and its logarithm is a sum that can be differentiated.
Setting the score to zero gives the estimator; sometimes it has a closed form, as for a Bernoulli proportion or the slopes of a normal linear model, and sometimes it must be found numerically because the parameters sit inside a nonlinear function.
The curvature of the log-likelihood, measured by the Hessian, supplies a covariance estimator, and comparing maximised log-likelihoods gives the likelihood ratio test, read against chi-squared critical values. The same machinery later estimates probit, logit and ARMA models. The chapter connects this machinery to testing in two ways.
A likelihood ratio compares the maximised log-likelihoods with and without restrictions, and an intercept-only binary model supplies its restricted value from the sample proportion alone. Pseudo-R-squared measures reuse the same two numbers, which is why significance and fit should be reported separately.
What this chapter covers
- 01
Likelihood and log-likelihood for IID samples
- 02
The score and first-order conditions
- 03
Closed-form and numerical solutions
- 04
The Hessian and covariance estimation
- 05
Likelihood ratio tests against chi-squared values
- 06
Normal linear models: ML versus OLS variance estimators
Worked example · free
Estimate a Bernoulli probability and test it with a likelihood ratio
- 2The log-likelihood is 20 ln p + 60 ln(1 - p). Setting the score 20/p - 60/(1 - p) to zero gives p = 20/80 = 0.25.
- 2At the estimate the log-likelihood is 20 ln 0.25 + 60 ln 0.75 = -44.99; under the null it is 80 ln 0.5 = -55.45.
- 2LR = 2(-44.99 + 55.45) = 20.93, far above the chi-squared critical value 3.841 for one restriction.
Key terms
- Log-likelihood
- The natural logarithm of the likelihood, which turns a product of densities into a sum.
- Hessian matrix
- The matrix of second derivatives of the log-likelihood, whose negative inverse estimates the covariance of the estimator.
- Restricted log-likelihood
- The maximised log-likelihood of a model with the null hypothesis imposed, used as the baseline in a likelihood ratio test.
Maximum Likelihood Estimation for IID Data FAQ
When does a maximum likelihood estimator have a closed form?
It has one when the score equations can be solved algebraically, as for a Bernoulli proportion or the slopes of a normal linear model. When parameters enter through a nonlinear function summed over observations, such as a probit, the equations must be solved numerically.
Why is the ML variance estimate different from s²?
In the normal linear model maximum likelihood divides the residual sum of squares by n, while the unbiased OLS estimator divides by n - k - 1. Both are consistent, but only the OLS version is unbiased in finite samples.
How many degrees of freedom does a likelihood ratio test use?
It uses the number of restrictions imposed by the null hypothesis. Testing that one slope is zero uses one, and testing that three slopes are jointly zero uses three, whatever the total number of parameters.
What is the score function?
The score is the vector of first derivatives of the log-likelihood with respect to the parameters. Setting it to zero gives the first-order conditions, and for an independent sample it is a sum of contributions, one from each observation, which makes it straightforward to write down.
How do I get a standard error from a likelihood?
Take the second derivatives of the log-likelihood, change their sign, evaluate them at the estimate and invert the result. For a Bernoulli probability this gives the variance p(1 - p)/n, whose square root is the standard error used for intervals and Wald tests.
Exam move
Work every likelihood question through the same six steps: density, log-likelihood, score, solution, curvature and test. Write the density for one observation first and check that it respects the support of the response. Differentiate with the chain rule and keep the regressor vector visible in each term of the score.
Memorise the intercept-only shortcut for binary data, because it gives the restricted log-likelihood from the sample proportion alone. Finish each test with the statistic, the degrees of freedom, the critical value and a sentence of conclusion.
Keep a page of standard densities beside your notes, Bernoulli, normal and the exponential-type densities used in the exercise sheets, and practise writing each log-likelihood within a minute. For every test you compute, also write the null hypothesis in words, because a correct statistic with an unclear hypothesis loses its meaning.
Working through Maximum Likelihood Estimation for IID Data in ECON31031? Sia is AskSia’s AI Economics tutor — ask any ECON31031 Maximum Likelihood Estimation for IID Data question and get a clear, step-by-step explanation grounded in how ECON31031 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.