ETC3400 Chap.1 Likelihood and Maximum Likelihood Estimation
Likelihood and Maximum Likelihood Estimation
Likelihood and Maximum Likelihood Estimation starts with the statistical model: sample space, parameter space, sampling assumptions and a probability law. Once the sample is observed, its joint density is viewed as a function of the parameter.
The chapter explains why independence creates a product likelihood, why a strictly increasing logarithm preserves the maximiser, and why the score equation locates only a candidate until curvature and boundaries are checked. A Bernoulli derivation makes every step visible, but the workflow is designed to transfer to unfamiliar distributions rather than produce one memorised answer.
What this chapter covers
- 01
Statistical model: possible samples, admissible parameters and a probability law
- 02
Independent sampling and the product form of the joint density
- 03
Likelihood as a function of the parameter with observed data fixed
- 04
Log likelihood equivalence through strict monotonicity of the logarithm
- 05
Score function and first order condition for an interior candidate
- 06
Hessian, concavity and second order verification
- 07
Parameter boundaries, supremum and existence of an MLE
- 08
Numerical evaluation as a diagnostic rather than a substitute for proof
Worked example · free
Derive a Poisson maximum likelihood estimator
- +2Independence gives the joint likelihood exp(-9 lambda) times lambda to the power 27, divided by the product of the observed factorials.
- +2The factorial product depends only on the fixed sample, so the log likelihood relevant to maximisation is -9 lambda plus 27 log lambda plus a constant.
- +2The score is -9 plus 27 divided by lambda. Setting it to zero gives lambda hat equal to 3.
- +1The second derivative is -27 divided by lambda squared, which is negative throughout the positive parameter space.
- +1Substitution makes the score zero and the global concavity confirms the unique maximum.
Key terms
- Statistical model
- A collection of probability distributions indexed by admissible parameter values, together with the sample and parameter spaces.
- Likelihood function
- The observed sample joint density or probability mass viewed as a function of the parameter. It compares parameter values rather than assigning them probabilities.
- Log likelihood
- The logarithm of the likelihood. It shares the same maximiser on the positive likelihood region and turns products into sums.
- Score function
- The first derivative or gradient of the log likelihood with respect to the parameter. A zero score is an interior candidate condition.
- Hessian matrix
- The matrix of second derivatives of a multivariate log likelihood. Negative definiteness supports a local maximum.
- Parameter space
- The set of parameter values permitted by the model. Its boundaries determine whether an endpoint can be an estimator.
Likelihood and Maximum Likelihood Estimation FAQ
Why can the log likelihood replace the likelihood?
The logarithm is strictly increasing for positive inputs, so it preserves the ordering of all likelihood values and therefore the location of the maximum. It also improves numerical stability by replacing products with sums.
Does a zero score prove that I found the MLE?
No. A zero score identifies an interior stationary point. You still need curvature, global concavity, objective comparisons or boundary analysis to establish that the candidate is the relevant maximum.
What is fixed inside a likelihood calculation?
The observed dataset is fixed and the parameter varies. Before observation, the same formula is a function of random variables; after observation, likelihood analysis compares admissible parameter values for that realised sample.
What happens when the maximum lies at a boundary?
Inspect the parameter space explicitly. A closed boundary can contain the maximum, while an open parameter space may have only a supremum approached by a sequence and no actual maximiser.
Exam move
Practise writing a complete derivation in the same order every time: model, joint density, likelihood, logarithm, score, solution, curvature and boundaries. Use a different distribution for each attempt so the procedure does not become a memorised Bernoulli calculation. After deriving the analytic answer, evaluate the score and objective numerically near the estimate to catch sign errors.
When software is used, code the log likelihood, reject inadmissible parameter values and record the final objective as well as the estimate. The assignment briefs reward detailed algebra, so keep sampling assumptions and the reason for every likelihood contribution visible. Build a weekly error log with separate columns for omitted constants, parameter-dependent terms, invalid domains, stationary points and boundary cases.
Rework each error on a fresh distribution two days later. For a timed drill, allow five minutes to define the model and parameter space, ten minutes for algebra, and five minutes for verification. Mark every term that can be removed during maximisation and explain why it is constant in the parameter rather than merely looking inconvenient.
Compare the objective on both sides of an interior candidate, and inspect each admissible endpoint when the space is closed. Finish by stating the estimate in the original parameterisation, including its units or interpretation, so a formally correct derivative becomes a complete statistical answer.
Working through Likelihood and Maximum Likelihood Estimation in ETC3400? Sia is AskSia’s AI Economics tutor — ask any ETC3400 Likelihood and Maximum Likelihood Estimation question and get a clear, step-by-step explanation grounded in how ETC3400 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
