ETC3400 Chap.6 Numerical Likelihood Optimisation
Numerical Likelihood Optimisation
Numerical Likelihood Optimisation treats software output as a result that must be audited. Newton Raphson uses the score and Hessian to predict the next root, while general routines such as BFGS search using objective and gradient information. Starting values, scaling, curvature and parameterisation affect the path and computational effort.
Log transforming a positive scale parameter turns a constrained problem into an unconstrained one without changing the statistical model. Joint, concentrated and analytical solutions should agree when they target the same global likelihood maximum, making their comparison a powerful code check.
What this chapter covers
- 01
Newton Raphson as a local linear approximation to the score
- 02
General numerical minimisation of the negative log likelihood
- 03
Starting values and basins of attraction
- 04
Convergence codes, final gradients and objective values
- 05
Positive scale parameters and log transformations
- 06
Bound constrained versus unconstrained algorithms
- 07
Concentrated likelihood after solving nuisance parameters analytically
- 08
Agreement among analytical, joint and concentrated solutions
Worked example · free
Reparameterise a positive standard deviation
- +2Define lambda=log sigma, so sigma=exp lambda is positive for every real lambda.
- +2Substitute exp lambda into the log likelihood and minimise its negative over an unrestricted real lambda.
- +1Transform the solution back with sigma hat=exp lambda hat and evaluate the original log likelihood there.
- +2Compare estimates, maximised objective values, convergence codes, final gradients and evaluation counts with the bounded direct optimisation.
- +1Recommend the parameterisation based on validity, stability and effort rather than on the coefficient vector alone.
Key terms
- Newton Raphson method
- An iterative root finding method that updates a parameter using the current score divided by its derivative or Hessian.
- Starting value
- The initial parameter vector supplied to an optimiser, which can affect its route, effort and located solution.
- Convergence code
- A software status indicating how an optimisation ended. It must be read together with gradients and objective values.
- Parameterisation
- A one to one representation of model parameters that can change numerical geometry without changing the underlying statistical model.
- Concentrated likelihood
- A lower dimensional objective obtained after maximising analytically over nuisance parameters conditional on parameters of interest.
- BFGS
- A quasi Newton optimisation method that builds a curvature approximation from successive objective and gradient information.
Numerical Likelihood Optimisation FAQ
Does a successful convergence code prove the estimate is correct?
No. It reports the algorithm status, not the validity of the model, derivatives or located optimum. Check the final gradient, restrictions, objective value and agreement across independent solution routes.
Why optimise a log scale parameter?
A logarithm maps every positive scale to the full real line and its inverse automatically returns a valid positive value. This avoids invalid negative steps and often improves numerical scaling.
Why run the optimiser from several starting values?
Agreement across materially different starts provides evidence that the same solution basin was reached and reveals how starting points change computational effort. It does not alone prove global optimality.
What should match between joint and concentrated optimisation?
After transforming estimates to a common parameterisation, the parameter estimates and maximised likelihood should agree within numerical tolerance. A material gap signals inconsistent code, signs or restrictions.
Exam move
Implement the likelihood twice when possible: one direct joint version and one analytical or concentrated benchmark. Test both on a small dataset where outputs can be inspected. Record starting vectors, algorithm names, bounds, convergence codes, objective values, final gradients and function or gradient counts. Run several starts and compare transformed results at full precision.
For a positive parameter, code both a bounded direct version and a log scale version, then explain why they describe the same model. Finish by evaluating the analytical score at the numerical estimate and checking that it is small relative to the local scale of score changes. Keep a reproducibility table for every run rather than copying only the most favourable output.
Include the data checksum or sample summary, software version, seed where randomness is used, parameter order and whether the routine minimises or maximises. Before trusting a result, perturb each coordinate around the reported estimate and confirm the objective behaves as expected. Compare analytical and finite-difference gradients on a small case, then investigate any scale-dependent discrepancy.
When two parameterisations are used, transform estimates back before comparing objectives and scores. Treat warnings, boundary solutions and singular Hessians as evidence to diagnose, not messages to suppress. A final write-up should separate model validity, implementation correctness, optimiser convergence and statistical interpretation because success in one layer cannot certify the others.
Working through Numerical Likelihood Optimisation in ETC3400? Sia is AskSia’s AI Economics tutor — ask any ETC3400 Numerical Likelihood Optimisation question and get a clear, step-by-step explanation grounded in how ETC3400 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
