ECON4916 Chap.5 Likelihood and ARMA Estimation
Likelihood and ARMA Estimation
ARMA estimation is difficult because lagged innovations are not directly observed. For every candidate parameter vector, the model must recover a new residual path before a likelihood or conditional sum of squares can be evaluated. This chapter compares OLS, method of moments and maximum likelihood by the information and assumptions each uses.
A simple Gaussian example derives the score and curvature of a log-likelihood, then ARMA recursion shows how candidate coefficients and initial conditions generate shocks. The final clinic treats optimiser convergence as computational evidence rather than proof of a statistically adequate global optimum.
Students are asked to inspect objective shape, parameter boundaries, alternative starting values, residual behaviour and forecast performance before trusting an estimate. The resulting workflow joins mathematics, software and diagnostics without confusing their roles.
What this chapter covers
- 01
OLS and moment estimators
- 02
Likelihood and log-likelihood
- 03
Scores and curvature
- 04
Recursive shock recovery
- 05
Initial-condition choices
- 06
Optimisation and estimator diagnostics
Worked example · free
Gaussian mean by maximum likelihood
- +1Write the log-likelihood up to a constant as negative one half times the sum of squared deviations from mu.
- +1Differentiate to obtain the score equal to the sum of y_t minus mu.
- +1Set the score to zero so three times mu equals the sum of the observations.
- +1Substitute 1, 2 and 3 to obtain mu-hat = 2.
- +1Use the negative second derivative, -3, to confirm that the stationary point is a maximum.
Key terms
- Likelihood Function
- The joint model density of the observed sample viewed as a function of unknown parameters.
- Log-Likelihood
- The logarithm of likelihood, turning products into sums while preserving the maximising parameter.
- Score
- The derivative of log-likelihood with respect to a parameter or parameter vector.
- Curvature
- The local second-derivative behaviour indicating how sharply an objective bends around a candidate optimum.
- Generated Regressor
- An estimated quantity, such as a recovered lagged innovation, used inside another estimating equation.
- Initial Condition
- A presample value or shock assumption needed to begin a recursive model calculation.
Likelihood and ARMA Estimation FAQ
Why can AR coefficients be estimated by OLS?
Lagged outcomes are observed regressors, and the current innovation is uncorrelated with those past values under the model's assumptions. That makes a direct regression available. The conclusion still needs stationarity, rank and error conditions appropriate to the desired consistency and inference results.
Why are MA parameters harder to estimate?
Lagged innovations appearing in an MA equation are not observed. They must be recovered from observations using candidate parameter values, making the residual path and estimation objective nonlinear in those parameters. A different candidate coefficient produces a different generated shock sequence.
Does optimiser convergence prove a correct estimate?
No. Convergence means a numerical stopping rule was met. The solver may reach a local optimum, a flat region or a boundary, and the maintained model may still be misspecified. Try alternative starting values, inspect nearby objective values, check constraints and examine residual and forecast diagnostics.
Why take logarithms of likelihood?
Logarithms turn a product of many densities into a sum, which is more stable numerically and easier to differentiate. Because logarithm is strictly increasing, the parameter that maximises likelihood also maximises log-likelihood. Constants that do not depend on parameters can be omitted during optimisation.
Exam move
Organise estimation methods in a comparison table with four columns: observed inputs, sample criterion, key assumptions and diagnostic checks. For OLS, show the regression and orthogonality condition. For moments, write the population moment and its sample analogue. For likelihood, derive the density contribution, log it and identify whether the objective is maximised.
Rehearse a simple closed-form likelihood before opening the ARMA case so score and curvature retain a clear meaning. For ARMA recursion, pick candidate coefficients, an initial shock and a short artificial series, then recover every residual by hand. Repeat after changing one coefficient to see why a residual path cannot be reused across candidates.
In software, record starting values, parameter constraints, termination messages and objective values. Compare multiple starts and inspect a profile of the objective near the reported estimate. Finally, check implied moments, residual dependence and out-of-sample forecasts. A point estimate is persuasive only when its mathematical derivation, numerical optimisation and model diagnostics tell a coherent story.
Treat extra decimal places as formatting, not evidence of identification or economic importance.
Working through Likelihood and ARMA Estimation in ECON4916? Sia is AskSia’s AI Economics tutor — ask any ECON4916 Likelihood and ARMA Estimation question and get a clear, step-by-step explanation grounded in how ECON4916 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.