ETC3400 Chap.5 Vector Likelihood and Normal Linear Models
Vector Likelihood and Normal Linear Models
Vector Likelihood and Normal Linear Models extends the scalar likelihood argument to parameter vectors, score vectors, Hessian matrices and covariance ordering. In the classical normal linear model, full column rank makes the cross product matrix invertible and yields the maximum likelihood estimator of the regression coefficient vector.
Projection geometry explains fitted values, residuals and the n minus k residual degrees of freedom. Joint normality turns zero covariance between the coefficient estimator and residual vector into independence. The chapter then compares maximum likelihood and unbiased variance estimators through expectation, variance and mean squared error.
What this chapter covers
- 01
Score vector, Hessian matrix and multivariate information
- 02
Classical normal linear model Y equals X beta plus u
- 03
Full column rank and invertibility of X prime X
- 04
Maximum likelihood estimator of the coefficient vector
- 05
Projection and residual maker matrices
- 06
Zero covariance, joint normality and independence
- 07
Maximum likelihood and unbiased disturbance variance estimators
- 08
Finite sample bias, degrees of freedom and mean squared error
Worked example · free
Compare two disturbance variance estimators
- +2Use residual sum of squares divided by sigma squared having chi squared distribution with n-k degrees of freedom.
- +2Its expectation is (n-k) sigma squared, so dividing by n gives E[sigma hat squared ML]=((n-k)/n) sigma squared.
- +1Dividing by n-k instead gives expectation sigma squared, so s squared is unbiased.
- +2The ML bias is -k sigma squared divided by n and tends to zero when k is fixed and n grows.
- +1Compare mean squared errors before choosing between the estimators in a finite sample.
Key terms
- Score vector
- The gradient of a log likelihood with one component for each parameter in a vector model.
- Information matrix
- Expected negative Hessian or score covariance matrix that describes joint local precision under regularity conditions.
- Full column rank
- A design condition ensuring independent columns and invertibility of X prime X in the normal linear model.
- Residual maker
- The matrix M equal to I minus X(X prime X) inverse X prime, which projects responses into the residual space.
- Joint normality
- A multivariate normal relationship under which zero covariance implies independence for the relevant linear transformations.
- Degrees of freedom
- The dimension of available independent variation after fitted restrictions have been imposed; residual degrees of freedom equal n minus k.
Vector Likelihood and Normal Linear Models FAQ
Why should I write matrix dimensions?
Dimensions expose an invalid multiplication or transpose immediately and identify the shape of the final statistical object. They are especially useful for score vectors, covariance sandwiches and linear target transformations.
When does zero covariance imply independence?
It implies independence for jointly multivariate normal quantities. In the classical model, coefficient estimates and residuals are linear transformations of a normal response vector, which supplies that additional condition.
Why does the unbiased variance estimator divide by n minus k?
Estimating k regression coefficients uses k dimensions of the response variation. The residual projection has rank n minus k, so its quadratic form has that many chi squared degrees of freedom.
Can the biased ML variance estimator have lower MSE?
Yes. Its smaller denominator adjustment changes both variance and bias. Mean squared error combines both terms, so the biased estimator can outperform the unbiased one for some sample size and parameter dimension combinations.
Exam move
Begin each matrix exercise with a dimension ledger for Y, X, beta, residuals and every transformation matrix. Prove projection identities such as symmetry, idempotence and X prime M equal to zero before using them in covariance calculations. Keep the covariance step separate from the normality step that establishes independence.
For variance estimators, derive expectation and variance from the residual chi squared distribution and calculate MSE rather than ranking procedures by bias alone. Rework one linear target A beta problem and verify that its covariance output is r by r. This approach matches the course emphasis on detailed, justified multivariate derivations.
Use coloured annotations to distinguish algebraic facts, distributional assumptions and inferential conclusions. A zero covariance calculation belongs to the first two categories; independence needs the joint normality statement written separately. Reconstruct the residual degrees of freedom geometrically by identifying the fitted subspace and its orthogonal complement, then connect that rank to the chi-squared result.
For every covariance sandwich, read its dimensions from right to left and confirm symmetry at the end. Compare the maximum-likelihood and unbiased variance estimators at two admissible pairs of n and k, retaining sigma to the fourth power throughout the MSE calculation. Finally, translate each row of a target matrix into words so coefficient-order mistakes appear before the formal proof.
Working through Vector Likelihood and Normal Linear Models in ETC3400? Sia is AskSia’s AI Economics tutor — ask any ETC3400 Vector Likelihood and Normal Linear Models question and get a clear, step-by-step explanation grounded in how ETC3400 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
