ETC3400 Chap.3 Information and Minimum Variance Bounds
Information and Minimum Variance Bounds
Information and Minimum Variance Bounds connects likelihood curvature with the precision available from a sample. Under regularity conditions, Fisher information can be expressed as score variance or as negative expected log likelihood curvature. Independent observations add information, producing the familiar inverse sample size variance scale.
The Cramer Rao inequality turns information into a lower variance bound for unbiased estimators of a differentiable target. Attainment requires an explicit equality comparison. The chapter also shows why censoring changes the likelihood contribution and usually reduces information by collapsing several possible exact outcomes into one recorded event.
What this chapter covers
- 01
Score mean and variance under regular likelihood models
- 02
Observed information from sample curvature
- 03
Expected Fisher information and its two equivalent formulas
- 04
Additivity of information for independent observations
- 05
Scalar Cramer Rao lower bound for an unbiased target estimator
- 06
Attainment and finite sample efficiency within a reference class
- 07
Positive semidefinite covariance ordering for vector targets
- 08
Information loss when exact observations are censored
Worked example · free
Derive and attain the Bernoulli information bound
- +2The one observation score is Y divided by p minus (1-Y) divided by (1-p).
- +2Using E[Y]=p gives one observation information 1 divided by p(1-p).
- +1Independence gives sample information n divided by p(1-p).
- +1The unbiased Cramer Rao bound for p is p(1-p) divided by n.
- +2The sample proportion has exactly this variance, so it attains the bound among unbiased estimators under the stated regular model.
Key terms
- Fisher information
- Expected local curvature of the log likelihood, equivalently score variance when the regularity identity is valid.
- Observed information
- Negative sample Hessian evaluated from the realised data, rather than averaged over repeated samples.
- Cramer Rao bound
- A lower variance bound for unbiased estimators of a differentiable target under stated regularity conditions.
- Efficient estimator
- An estimator that attains the relevant lower bound within the specified comparison class and model.
- Positive semidefinite order
- A covariance matrix ordering defined through nonnegative quadratic forms for every vector, covering all linear combinations.
- Right censoring
- An observation scheme that records an exact value below a threshold and only the event of exceeding that threshold otherwise.
Information and Minimum Variance Bounds FAQ
Why are there two formulas for Fisher information?
Regular likelihood identities make expected squared score equal negative expected second derivative. The equality can fail in irregular settings, so its conditions matter whenever support or differentiation behaves unusually.
Does attaining the bound make an estimator best in every sense?
No. It establishes minimum variance within the relevant unbiased class under the model. A biased estimator can still have lower mean squared error, and another loss function can rank procedures differently.
Why does information add across observations?
Independent log likelihood contributions add, so their scores add. Cross terms vanish under zero expected scores, leaving the sum of individual information contributions. Thus n identically distributed observations usually carry n times the one-observation information under the regular model.
How does censoring affect a likelihood?
Exact records contribute densities, while censored records contribute event probabilities such as survival functions. Treating a censored value as an exact point uses the wrong observed data model and misstates information.
Exam move
Derive information from both score variance and expected curvature in one simple model, then check that the answers agree. When using the Cramer Rao result, list the target, the unbiased estimator class and the regularity assumptions before writing the bound. For vector questions, translate a matrix inequality into a quadratic form and explain why this compares every linear combination rather than only diagonal variances.
Rebuild a censored likelihood from the recording rule and label the density and survival contributions. Finish each efficiency answer with the candidate estimator variance and an equality or strict inequality statement. This sequence matches the detailed derivation style rewarded in the assignment material.
Maintain a condition checklist beside every calculation: parameter-independent support, permissible differentiation under the integral, zero expected score, finite information and an identified target. If the two information formulas disagree, return to those conditions before repairing algebra.
For censoring exercises, draw the data-recording map first and write the probability attached to every recorded outcome; only then differentiate. Practise transforming the parameter and verify that information changes by the squared derivative while the resulting variance comparison remains coherent. In matrix problems, test a proposed covariance ordering with an arbitrary contrast vector and state its dimension.
A final timed answer should name the reference class, calculate the bound, calculate the candidate variance, compare them for all admissible parameter values and avoid extending the conclusion to biased procedures or unrelated loss functions.
Working through Information and Minimum Variance Bounds in ETC3400? Sia is AskSia’s AI Economics tutor — ask any ETC3400 Information and Minimum Variance Bounds question and get a clear, step-by-step explanation grounded in how ETC3400 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
