PHAS0083 Chap.7 Point Estimation and Maximum Likelihood
Point Estimation and Maximum Likelihood
Point Estimation and Maximum Likelihood asks how to choose a single value for a parameter and how to judge the choice. An estimator is a statistic, so it has a sampling distribution; its bias measures accuracy, its variance measures precision, and the mean squared error combines them as variance plus squared bias.
Centred is not the same as best: a slightly biased estimator can have the smaller mean squared error, as the comparison of the two variance estimators shows, and a variance is estimated far less precisely than a mean from the same sample. The likelihood is the joint pdf or pmf of the observed data read as a function of the parameter; it is not a probability distribution over the parameter.
The maximum likelihood estimate is the parameter value at its peak, found analytically by setting derivatives of the log-likelihood to zero and checking the maximum and the boundaries, or numerically by minimising the negative log-likelihood.
The chapter introduces sufficiency and the likelihood principle, and states that under regularity conditions such estimators converge to the truth and reach the smallest large-sample variance, while possibly biased at finite sample size.
What this chapter covers
- 01
Point estimators as statistics
- 02
Bias, variance and mean squared error
- 03
The likelihood function
- 04
Maximum likelihood estimation
- 05
Numerical minimisation of the negative log-likelihood
- 06
Sufficiency and the likelihood principle
- 07
Consistency and efficiency
Worked example · free
Estimate a detection efficiency by maximum likelihood
- 1Log-likelihood: l(p) = 14 log p + 26 log(1 - p) plus a constant. This allocation is practice weighting, not a UCL marking scheme.
- 2Set the derivative 14/p - 26/(1 - p) to zero, giving p hat = 14/40 = 0.35.
- 1The second derivative is negative for every p, so this is a maximum.
- 1The binomial variance of the estimator is p(1 - p)/n, about 0.35 times 0.65 over 40, giving a standard deviation of 0.075.
Key terms
- Mean squared error
- The expected squared distance between an estimator and the parameter, equal to variance plus squared bias.
- Likelihood function
- The joint pmf or pdf of the observed data viewed as a function of the parameter.
- Sufficient statistic
- A statistic given which the conditional distribution of the sample no longer depends on the parameter.
- Consistent estimator
- An estimator that converges in probability to the parameter at every parameter value.
Point Estimation and Maximum Likelihood FAQ
What should I check after setting the derivative of the log-likelihood to zero?
Check that the stationary point is a maximum, using the second derivative or the shape of the curve, and that it lies inside the allowed parameter range. If the likelihood keeps increasing towards a boundary, as for a uniform upper limit or a non-negative signal rate, the maximum sits on that boundary instead.
Is the likelihood a probability distribution for the parameter?
No. It is the probability or density of the fixed observed data at each parameter value. Its area over the parameter has no meaning and is usually not one, so it should not be integrated or normalised as if it described the parameter.
Why minimise the negative log-likelihood instead of maximising the likelihood?
Products of many densities underflow numerically, while sums of logarithms are stable. The logarithm is increasing, so the maximum is in the same place, and standard optimisers are written to minimise, hence the negative sign.
Is the maximum likelihood estimator always unbiased?
No. The maximum likelihood estimate of a normal variance divides by n and is biased low by sigma squared over n. Under regularity conditions these estimators converge to the true value and are asymptotically efficient, but at finite sample size they can carry bias.
Exam move
Write likelihoods for every family in Chapter 2 until it is automatic: product of densities, logarithm, sum. For each, differentiate, solve, check the second derivative and look for a boundary maximum, then confirm the answer numerically with a short optimiser script on simulated data.
Separately, practise estimator evaluation: compute the bias, variance and mean squared error of two competing estimators analytically for one case, then by simulating many samples of the real size, and compare. Keep the vocabulary precise, since definition questions test it: estimator versus estimate, likelihood versus probability, consistency versus unbiasedness.
Finally sketch log-likelihood curves for a small and a large sample and describe in words how the width of the peak relates to precision.
Working through Point Estimation and Maximum Likelihood in PHAS0083? Sia is AskSia’s AI Statistics tutor — ask any PHAS0083 Point Estimation and Maximum Likelihood question and get a clear, step-by-step explanation grounded in how PHAS0083 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.