University College London · FACULTY OF STATISTICS

PHAS0083 Chap.7 Point Estimation and Maximum Likelihood

- one subject, every graph, every model, every mark
7 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 7 of 9 · PHAS0083

Point Estimation and Maximum Likelihood

Point Estimation and Maximum Likelihood asks how to choose a single value for a parameter and how to judge the choice. An estimator is a statistic, so it has a sampling distribution; its bias measures accuracy, its variance measures precision, and the mean squared error combines them as variance plus squared bias.

Centred is not the same as best: a slightly biased estimator can have the smaller mean squared error, as the comparison of the two variance estimators shows, and a variance is estimated far less precisely than a mean from the same sample. The likelihood is the joint pdf or pmf of the observed data read as a function of the parameter; it is not a probability distribution over the parameter.

The maximum likelihood estimate is the parameter value at its peak, found analytically by setting derivatives of the log-likelihood to zero and checking the maximum and the boundaries, or numerically by minimising the negative log-likelihood.

The chapter introduces sufficiency and the likelihood principle, and states that under regularity conditions such estimators converge to the truth and reach the smallest large-sample variance, while possibly biased at finite sample size.

In this chapter

What this chapter covers

  • 01

    Point estimators as statistics

  • 02

    Bias, variance and mean squared error

  • 03

    The likelihood function

  • 04

    Maximum likelihood estimation

  • 05

    Numerical minimisation of the negative log-likelihood

  • 06

    Sufficiency and the likelihood principle

  • 07

    Consistency and efficiency

Worked example · free

Estimate a detection efficiency by maximum likelihood

Q [5 marks]. A detector registers 14 of 40 test pulses, each independently with probability p. Find the maximum likelihood estimate of p and an approximate standard deviation for it. The marks shown are AskSia practice weighting, not a published UCL marking scheme.
For a normal sample of size 8, dividing the sum of squares by n adds a small bias but lowers the variance enough that its mean squared error, 0.234, beats the uFor a normal sample of size 8, dividing the sum of squares by n adds a small bias but lowers the variance enough that its mean squared error, 0.234, beats the unbiased 0.286.S squareddivide by n00.10.20.3mean squared error0.2860.234violet: varianceorange: bias squarednormal sample, n = 8,true variance 1
For a normal sample of size 8, dividing the sum of squares by n adds a small bias but lowers the variance enough that its mean squared error, 0.234, beats the unbiased 0.286.
  • 1Log-likelihood: l(p) = 14 log p + 26 log(1 - p) plus a constant. This allocation is practice weighting, not a UCL marking scheme.
  • 2Set the derivative 14/p - 26/(1 - p) to zero, giving p hat = 14/40 = 0.35.
  • 1The second derivative is negative for every p, so this is a maximum.
  • 1The binomial variance of the estimator is p(1 - p)/n, about 0.35 times 0.65 over 40, giving a standard deviation of 0.075.
The efficiency estimate is 0.35 with a standard deviation of about 0.075, so the measurement alone does not distinguish 0.30 from 0.40. The practice marks are not an official UCL allocation.
Sia tip — State the support of the parameter before maximising; a stationary point outside zero to one, or a maximum on the boundary, changes the answer.
Glossary

Key terms

Mean squared error
The expected squared distance between an estimator and the parameter, equal to variance plus squared bias.
Likelihood function
The joint pmf or pdf of the observed data viewed as a function of the parameter.
Sufficient statistic
A statistic given which the conditional distribution of the sample no longer depends on the parameter.
Consistent estimator
An estimator that converges in probability to the parameter at every parameter value.
FAQ

Point Estimation and Maximum Likelihood FAQ

What should I check after setting the derivative of the log-likelihood to zero?

Check that the stationary point is a maximum, using the second derivative or the shape of the curve, and that it lies inside the allowed parameter range. If the likelihood keeps increasing towards a boundary, as for a uniform upper limit or a non-negative signal rate, the maximum sits on that boundary instead.

Is the likelihood a probability distribution for the parameter?

No. It is the probability or density of the fixed observed data at each parameter value. Its area over the parameter has no meaning and is usually not one, so it should not be integrated or normalised as if it described the parameter.

Why minimise the negative log-likelihood instead of maximising the likelihood?

Products of many densities underflow numerically, while sums of logarithms are stable. The logarithm is increasing, so the maximum is in the same place, and standard optimisers are written to minimise, hence the negative sign.

Is the maximum likelihood estimator always unbiased?

No. The maximum likelihood estimate of a normal variance divides by n and is biased low by sigma squared over n. Under regularity conditions these estimators converge to the true value and are asymptotically efficient, but at finite sample size they can carry bias.

Study strategy

Exam move

Write likelihoods for every family in Chapter 2 until it is automatic: product of densities, logarithm, sum. For each, differentiate, solve, check the second derivative and look for a boundary maximum, then confirm the answer numerically with a short optimiser script on simulated data.

Separately, practise estimator evaluation: compute the bias, variance and mean squared error of two competing estimators analytically for one case, then by simulating many samples of the real size, and compare. Keep the vocabulary precise, since definition questions test it: estimator versus estimate, likelihood versus probability, consistency versus unbiasedness.

Finally sketch log-likelihood curves for a small and a large sample and describe in words how the width of the peak relates to precision.

Working through Point Estimation and Maximum Likelihood in PHAS0083? Sia is AskSia’s AI Statistics tutor — ask any PHAS0083 Point Estimation and Maximum Likelihood question and get a clear, step-by-step explanation grounded in how PHAS0083 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 32 of your subjects at University College London, plus 1,000+ more Bibles across the AskSia library.
Sia - your PHAS0083 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works