STAT 400 Chap.8 Point Estimation with Likelihood and Moments
Point Estimation with Likelihood and Moments
From probability to statistics
The third content block turns the direction of reasoning around. Instead of a known model predicting data, we now have data and an unknown parameter. A point estimator is a statistic, a function of the sample, used to guess that parameter, and an estimate is its value for one particular data set.
The course teaches two construction methods and three ways of judging the result.
Maximum likelihood
The likelihood is the joint pmf or pdf of the observed data viewed as a function of the parameter. The maximum likelihood estimator is the parameter value that makes the observed data most probable.
In practice the recipe is fixed: write the likelihood as a product, take logarithms to turn it into a sum, differentiate, set the derivative to zero and solve, then confirm that the solution is a maximum. For many standard models the answer is the sample mean, for example the Bernoulli p, the Poisson λ and the exponential θ.
When the parameter sets the edge of the support, as for a uniform distribution on zero to θ, calculus fails and the maximum is found by inspecting the likelihood directly, which gives the sample maximum.
Method of moments
The method of moments equates population moments to sample moments and solves. With one parameter, setting E[X] equal to the sample mean is usually enough; with two, the second moment is added.
It is often quicker than likelihood and gives a useful cross-check, but the two methods can disagree, and neither is automatically better.
Judging an estimator
An estimator is unbiased when its expected value equals the parameter for every possible parameter value. Bias is the difference between the two, and the mean square error combines bias and variance as MSE = variance + bias².
The sample variance with divisor n − 1 is unbiased for σ², while the normal maximum likelihood estimator with divisor n is biased low. Mean square error is the fairer comparison, because a slightly biased estimator with much smaller variance can be closer to the truth on average.
Invariance and parameter spaces
Two practical points save marks.
First, the maximum likelihood estimator of a function of a parameter is that function of the estimator, so once the Poisson mean is estimated by the sample mean, the probability of a zero count is estimated by e raised to minus that mean.
Second, always respect the parameter space: a probability estimate must lie between zero and one, and a variance estimate cannot be negative.
What the course asks for
The objectives list explaining the rationale for each method, calculating MLEs and method of moments estimators for a given distribution, and calculating bias and mean square error, so expect both derivations and short explanations.
What this chapter covers
- 01
Estimators and estimates
- 02
Likelihood and log-likelihood
- 03
Maximum likelihood estimators
- 04
Method of moments
- 05
Unbiasedness and bias
- 06
Mean square error
Worked example · free
Maximum likelihood for a Poisson rate
- 2Likelihood: L(λ) = ∏ e^(−λ) λ^(x_i)/x_i!, so ln L = −nλ + (Σx_i) ln λ − Σ ln(x_i!).
- 2Differentiate: −n + Σx_i/λ = 0 gives λ̂ = x̄.
- 2Compute: Σx_i = 24 and n = 6, so λ̂ = 4; the second derivative −Σx_i/λ² is negative, confirming a maximum.
Key terms
- Point Estimator
- A statistic used to estimate an unknown parameter from a random sample.
- Likelihood Function
- The joint pmf or pdf of the observed data regarded as a function of the unknown parameter.
- Method of Moments
- An estimation method that sets population moments equal to the corresponding sample moments and solves for the parameters.
- Unbiased Estimator
- An estimator whose expected value equals the parameter it estimates for every value of that parameter.
- Mean Square Error
- The expected squared distance between an estimator and the parameter, equal to its variance plus its squared bias.
Point Estimation with Likelihood and Moments FAQ
Why do we take the logarithm of the likelihood?
The logarithm turns a product of many terms into a sum, which is far easier to differentiate, and because the logarithm is increasing it does not move the location of the maximum.
Can the MLE and the method of moments estimator be different?
Yes. They are different principles, so for many models they give different formulas and different estimates from the same data. Comparing their bias and mean square error decides which is preferable.
Is an unbiased estimator always the best choice?
Not necessarily. An unbiased estimator can have a large variance, while a slightly biased one can have a smaller mean square error and therefore be closer to the parameter on average.
Exam move
Write the four-step likelihood recipe at the top of a page and apply it to the Bernoulli, Poisson, exponential and a one-parameter density on (0, 1). For each, also find the method of moments estimator and note whether the two agree.
Working through Point Estimation with Likelihood and Moments in STAT 400? Sia is AskSia’s AI Statistics tutor — ask any STAT 400 Point Estimation with Likelihood and Moments question and get a clear, step-by-step explanation grounded in how STAT 400 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
