University College London · FACULTY OF STATISTICS

PHAS0083 Chap.8 Hypothesis Testing and Likelihood Ratio Tests

- one subject, every graph, every model, every mark
7 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 8 of 9 · PHAS0083

Hypothesis Testing and Likelihood Ratio Tests

Hypothesis Testing and Likelihood Ratio Tests builds the framework for deciding between hypotheses about a population parameter. A test needs a null and an alternative hypothesis, a test statistic and a rule that divides the sample space into rejection and acceptance regions. Two errors are possible, and a single power function, the probability of rejecting at each parameter value, carries both rates.

The size is the largest power over the null region and a test has level alpha when that largest value does not exceed alpha; practice fixes the level and then seeks power, which physicists express as a confidence level.

The likelihood ratio test compares the maximised likelihood under the null with the unrestricted maximum, rejecting for small ratios or large values of minus twice the log ratio, and handles nuisance parameters by maximising over them in both places. The Neyman-Pearson lemma makes the likelihood ratio test most powerful for simple hypotheses, with no such guarantee for composite ones.

Because the null distribution of the statistic rarely has a formula, the chapter calibrates thresholds and power curves by simulation and explains why discrete counts cannot hit the level exactly.

In this chapter

What this chapter covers

  • 01

    Null and alternative hypotheses

  • 02

    Rejection regions and test statistics

  • 03

    Type I and Type II errors

  • 04

    Power function, size and level

  • 05

    Likelihood ratio tests and nuisance parameters

  • 06

    Neyman-Pearson lemma and most powerful tests

  • 07

    Calibrating thresholds by simulation

Worked example · free

Choose a count threshold above a known background

Q [5 marks]. A background-only count is Poisson with mean 4. A signal is claimed if the count reaches a threshold c. Find the smallest c giving level 0.05, its actual size, and the power if the total mean is 10. The marks shown are AskSia practice weighting, not a published UCL marking scheme.
The power function of the one-sided rule rises from 0.05 at the edge of the null to 0.91 at theta = 1, and stays below 0.05 deeper inside the null region.The power function of the one-sided rule rises from 0.05 at the edge of the null to 0.91 at theta = 1, and stays below 0.05 deeper inside the null region.-0.500.511.50.050.51true thetaprobability of rejectingpower at theta = 1is 0.91grey: the null regiontheta at most 0
The power function of the one-sided rule rises from 0.05 at the edge of the null to 0.91 at theta = 1, and stays below 0.05 deeper inside the null region.
  • 2Compute upper tails under the null: P(N at least 8) = 0.051 and P(N at least 9) = 0.021. This allocation is practice weighting, not a UCL marking scheme.
  • 1Level 0.05 requires the tail to be at most 0.05, so c = 9 with actual size 0.021.
  • 1Power at total mean 10: P(N at least 9) = 0.667.
  • 1Report both numbers: the discrete count makes the test conservative, and a third of such signals would be missed.
The threshold is 9 counts, with actual size 0.021 rather than 0.05, and power 0.667 against a total mean of 10. The jump from 0.051 to 0.021 shows why a discrete statistic cannot reach the nominal level exactly. The practice marks are not an official UCL allocation.
Sia tip — Compute the tail at the candidate threshold and at the one below it; the level condition is decided by comparing both with alpha.
Glossary

Key terms

Rejection region
The set of sample values for which the test rejects the null hypothesis.
Power function
The probability of rejecting the null hypothesis, as a function of the true parameter.
Nuisance parameter
A parameter not under test whose unknown value must still be handled, usually by maximising over it.
FAQ

Hypothesis Testing and Likelihood Ratio Tests FAQ

Where should I simulate when the null hypothesis is composite?

The threshold must keep the level for every parameter value in the null. When the power function increases monotonically towards the alternative, the largest rejection probability inside the null occurs at its edge, so simulating at that boundary value is enough. Without monotonicity the whole null region would have to be scanned.

What is the difference between the size and the level of a test?

The size is the largest probability of rejecting over all parameter values in the null hypothesis. A test has level alpha if its size is at most alpha. Level is the term used in practice because with discrete data an exact size is often impossible.

Does a 5% level control the chance of missing a real effect?

No. The level controls only the Type I error rate. The probability of missing an effect is one minus the power at the true parameter value, and it must be computed separately for the alternatives of interest.

Why calibrate a likelihood ratio test by simulation?

The threshold must give the required level, which needs the distribution of the statistic when the null is true. For most realistic models that distribution has no closed form, so many data sets are simulated under the null and the threshold is read from their sorted statistics.

Study strategy

Exam move

Learn the framework vocabulary until you can define each term in one sentence: hypothesis, statistic, rejection region, Type I and Type II error, power function, size, level. Then compute power functions for two simple rules on a binomial or Poisson count and decide which is better and at what level each operates.

Work likelihood ratio statistics for three cases, a normal mean with known variance, a normal mean with unknown variance, and a Poisson signal over a known background, and always state the rejection direction explicitly.

Write out the simulation recipe for calibration in four lines, including where to simulate a composite null, and then implement it once for a case with a known answer so you can see the simulated threshold agree. Finish with discrete examples where the achievable size jumps, and practise reporting the actual size.

Working through Hypothesis Testing and Likelihood Ratio Tests in PHAS0083? Sia is AskSia’s AI Statistics tutor — ask any PHAS0083 Hypothesis Testing and Likelihood Ratio Tests question and get a clear, step-by-step explanation grounded in how PHAS0083 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 32 of your subjects at University College London, plus 1,000+ more Bibles across the AskSia library.
Sia - your PHAS0083 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works