PHAS0083 Chap.8 Hypothesis Testing and Likelihood Ratio Tests
Hypothesis Testing and Likelihood Ratio Tests
Hypothesis Testing and Likelihood Ratio Tests builds the framework for deciding between hypotheses about a population parameter. A test needs a null and an alternative hypothesis, a test statistic and a rule that divides the sample space into rejection and acceptance regions. Two errors are possible, and a single power function, the probability of rejecting at each parameter value, carries both rates.
The size is the largest power over the null region and a test has level alpha when that largest value does not exceed alpha; practice fixes the level and then seeks power, which physicists express as a confidence level.
The likelihood ratio test compares the maximised likelihood under the null with the unrestricted maximum, rejecting for small ratios or large values of minus twice the log ratio, and handles nuisance parameters by maximising over them in both places. The Neyman-Pearson lemma makes the likelihood ratio test most powerful for simple hypotheses, with no such guarantee for composite ones.
Because the null distribution of the statistic rarely has a formula, the chapter calibrates thresholds and power curves by simulation and explains why discrete counts cannot hit the level exactly.
What this chapter covers
- 01
Null and alternative hypotheses
- 02
Rejection regions and test statistics
- 03
Type I and Type II errors
- 04
Power function, size and level
- 05
Likelihood ratio tests and nuisance parameters
- 06
Neyman-Pearson lemma and most powerful tests
- 07
Calibrating thresholds by simulation
Worked example · free
Choose a count threshold above a known background
- 2Compute upper tails under the null: P(N at least 8) = 0.051 and P(N at least 9) = 0.021. This allocation is practice weighting, not a UCL marking scheme.
- 1Level 0.05 requires the tail to be at most 0.05, so c = 9 with actual size 0.021.
- 1Power at total mean 10: P(N at least 9) = 0.667.
- 1Report both numbers: the discrete count makes the test conservative, and a third of such signals would be missed.
Key terms
- Rejection region
- The set of sample values for which the test rejects the null hypothesis.
- Power function
- The probability of rejecting the null hypothesis, as a function of the true parameter.
- Nuisance parameter
- A parameter not under test whose unknown value must still be handled, usually by maximising over it.
Hypothesis Testing and Likelihood Ratio Tests FAQ
Where should I simulate when the null hypothesis is composite?
The threshold must keep the level for every parameter value in the null. When the power function increases monotonically towards the alternative, the largest rejection probability inside the null occurs at its edge, so simulating at that boundary value is enough. Without monotonicity the whole null region would have to be scanned.
What is the difference between the size and the level of a test?
The size is the largest probability of rejecting over all parameter values in the null hypothesis. A test has level alpha if its size is at most alpha. Level is the term used in practice because with discrete data an exact size is often impossible.
Does a 5% level control the chance of missing a real effect?
No. The level controls only the Type I error rate. The probability of missing an effect is one minus the power at the true parameter value, and it must be computed separately for the alternatives of interest.
Why calibrate a likelihood ratio test by simulation?
The threshold must give the required level, which needs the distribution of the statistic when the null is true. For most realistic models that distribution has no closed form, so many data sets are simulated under the null and the threshold is read from their sorted statistics.
Exam move
Learn the framework vocabulary until you can define each term in one sentence: hypothesis, statistic, rejection region, Type I and Type II error, power function, size, level. Then compute power functions for two simple rules on a binomial or Poisson count and decide which is better and at what level each operates.
Work likelihood ratio statistics for three cases, a normal mean with known variance, a normal mean with unknown variance, and a Poisson signal over a known background, and always state the rejection direction explicitly.
Write out the simulation recipe for calibration in four lines, including where to simulate a composite null, and then implement it once for a case with a known answer so you can see the simulated threshold agree. Finish with discrete examples where the achievable size jumps, and practise reporting the actual size.
Working through Hypothesis Testing and Likelihood Ratio Tests in PHAS0083? Sia is AskSia’s AI Statistics tutor — ask any PHAS0083 Hypothesis Testing and Likelihood Ratio Tests question and get a clear, step-by-step explanation grounded in how PHAS0083 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.