The Hong Kong Polytechnic University · FACULTY OF STATISTICS

RS517 Chap.5 Hypothesis Tests, Errors and Power

- one subject, every graph, every model, every mark
6 Chapters3-page Bible
Our own words - no uploaded lecturer files
Updated for this semester
Chapter 5 of 10 · RS517

Hypothesis Tests, Errors and Power

A decision under a model

The taught sequence states the null, sets alpha, chooses and computes a statistic, obtains a p-value and compares it with alpha. The outcome is to reject or not reject the null, never to prove a hypothesis true.

Errors trade against information

Type I error rejects a true null, while Type II error fails to reject a false null.

Lowering alpha at fixed information reduces the first risk and increases the second. Larger samples, reliable outcomes and efficient designs can improve power without the same trade-off.

Intervals preserve scale

A p-value describes compatibility with a null model, not effect magnitude or the probability the null is true. Confidence intervals show the estimate and precision on the clinical scale.

Multiplicity requires a defined family of tests and a control strategy matched to confirmatory or exploratory aims.

Define the parameter before the hypothesis

A null hypothesis concerns a parameter under a model, such as a mean difference of zero or an odds ratio of one. It is not automatically the claim that nobody benefits, the distributions are identical or the treatment is clinically useless.

The alternative should reflect the scientific question. A one-sided form is defensible only when an effect in the opposite direction would not alter the decision. State the effect scale before reading the test result.

Interpret estimates, intervals and p-values together

The p-value measures compatibility between the observed statistic and the null model, conditional on assumptions.

It does not give the probability that the null is true and does not quantify effect size. A confidence interval displays effects compatible with the procedure and data; compare it with zero and a clinically important threshold.

A narrow biased interval remains wrong, while a wide interval can honestly reveal that the study cannot distinguish harm, trivial benefit and meaningful benefit.

Power is a design property, not a post hoc excuse

Power depends on the specified effect, variability, sample size, allocation, measurement reliability and analysis.

A Type I error rejects a true null under the model; a Type II error misses the specified alternative. Increasing sample size can reduce random uncertainty but cannot remove confounding or measurement bias. Post hoc power calculated from the observed estimate usually restates the p-value and adds little.

Discuss precision and the effects still compatible with the interval instead.

Revision drill

Take an interval that crosses zero and a clinical threshold. Write the statistical decision, the range of clinically plausible effects and the feature most likely to improve information. Then change only sample size, only measurement reliability and only the assumed target effect.

Explain how each change affects planned power and what it cannot repair.

In this chapter

What this chapter covers

  • 01

    Null and alternative hypotheses

  • 02

    Alpha and the testing sequence

  • 03

    Type I and Type II errors

  • 04

    Power and sample information

  • 05

    Confidence intervals and clinical thresholds

  • 06

    Multiplicity and analysis families

Worked example · free

Interpret an inconclusive interval

Q [8 marks]. Practice note: the marks used here are not part of the university's published assessment scheme. A difference is 2.4 points, with a 95% interval from -0.6 to 5.4 and p = 0.11. Four points is clinically important.
The two-by-two grid shows that rejecting and not rejecting can each be correct or create a different inferential error.The two-by-two grid shows that rejecting and not rejecting can each be correct or create a different inferential error.True stateDecisionH0 trueH0 falsedo not reject: correctdo not reject: Type IIreject: Type Ireject: correct
The two-by-two grid shows that rejecting and not rejecting can each be correct or create a different inferential error.
  • 3State the statistical decision without accepting the null.
  • 5Interpret the interval against zero and the clinical threshold.
The data do not provide enough evidence to reject a zero difference at the chosen threshold. The interval includes no effect and improvements beyond the four-point clinical threshold, so the result remains inconclusive for both statistical and clinical decisions rather than proving no effect.
Sia tip — Read the estimate and interval before using the significance label.
Glossary

Key terms

Null Hypothesis
The specified model claim tested against the observed statistic.
Type I Error
Rejecting a true null hypothesis under the stated model.
Type II Error
Failing to reject the null when the specified alternative is true.
Power
The probability of rejecting the null under a specified alternative and design.
Confidence Interval
A range produced by a procedure calibrated to cover the target parameter at a stated long-run rate.
FAQ

Hypothesis Tests, Errors and Power FAQ

What does p less than 0.05 mean?

Under the null model and analysis assumptions, a result at least as incompatible with that null as the observed result would occur less than five per cent of the time. It does not mean the null has less than five per cent probability or that the effect is important.

Can a non-significant result prove equivalence?

No. Failure to reject zero can reflect a small effect or imprecision. Equivalence requires prespecified acceptable margins and a design capable of excluding effects outside them. Examine the interval against those margins instead of treating the significance label as evidence of sameness.

How can power be increased?

Increase information through sample size, reliable measurement, balanced allocation, repeated measures used efficiently or stronger design control. Raising alpha also increases the rejection probability but accepts more Type I error. Power must be planned for a meaningful effect and plausible variability before the data are seen.

Study strategy

Exam move

Draw the truth-by-decision grid until Type I and Type II errors are automatic. For every result, write five clauses in order: estimate, confidence interval, p-value decision, clinical threshold and design limit. Practise correcting probability reversals and any sentence that accepts the null.

Working through Hypothesis Tests, Errors and Power in RS517? Sia is AskSia’s AI Statistics tutor — ask any RS517 Hypothesis Tests, Errors and Power question and get a clear, step-by-step explanation grounded in how RS517 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.

A+Everything unlocked
Unlocks this Bible + all 24 of your subjects at The Hong Kong Polytechnic University, plus 1,000+ more Bibles across the AskSia library.
Sia - your RS517 tutor, unlimited, worked the way the exam marks it
The full 3-page Bible + practice bank with worked solutions
Chrome extension - sync your LMS so Sia knows your deadlines
Bilingual EN / Chinese on every Bible and every Sia answer
$0.99 Trial
30-day money-back · cancel in one tap · how it works