DSC6008 Chap.3 Hypothesis Tests, Intervals and Error Rates
Hypothesis Tests, Intervals and Error Rates
Make the reference claim explicit
A hypothesis test begins with a parameter and a null value, not with software output. The statistic measures distance from that null in estimated standard-error units, and the p-value locates the observed distance in its null reference distribution.
The significance level sets the long-run Type I error rate for the decision rule.
Build the statistic from estimate, null and uncertainty
The numerator is the estimated effect minus the value claimed under the null. The denominator is the standard error justified by the design and model. A large absolute statistic can arise from a large effect, precise data or both.
Writing those pieces before opening R reveals wrong units, a mistaken independent sample size or a null value that does not match the research question.
Interpret a p-value inside the null world
A p-value is calculated assuming the null model and asks how often that model would produce a statistic at least as extreme as observed.
It does not assign a probability to the hypothesis, measure the effect or state the chance that the result is due to randomness. The selected tail must match the alternative fixed before the data were inspected. Switching direction afterward changes the procedure's error rate.
Join intervals to the same model
A confidence interval combines an estimate with a critical value and standard error.
For a matched two-sided test and interval, exclusion of the null value agrees with rejection at the corresponding significance level. The interval also displays plausible effect sizes, which keeps statistical detectability separate from scientific importance.
A wide interval crossing both zero and an important effect supports uncertainty, not proof of no effect.
Name both ways a decision can fail
A Type I error rejects a true null, while a Type II error fails to reject at a specified alternative. Alpha controls the first rate through the rejection rule.
Power equals one minus the Type II error probability at a particular alternative, so it cannot be reported without naming the effect. The practical costs of the two errors help justify alpha, target power and sample size.
Control families of related claims
Testing many comparisons and reporting only the smallest p-value raises the chance of a false discovery.
Define which conclusions form one family before choosing a correction. Planned contrasts answer focused questions, while broader pairwise procedures protect a larger family.
Adjusted evidence still needs an effect estimate and interval because multiplicity control does not create practical importance.
Write the conclusion in the scale of the experiment
State the estimated direction and magnitude, report uncertainty, give the test decision under the named rule and connect the result to the units represented by the design.
Avoid accept the null, since non-rejection can reflect limited precision. A short conclusion is credible only when the hypotheses, statistic, assumptions and error rule are already explicit.
What this chapter covers
- 01
Null hypotheses
- 02
Test statistics
- 03
P-values
- 04
Type I errors
- 05
Confidence intervals
- 06
Practical significance
Worked example · free
Test a process mean without misreading the p-value
- 2Compute (10.8 - 10) / 0.32 = 2.50.
- 2Compare the stated p-value 0.018 with the chosen significance level.
- 3Interpret the probability under the null rather than as a probability that the null is true.
Key terms
- Null Hypothesis
- The parameter claim used to construct the reference distribution for a test.
- Type I Error
- Rejecting the null hypothesis when it is true.
- Confidence Interval
- An estimate plus and minus a reference multiplier times its standard error.
Hypothesis Tests, Intervals and Error Rates FAQ
Is a p-value the probability that the null is true?
No. It is calculated under the assumption that the null model is true and measures how often that model would produce a statistic at least as extreme as the observed one. A probability assigned to the hypothesis itself would require a different inferential framework.
Why report an interval as well as a test?
A test gives a decision relative to one null value and significance rule. An interval displays a range of parameter values compatible with the data and model, allowing the reader to judge magnitude and precision rather than only whether a threshold was crossed.
What determines a Type I error rate?
It is controlled by the rejection rule chosen before seeing the data, typically through the significance level. Observing a small p-value does not reveal whether this particular decision is an error. The rate describes repeated use of the procedure when the null is true.
Assessment move
Use a four-line template: define the parameter, state the null and alternative, compute the statistic from estimate minus null divided by standard error, and interpret the tail probability under the null. Add an interval whenever magnitude matters, and compare its scale with a scientifically meaningful effect.
Working through Hypothesis Tests, Intervals and Error Rates in DSC6008? Sia is AskSia’s AI Statistics tutor — ask any DSC6008 Hypothesis Tests, Intervals and Error Rates question and get a clear, step-by-step explanation grounded in how DSC6008 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
