7SSGN110 Chap.5 Hypotheses, P-values and Assumption Checks
Hypotheses, P-values and Assumption Checks
Use tests as conditional decisions
A hypothesis test asks whether an observed statistic is unusually extreme under a specified null model. The null and alternative should name a population parameter and a direction before the output is seen. Alpha sets a long-run Type I error rate for the decision rule; it is not a measure of effect size.
Keeping those roles separate prevents a threshold from replacing the research question.
Interpret the p-value under the null
A p-value is the probability, assuming the null model and its conditions, of a statistic at least as extreme as the observed one. When it falls below alpha, reject the null at that level. Otherwise, say that the analysis cannot reject the null.
A large p-value does not prove no effect, while a small value is not the probability that the null is true or that a causal explanation is correct.
Report magnitude with the decision
Two results on opposite sides of 0.05 need not be environmentally different. Give the estimated effect, its unit and a confidence interval so the reader can judge direction, plausible magnitude and precision.
A tiny effect can be statistically detectable in a large sample, and an important effect can remain uncertain in a small sample. Statistical significance and substantive importance therefore require separate sentences.
Diagnose distinct assumptions
Normality, equal variance, linearity and independence are not interchangeable.
Histograms and QQ plots reveal distribution shape; residual plots can expose curvature or changing spread; design and identifiers determine independence. A Shapiro-Wilk result that does not reject normality is not proof of a perfect normal model, especially with a small sample.
Combine formal checks with plots, measurement knowledge and the robustness of the planned method.
Choose repairs before chasing thresholds
A justified transformation may stabilise variance or improve shape, but diagnostics must be repeated on the new scale and interpretation must acknowledge the change. Dependence needs a design-aware model or a narrower unit of analysis, not a cosmetic transformation.
Multiple related tests require a declared family and suitable control. The final conclusion returns to the variables, population and design, stating what the sample supports and what remains unresolved.
Balance the two error risks
A Type I error rejects a true null under the procedure; a Type II error fails to reject for a specified real alternative. Power depends on effect size, noise, sample size and design.
Lowering alpha without adding information can reduce false positives while increasing false negatives. Non-rejection may therefore reflect limited precision rather than evidence of equivalence.
Examine the confidence interval to see which effect sizes remain plausible, and avoid turning absence of threshold significance into an absence claim.
For revision, pair each tempting p-value claim with a correction that includes an estimate and design boundary. Change sample size while holding effect size stable and anticipate what can happen to significance and precision.
This exposes why a decision rule cannot substitute for magnitude, uncertainty or environmental relevance.
What this chapter covers
- 01
Null hypothesis
- 02
Alternative hypothesis
- 03
Alpha
- 04
P-value
- 05
Normality
- 06
Equal variance
- 07
Transformation
Worked example · free
Interpret a two-sided test
- 2Compare p with the prespecified alpha.
- 3Report the effect direction and interval.
- 3Qualify the design and causal scope.
Key terms
- Null Hypothesis
- A Null Hypothesis specifies the reference model used to generate a test statistic's comparison distribution.
- P-value
- A P-value is the null-model probability of a statistic at least as extreme as the observed one.
- Type I Error
- A Type I Error rejects a null hypothesis that is true within the specified model.
Hypotheses, P-values and Assumption Checks FAQ
What does p below alpha mean?
It means the observed statistic is sufficiently incompatible with the null model under the prespecified decision rule, so reject the null at that alpha level. It does not give the probability that the null hypothesis is true.
What should I write when p is large?
Write that the analysis cannot reject the null hypothesis at the chosen alpha. Report the estimate and confidence interval because a non-rejection may reflect a small effect, high noise, limited sample size or a genuinely negligible difference.
Does a normality test prove normal data?
No. A non-significant diagnostic means the test did not detect a departure at its power and sample size. Combine histograms and QQ plots with the diagnostic, and evaluate independence and variance separately from distribution shape.
Assessment move
For every output, cover the p-value and predict the correct conclusion from the design, estimate and interval. Then uncover p, compare it with alpha and audit the language for false probability claims, proof language or confusion between statistical and environmental importance.
Working through Hypotheses, P-values and Assumption Checks in 7SSGN110? Sia is AskSia’s AI Statistics tutor — ask any 7SSGN110 Hypotheses, P-values and Assumption Checks question and get a clear, step-by-step explanation grounded in how 7SSGN110 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.