PSY2041 Chap.5 Hypothesis Testing
Hypothesis Testing
Hypothesis Testing compares sample evidence with a reference distribution implied by a null model. The test statistic places the observed result in that distribution, while the p value measures how unusual an equally or more extreme result would be if the null and assumptions held. A decision to reject or fail to reject is not proof.
Error rates, power, effect magnitude, confidence intervals, design and assumptions complete the statistical conclusion.
What this chapter covers
- 01
Null and alternative hypotheses as population claims
- 02
Sampling distributions and standardised test statistics
- 03
P values, alpha levels and decision language
- 04
Type I error, Type II error and statistical power
- 05
One-sample, independent-samples and paired-samples t designs
- 06
Correlation tests for population association
- 07
Confidence intervals, precision and null values
Worked example · free
Select the correct t procedure
- 2One sample mean compared with a reference value calls for a one-sample t test.
- 2Two unrelated groups contributing one score each call for an independent-samples t test.
- 2The same participants measured twice produce linked differences and call for a paired-samples t test.
Key terms
- Null hypothesis
- The population claim used to derive the reference distribution for a statistical test.
- Test statistic
- A standardised sample result whose distribution under the null is known or modelled.
- P value
- The probability under the null model of a result at least as extreme as the observed result.
- Type I error
- Rejecting a null hypothesis that is true in the population.
- Type II error
- Failing to reject a null hypothesis that is false in the population.
- Statistical power
- The probability of detecting an effect of a specified size when it exists.
Hypothesis Testing FAQ
What does a p value actually mean?
It is a probability about possible data under a null model, assuming the analysis conditions. It is not the probability that the null hypothesis is true, the probability of replication or a direct measure of effect size.
What should I write when the result is not significant?
State that the analysis failed to reject the null at the specified alpha level. Do not claim that groups are identical or that the null is proven. Report the estimate and confidence interval to show which effects remain compatible with the data.
How do paired and independent t tests differ?
A paired test analyses linked differences, such as the same people measured twice. An independent test compares scores from different participants in separate groups. Both concern mean differences and use t reference distributions, but their designs and error terms differ.
Exam move
Draw the full inference chain from population claim to sample, test statistic, null distribution, p value and decision. Practise correcting three prohibited sentences: the p value is not the probability the null is true, failure to reject is not proof of no effect, and significance is not practical importance. Choose tests from the design before checking assumptions.
For error questions, cross reality with the statistical decision in a two-by-two table. For every numerical result, report statistical evidence, magnitude and precision as separate clauses. In extended answers, define assumptions in the abstract, explain why interpretation depends on them and then detail one named test. Create a design-identification deck before calculating anything.
Each card should specify the outcome, comparison, number of groups and whether observations are independent or paired. Say one-sample, independent-samples or paired-samples reasoning aloud and explain which difference the statistic represents. Next, draw a null distribution and place several hypothetical statistics on it.
Shade equally or more extreme outcomes in the direction required by the alternative so the p value remains a conditional probability about data under the null model. Rewrite common errors into precise alternatives, especially after a nonsignificant result. Failing to reject may reflect a small effect, noisy measurement, limited power or a genuinely negligible difference; the decision alone cannot choose among them.
Use the error table to connect alpha with Type I risk and power with detecting effects that exist. Change sample size while holding an effect constant and predict what happens to uncertainty and the p value. Pair each test decision with an effect estimate and confidence interval so evidence, magnitude and precision stay separate. Finally, audit assumptions by naming the data object involved and the consequence of violation.
A polished answer should identify the design, state the model and conditions, interpret the statistic and p value, report magnitude and uncertainty, and preserve the population boundary. Use one counterexample for every inference error: a large sample with a trivial effect, a small study with wide uncertainty and a paired design analysed as independent.
Explaining why each conclusion fails strengthens method selection more than memorising prohibited phrases.
Working through Hypothesis Testing in PSY2041? Sia is AskSia’s AI Psychology tutor — ask any PSY2041 Hypothesis Testing question and get a clear, step-by-step explanation grounded in how PSY2041 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
