PSY2041 Chap.8 Effect Sizes and Test Assumptions
Effect Sizes and Test Assumptions
Effect Sizes and Test Assumptions completes the inferential picture by separating statistical evidence from magnitude and precision. Cohen's d expresses a mean difference in standard-deviation units, while correlation expresses association on a standardised scale. Sample size can make a tiny effect statistically significant, so p values cannot answer how much.
Assumptions identify the conditions that allow a statistic, standard error and reference distribution to support their advertised interpretation.
What this chapter covers
- 01
Effect magnitude compared with statistical significance
- 02
Cohen's d for standardised differences between means
- 03
Pearson correlation as covariance standardised by variability
- 04
Degrees of freedom and the t distribution
- 05
Independence, scale, shape, variance and linearity conditions
- 06
Outliers and their influence on statistical conclusions
- 07
Reporting evidence, magnitude and precision together
Worked example · free
Calculate a standardised mean difference
- 1Compute the stated mean difference: 28 minus 24 = 4 points.
- 1Divide by the pooled standard deviation: 4 divided by 8 = 0.50.
- 1Interpret direction: Group A scored higher than Group B.
- 1Interpret magnitude: the difference is half a pooled standard deviation, conventionally medium, with practical meaning still requiring context.
Key terms
- Effect size
- A quantitative expression of the magnitude of a difference or association.
- Cohen's d
- A difference between means expressed in standard-deviation units.
- Degrees of freedom
- The amount of independent information available for estimating variability in a statistical model.
- T distribution
- A reference distribution with heavier tails at low degrees of freedom that approaches the normal distribution as information increases.
- Statistical assumption
- A condition about data or design required for a procedure's interpretation to be defensible.
- Linearity
- The condition that an association follows an approximately straight-line form when using a linear coefficient.
Effect Sizes and Test Assumptions FAQ
Can a very small effect be statistically significant?
Yes. A large sample can estimate a small effect precisely enough to produce a small p value. Report the effect size and interval, then interpret practical importance in context. Significance does not convert a small magnitude into a large one.
What do assumptions do in a statistical test?
They connect the observed statistic to its standard error and reference distribution. If a relevant condition is badly violated, the p value or interval may not have the intended interpretation. The exact assumptions depend on the named procedure and design.
Why does the t distribution change with degrees of freedom?
When less information is available, estimating population variability adds uncertainty, represented by heavier tails. As degrees of freedom increase, the t distribution approaches the normal distribution and critical values become less extreme.
Exam move
For every statistical result, create three lines labelled evidence, magnitude and precision. Practise Cohen's d with both subtraction orders so the sign remains tied to named groups. Memorise the course benchmark sequence 0.2, 0.5 and 0.8, but always add that context determines importance. Build assumption cards for each analysis rather than one generic list.
Explain what data object the condition concerns, how a violation could alter interpretation and what plot or design check is relevant. Rehearse the marking-key structure for an assumptions paragraph: abstract definition, reason interpretation depends on assumptions, then details for one named procedure. Work three Cohen's d examples with the group order printed above the calculation.
Reverse that order and verify that the magnitude stays constant while the sign changes, then write which group scored higher in words. Compare the same effect in studies with different sample sizes to separate the estimated difference from the strength of statistical evidence.
Treat benchmark labels as orientation only: list the measurement quality, outcome importance, intervention cost and practical consequence that could make a numerically small effect meaningful or a large one unhelpful. Build a separate assumption map for mean comparison and correlation.
Connect independence to study design, scale to meaningful arithmetic, distribution shape to the reference model, equal variance to standard-error construction and linearity to what a correlation summarises. For each condition, write a plot or design check and a plausible distortion if it fails. Avoid the ritual statement that all assumptions were met; explain what was examined and why it matters.
Complete an extended response in three layers: define assumptions as conditions underlying interpretation, show how the sampling distribution or standard error depends on them, and apply named checks to the proposed analysis. Finish by reporting evidence, effect magnitude, uncertainty and scope in distinct clauses. Practise with an assumption violation that changes the standard error but not the observed group means.
This separates the data summary from the inferential reference model and clarifies why a plausible effect estimate can still carry unreliable uncertainty.
Working through Effect Sizes and Test Assumptions in PSY2041? Sia is AskSia’s AI Psychology tutor — ask any PSY2041 Effect Sizes and Test Assumptions question and get a clear, step-by-step explanation grounded in how PSY2041 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
