RS517 Chap.6 Choosing Tests for Group Comparisons
Choosing Tests for Group Comparisons
Structure before assumptions
Test selection begins with the question and outcome scale, then asks whether observations are independent or related and whether two or several groups are compared. Distributional assumptions refine the branch after those structural decisions.
Paired data carry within-person information
An independent t test compares group means using between-person variation.
A paired t test analyses within-person differences and can remove stable person-level variation. ANOVA provides a global test across several means, while repeated-measures models preserve dependence across occasions.
Rank tests change the target
Mann-Whitney, Wilcoxon, Kruskal-Wallis and Friedman provide rank-based alternatives for related structures. They are not assumption-free copies of mean tests.
Report the distributions and explain what ordering or difference the selected procedure addresses.
Use the data structure as the first branch
Test selection begins with the outcome scale, number of groups or occasions and whether observations are independent, paired or clustered.
The same numerical values require different standard errors when they come from different people versus repeated measurements on the same people. Distributional features refine a branch only after that structure is clear.
Coding categories as integers does not create a continuous outcome, and similar baseline means do not create pairing.
Name the target of each route
An independent t test compares means under its model, with Welch's version avoiding a pooled equal-variance estimate. A Mann-Whitney procedure concerns ordering or distributions and should not automatically be described as a median test.
Paired tests analyse within-pair change rather than unrelated group summaries. Chi-square methods work with contingency-table counts, while sparse cells can require an exact route.
For three or more groups, an omnibus test asks whether some difference exists; it does not identify every pair.
Plan follow-up comparisons before seeing the pattern
Planned contrasts can target a scientific hypothesis directly, whereas post hoc procedures protect a broader family after a global test. Multiplicity control should match the family of claims rather than be applied mechanically to every number.
Alongside any test, give group summaries, an effect estimate with uncertainty and a practical interpretation. Assumption checks should diagnose the fitted model, not act as a ritual that automatically switches procedures at one preliminary p-value.
Revision drill
For five scenarios, write four labels before naming a procedure: outcome scale, groups or occasions, dependence and target effect.
Include a sparse two-by-two table, paired continuous change, three independent skewed groups, repeated ordinal scores and clustered binary outcomes. State one tempting wrong test and the structural reason it fails. Then add the effect measure that would make each result interpretable beyond a significance label.
What this chapter covers
- 01
Outcome scale and test families
- 02
Independent and paired observations
- 03
Independent and paired t tests
- 04
One-way and repeated-measures ANOVA
- 05
Rank-based comparison procedures
- 06
Global tests and follow-up contrasts
Worked example · free
Choose a route for repeated skewed scores
- 3Identify repeated observations and several occasions.
- 5Choose and interpret a repeated rank-based route.
Key terms
- Independent Sample
- A group of observations whose values do not share a participant or matched unit with another group.
- Paired Difference
- The within-pair subtraction analysed in a paired comparison.
- Global Test
- A test of whether any difference exists across a set of groups or occasions.
- Mann-Whitney U
- A rank-based procedure comparing the ordering of two independent groups.
- Friedman Test
- A rank-based global comparison across several related occasions or conditions.
Choosing Tests for Group Comparisons FAQ
What decides between an independent and paired test?
The data-generating structure decides. Use a paired route when observations are repeated on the same person or deliberately matched into meaningful pairs. Similar group sizes or baseline values do not create pairing. Treating repeated observations as independent gives the wrong standard error.
What does a significant ANOVA establish?
It rejects the global claim that all modelled group means are equal. It does not identify the differing groups or imply every pair differs. Use planned contrasts or multiplicity-aware follow-up comparisons, and report estimates and intervals that answer the substantive question.
Are non-parametric tests assumption-free?
No. They still require a correct independence or pairing structure and make assumptions about distributions for common interpretations. They analyse ranks rather than original-scale means, so the conclusion must match the rank-based target instead of being described automatically as a median comparison.
Exam move
Build a test-selection table from outcome scale, independence and group count. For each cell, pair a parametric and rank-based route and state how their targets differ. Practise justifying the selected test and rejecting one tempting alternative in a single sentence.
Working through Choosing Tests for Group Comparisons in RS517? Sia is AskSia’s AI Statistics tutor — ask any RS517 Choosing Tests for Group Comparisons question and get a clear, step-by-step explanation grounded in how RS517 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.