PYB350 Chap.5 Planned Contrasts, Post Hoc Tests and Type I Error
Planned Contrasts, Post Hoc Tests and Type I Error
A significant omnibus F says a difference exists somewhere; follow-up tests find it. This chapter separates planned contrasts, chosen before the data to test predictions, from post hoc tests, which compare every pair when there are no hypotheses.
It explains the per-comparison, experiment-wise and family-wise error rates, the Bonferroni adjustment and the Keppel and Wickens rule that up to a minus one planned contrasts need no correction, how to write contrast weights, and how to choose among Tukey, REGWQ, Gabriel, Hochberg GT2, Games-Howell and Dunnett tests.
The worked example tests two planned contrasts in a four-group design from the means and the error term, and the practice cards drill the alpha decision and the choice of post hoc test from Levene's result and the group sizes.
What this chapter covers
- 01
Planned contrasts against post hoc comparisons
- 02
Per-comparison, experiment-wise and family-wise error
- 03
Bonferroni and the a minus one rule
- 04
Writing contrast weights that sum to zero
- 05
Contrast values, t tests and the SPSS LSD option
- 06
Choosing a post hoc test by variance and group size
- 07
Reading a bootstrapped post hoc comparison through its interval
Worked example · free
Two planned contrasts in the background-sound study
- 1Check the weights sum to zero and that one contrast is within the a minus one = 3 limit, so alpha stays at .05.
- 1Contrast value: 18 + 16 minus 11 minus 9 = 14.
- 1Standard error: square root of (8 times 4 / 6) = 2.309.
- 2Test: t(20) = 14 / 2.309 = 6.06, p below .001, so the no-speech groups read better than the speech groups.
Key terms
- Post hoc test
- A comparison of every pair of means made without prior hypotheses, with the critical value adjusted for the number of comparisons.
- Experiment-wise error rate
- The probability of at least one false positive across all the comparisons in a study, approximated as alpha times the number of comparisons.
- Tukey test
- The usual post hoc procedure for pairwise comparisons when variances are equal and group sizes are equal.
- Contrast weight
- A coefficient given to each group mean so that the weights sum to zero and define one comparison.
Planned Contrasts, Post Hoc Tests and Type I Error FAQ
Do planned contrasts need a significant omnibus F?
No. The lecture allows planned comparisons whether or not the overall F is significant, because they test predictions made before the data were seen. Post hoc tests are different: they explore all pairs after a significant omnibus result, which is why they need stronger control of the Type I error rate.
When do planned contrasts need a Bonferroni correction?
Following Keppel and Wickens (2004), the lecture says no adjustment is needed when the number of planned contrasts does not exceed the degrees of freedom for the effect, a minus one. Plan more than that and divide .05 by the number of contrasts, so three tests use .0167 and four use .0125.
Which post hoc test should I choose?
Match it to your data. With equal variances and equal group sizes, use Tukey or REGWQ; with slightly unequal sizes, Gabriel's; with very unequal sizes, Hochberg's GT2. With unequal variances, use Games-Howell, or Dunnett C with at least 75 error degrees of freedom and Dunnett T3 for smaller samples. State the reason in your write-up.
Why does SPSS's Bonferroni option give a different answer from mine?
When you request Bonferroni-adjusted pairwise comparisons, SPSS corrects for every possible pair of means, not just the few comparisons you planned. For a small set of planned contrasts that is too strict. The lecture's advice is to ask for unadjusted (LSD) comparisons and compare each exact p value with the alpha you calculated for your own number of tests.
Exam move
For every design you meet, write down the number of groups, a minus one, and how many contrasts were planned, then decide the alpha before you look at any p value. Say every follow-up choice as a full sentence with its reason, such as equal variances and equal group sizes, therefore Tukey, because that is the sentence markers look for.
Working through Planned Contrasts, Post Hoc Tests and Type I Error in PYB350? Sia is AskSia’s AI Statistics tutor — ask any PYB350 Planned Contrasts, Post Hoc Tests and Type I Error question and get a clear, step-by-step explanation grounded in how PYB350 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
