BIOL2001 Chap.10 Confidence Intervals, t-Tests and Permutation Tests
Confidence Intervals, t-Tests and Permutation Tests
Intervals and tests for means
A confidence interval adds a range to a point estimate: at 95% confidence, it has that probability of enclosing the true mean. The t statistic divides the difference between means by its standard error, and for small samples with an estimated standard deviation it follows the t distribution.
A worked one-sample test checks whether dragonfly wingspans match a published mean and builds the matching interval.
Choosing and checking the test
A comparison table matches the one-sample, paired, two-sample and Welch t-tests to the design of a study, with the R call for each.
The chapter sets out the assumptions, how to check normality with histograms, QQ plots and the Shapiro-Wilk test, and why one-tailed tests are rarely appropriate. When normality fails, a permutation test builds the null distribution by rearranging the data: a computed figure shows all 252 rearrangements of ten seed masses and the exact p-value they give.
What this chapter covers
- 01
Confidence intervals for a mean
- 02
The t statistic as signal over noise
- 03
One-sample t-test
- 04
Paired t-test on differences
- 05
Two-sample and Welch t-tests
- 06
Normality checks: histogram, QQ plot, Shapiro-Wilk
- 07
One-tailed versus two-tailed tests
- 08
Permutation tests
Worked example · free
An exact permutation test on seed masses
- 1The observed means are 70/5 = 14.0 and 103/5 = 20.6, a difference of −6.6 mg.
- 1Under H0 the labels are arbitrary, and there are C(10, 5) = 252 ways to split the masses.
- 1Computing the difference for every split, 4 are at least as far from 0 as −6.6, including the observed split and its mirror.
- 1p = 4 / 252 = 0.016, below 0.05, so H0 of no difference is rejected.
Key terms
- T Statistic
- The difference between a sample mean and a hypothesised mean divided by its estimated standard error.
- Paired T-Test
- A t-test on the differences between two measurements of the same individuals.
- Welch's T-Test
- A two-sample t-test that does not assume equal variances in the two groups.
- QQ Plot
- A plot of sample quantiles against theoretical normal quantiles, used to judge normality.
- Permutation Test
- A test that builds the null distribution by repeatedly rearranging the data between groups.
- Two-Tailed Test
- A test that counts extreme results in either direction as evidence against the null hypothesis.
Confidence Intervals, t-Tests and Permutation Tests FAQ
Which t-test should I use for before and after measurements?
Use a paired t-test, because the same individuals are measured twice. It works on the differences, so each individual acts as its own control and the variation between individuals is removed from the standard error, making real changes easier to detect.
When should I use Welch's t-test?
Use Welch's t-test to compare two independent groups whose standard deviations clearly differ. The ordinary two-sample test assumes equal variances and is sensitive to violations, while Welch's version keeps the variances separate. In R, t.test() uses Welch unless you set var.equal = TRUE.
Why are one-tailed tests rarely appropriate?
A one-tailed test requires the direction of the effect to be fixed before the data are collected and gives up any test of the opposite direction. The lecture describes that as a very strong stance, rarely justified in investigative research, so two-tailed tests are the default.
What is the advantage of a permutation test?
A permutation test does not assume the data follow any particular distribution, so it remains valid when a t-test's normality assumption fails badly. It builds the null distribution directly by rearranging which observations belong to which group and recomputing the difference each time.
Assessment move
For each practice scenario, first name the design: one sample, paired or two independent samples. Run the same small dataset through t.test() with and without paired = TRUE to see how much the design matters. Draw a histogram and a QQ plot before every test, and report each result with its interval as well as its p-value.
If a dataset looks clearly non-normal, run a small permutation test as well and compare its p-value with the t-test's; agreement between the two is reassuring, and disagreement tells you the assumptions matter.
Working through Confidence Intervals, t-Tests and Permutation Tests in BIOL2001? Sia is AskSia’s AI Statistics tutor — ask any BIOL2001 Confidence Intervals, t-Tests and Permutation Tests question and get a clear, step-by-step explanation grounded in how BIOL2001 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
STAT1008 Quantitative Research Methods · BIOL1030 Diversity of Life · STAT7038 Regression Modelling
