MAST20034 · Critical Thinking With Data
Study Design: Experimental vs Observational
Week 3 is one of the two highest-value chapters in the subject, because the design decides which verb you are allowed to use — causes or merely is associated with. The chapter turns on a single question: did the researcher assign the exposure, or only observe it? Assignment (ideally random) is what licenses a causal claim; observation does not. You learn the four moves of a randomised controlled trial — randomisation, a control group, blinding and a placebo — and exactly what each one buys (randomisation balances confounders, blinding kills measurement and placebo bias, and so on). The design decision tree lets you classify any described study fast, and a clear table fixes what each design can and cannot establish. Finally you separate internal validity (is the conclusion sound for these subjects?) from external validity (does it generalise?), the trade-off behind the WEIRD-sample critique you meet later. Slow down here: most exam marks are won or lost on naming the design and the legal conclusion.
What this chapter covers
- 013.1 What study design is, and why it sets the verb
- 023.2 The central split: did the researcher assign or observe the exposure?
- 033.3 The four moves of an RCT — randomisation, control, blinding, placebo — and what each buys
- 043.4 The design decision tree
- 053.5 What each design can and cannot establish (causation vs association)
- 063.6 Internal vs external validity
Naming the design and its legal conclusion, mark by mark
- +1Name the design: participants were observed, not assigned to exercise, and followed forward in time — this is a prospective cohort (observational) study.
- +1State the design's limit: because the researcher did not assign the exposure, the design can establish association but not causation.
- +1Apply it (the confounder): exercisers may differ systematically — healthier diet, non-smoking, higher income — so a confounder could drive both the exercise and the lower heart-attack rate.
- +1Verdict + fix: the causal headline is not justified from this design; only a randomised controlled trial (or careful confounder adjustment) could license ‘prevents’.
Key terms
- Experimental vs observational
- An experiment assigns the exposure (ideally at random); an observational study only watches who already has it. Assignment is what licenses a causal claim, which is why this split sets the verb.
- Randomisation
- Assigning subjects to groups by chance. Its power is balancing both known and unknown confounders across groups on average, so a post-treatment difference can be attributed to the treatment.
- Blinding
- Keeping subjects (single-blind) and/or assessors (double-blind) unaware of group allocation. It guards against placebo and measurement bias — expectations changing behaviour or how outcomes are recorded.
- Internal validity
- Whether a study's causal conclusion is sound for the subjects in it — did the design rule out confounding and bias? High in a well-run RCT; threatened by confounding in observational designs.
- External validity
- Whether a study's conclusion generalises beyond its own sample to other people, settings or times. A tightly controlled trial on a narrow sample can be internally valid yet externally weak.
Study Design: Experimental vs Observational FAQ
Why can only experiments show causation?
Because random assignment balances confounders across groups, so a difference in outcome can be pinned on the treatment. In observational studies the groups self-select and may differ in unmeasured ways, leaving association as the strongest legal claim.
What does each part of an RCT actually buy?
Randomisation balances confounders; a control group gives a counterfactual baseline; blinding removes placebo and assessor bias; a placebo isolates the active ingredient from the act of being treated. Name the move and its purpose — that pairing is the mark.
How do internal and external validity trade off?
Tightening control (strict inclusion criteria, a lab setting) boosts internal validity but can shrink and narrow the sample, weakening external validity. The exam likes you to spot this tension explicitly.
Exam move
This is a make-or-break chapter — budget extra revision here. Drill the one reflex that unlocks most design questions: who chose the exposure? Subjects/nature → observational → association; researcher-randomised → experiment → causation possible. Put the design decision tree and the RCT four-moves table (each move → what it buys) on your notes sheet verbatim. Practise pairing every named design with the conclusion it legally supports, and always flag the internal-vs-external validity trade-off when a study controls heavily but samples narrowly.
