POLS1009 Chap.5 Causality, Counterfactuals and Experiments
Causality, Counterfactuals and Experiments
Why questions in politics are causal questions, and this chapter shows what a causal claim actually asserts. Every such claim compares the world we observed with a counterfactual world in which the cause was absent, and the causal effect is the gap between them.
Because no unit can be watched both with and without the treatment, that gap is never seen directly; Holland called this the fundamental problem of causal inference. Research design is the plan that supplies a credible substitute for the missing world, settled before any evidence is looked at, and the chapter warns against two shortcuts: starting from a favourite case or dataset, and steering clear of numbers.
Experiments build the substitute deliberately, letting chance alone decide who is treated, which makes treated and control groups alike on average and cuts any path from a confounder to the treatment.
The chapter then separates internal validity, which random assignment secures, from external validity, which it leaves open, and explains why manipulation, ethics and cost keep most political science on the observational branch.
What this chapter covers
- 01
Research design as a plan made before seeing the evidence
- 02
Two shortcuts to avoid when choosing a design
- 03
Two worlds, factual and counterfactual, behind every causal claim
- 04
Treatment, causal effect and the fundamental problem of causal inference
- 05
Experiments, control groups and random assignment
- 06
How randomisation breaks the link between a confounder and the treatment
- 07
Internal versus external validity
- 08
Why many political questions cannot be studied by experiment
Worked example · free
Factual and counterfactual worlds for Calder Bay's free buses
- 1Treatment and outcome. The treatment is the free bus service on polling day; the outcome is local election turnout in Calder Bay.
- 1Factual world. Calder Bay in 2025 with the buses running, where turnout was observed at 58%.
- 1Counterfactual world. The same city on the same 2025 polling day with no buses. Its turnout can never be observed, so it is the unknown half of the comparison.
- 1Why six points is not the effect. The 2021 figure belongs to a different election with different candidates, issues and weather, so it is a weak stand-in for the 2025 world without buses.
- 1Better design. At the next election, use a random draw to decide which half of the city's neighbourhoods get bus routes, then compare turnout between served and unserved neighbourhoods.
Key terms
- Treatment
- The cause whose influence a study sets out to evaluate, such as a policy, a message or a programme that some units receive and others do not.
- Causal effect
- The difference between the outcome a unit shows with the treatment and the outcome it would have shown without it, which can only be estimated because one of the two is never observed.
- Random assignment
- A procedure that lets chance alone, through a coin toss, a draw or a random-number generator, decide which units are treated, so that treated and control groups match on average.
- Internal validity
- Confidence that the difference in outcomes inside a study was produced by the treatment rather than by some other factor; random assignment secures it by design.
- External validity
- How far a result found in one study would hold for other populations, places, times or versions of the treatment, a question random assignment does not answer.
Causality, Counterfactuals and Experiments FAQ
What is the fundamental problem of causal inference?
It is the observation, associated with Holland in 1986, that the same unit cannot be seen both with and without the treatment at the same time. A city either runs a programme in a given year or it does not, and only one of those outcomes is ever recorded. Every causal claim therefore depends on inferring an unseen outcome, and research design is the tool for making that inference believable.
Why are randomised experiments called the gold standard?
Because chance rather than the researcher or the participants decides who is treated, the treated and control groups end up similar on average in every way, including traits nobody measured. Any later difference in outcomes can then be credited to the treatment. Observational studies have to approximate that fairness through case selection and statistical techniques instead.
How does internal validity differ from external validity?
Internal validity is about whether a study's own result is genuinely caused by the treatment. External validity is about whether that result would carry over to a different population, election, country or wording of the treatment. A study can be strong on the first and weak on the second, so the two need to be assessed separately.
Why doesn't political science just run experiments on every question?
Many causes cannot be assigned by anyone, such as wars, gender or economic crises. Some could be assigned but should not be, for ethical reasons that famous cases such as the Milgram studies made vivid. Running one is also expensive and usually isolates a single cause, so much of the discipline relies on observational designs.
Does random assignment prove the treatment works for every individual?
No. Randomisation makes groups comparable on average, so it supports a claim about the average effect across the people studied. No single participant is ever observed in both conditions, and some may respond strongly while others do not respond at all. Claims about particular individuals need other kinds of evidence.
Exam move
Treat this chapter as a vocabulary you must be able to apply on sight. For each causal claim you meet in a reading or a news story, write down the unit, the treatment, the outcome, the factual world and the counterfactual world in five short lines. Then ask how the author built a stand-in for the missing world: by random assignment, or by finding comparable cases in existing data.
Finally, sort every criticism you can think of into an internal validity worry or an external validity worry. Repeating this routine on a handful of claims each week turns definitions into a habit, and it prepares you for multiple-choice items that hinge on telling these terms apart.
Working through Causality, Counterfactuals and Experiments in POLS1009? Sia is AskSia’s AI Arts and Humanities tutor — ask any POLS1009 Causality, Counterfactuals and Experiments question and get a clear, step-by-step explanation grounded in how POLS1009 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.