CMCE10002 Chap.11 Causal Analytics and Randomised Experiments
Causal Analytics and Randomised Experiments
The comparison you need is with a world that did not happen
A causal question asks whether one thing produced another, and the subject gives a reliable tell: the analysis is being used to argue for changing a procedure, policy or practice. Each unit has two potential outcomes, one with the treatment and one without, and the causal effect is the difference between them.
Only one of the two is ever observed.
Two routes to the missing half
Model-based inference uses what you believe about how the world works to adjust the comparison, and is credible exactly to the extent that those beliefs are correct.
Design-based inference builds a fair comparison into how the data is generated, because deciding by chance scatters whatever differences already existed across the two arms, including the ones nobody measured.
Design still assumes something
Comparability requires equal assignment odds, two arms separated by nothing but the intervention, and units that really did get the condition allocated to them.
Stability requires that no unit's result is disturbed by another's, that every allocated unit yields a measurable result, and that the intervention is one consistent thing rather than several.
Two ways to state the same effect
A difference between two rates can be reported absolutely, in percentage points, or relatively, as a proportion of the control rate.
Both describe one result, and quoting the relative figure while calling it percentage points can overstate an effect by roughly an order of magnitude. The absolute form is what a reader needs to work out how many people are involved; the relative form is what allows comparison with another experiment measured on a different base.
What this chapter covers
- 01
Spotting a causal question from the decision it is meant to support
- 02
Potential outcomes, and why only one of them is ever observed
- 03
Omitted variables that drive both the treatment and the outcome
- 04
Model-based adjustment against design-based randomisation
- 05
Comparability and stability, and telling which one breaks in a given setting
Stating the size of an effect in both of its forms
- 1State the difference in percentage points and what it is for.
- 1State the difference in relative terms and what it is for.
- 1Attach the judgement the reader needs.
Key terms
- Potential outcome
- What a unit's result would be under a given treatment, only one of which is ever observed.
- Omitted variable
- Something left out of the analysis that influences both the treatment and the outcome.
- Random assignment
- Deciding by chance which units are treated, so unmeasured differences scatter across both arms.
- Control arm
- The group that does not receive the intervention, standing in for what would otherwise have happened.
- Spillover
- An effect on a unit produced by another unit's treatment, which breaks the stability assumption.
- Percentage point
- The absolute difference between two percentages, distinct from the relative change between them.
Causal Analytics and Randomised Experiments FAQ
Why is a before-and-after comparison weak evidence?
Because everything else that changed between the two periods is absorbed into the difference: seasonality, campaigns, news events and competitor behaviour among them. The observed change is the true effect plus all of those, and nothing in the data separates them.
What does random assignment actually buy?
It leaves the two arms alike on average in everything, including whatever nobody thought to record. Adjustment can only balance what was measured, which is why design-based evidence rests on a shorter and more visible list of assumptions.
Does randomisation remove all assumptions?
No. It still requires comparability, meaning equal assignment odds and actual receipt of the allocated condition, and stability, meaning no unit disturbed by another's treatment, no units dropping out, and one consistent intervention rather than several.
Exam move
Practise the three-clause causal answer: name the assumption, name the specific way it fails in this setting, name the design that would fix it. A general warning about correlation earns very little on its own.
Working through Causal Analytics and Randomised Experiments in CMCE10002? Sia is AskSia’s AI Statistics tutor — ask any CMCE10002 Causal Analytics and Randomised Experiments question and get a clear, step-by-step explanation grounded in how CMCE10002 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
