BIOS6902 Chap.10 Confounding, Propensity Scores and Adjustment
Confounding, Propensity Scores and Adjustment
The comparison that invents an effect
Whether tobacco use causes cancer of the lung. Whether a change in schooling policy raises attainment. Whether antibiotics given early in life shape the microbiome years later. No ethics committee will let you allocate any of these at random, so the analysis must work with data in which people chose, or were chosen, and the resulting danger has a name: confounding.
The worked illustration is deliberately plain. Split a population into two health states. Those in the better state tend to be the ones who choose a wellness product, and they would have fared better without it too. The product changes nothing for anyone.
Set users beside non-users anyway and a large apparent benefit appears, entirely an artefact.
The accompanying simulation sharpens it: fix the effect at zero by construction, watch a strongly positive unadjusted contrast appear, then split by health state and watch the distortion vanish. The arithmetic is worth carrying out by hand once, because it shows that the naive number was not a poor estimate of the effect.
It was a good estimate of a different quantity. It compared people, and the people differed.
Three symptoms, and the assumption that licenses a repair
Three signals should stop you before a causal sentence is written. Exposure was chosen rather than allocated, which is visible in the protocol and needs no data at all to notice.
The two groups already differ on characteristics recorded before exposure began, and nothing occurring later explains a difference that predates it. And the estimate swings once an obvious background variable enters the model, which is the empirical fingerprint of the story above.
The repair requires an assumption that cannot be tested.
Ignorability says there are measured covariates such that, conditional on them, assignment is independent of both potential outcomes: within levels of those covariates, treatment looks as good as randomised. Under it the average effect is identified as the average over the covariate distribution of the within-stratum contrast. A second condition travels silently with it.
Positivity requires every covariate value to have a genuine chance of both arms, because a stratum containing only treated units supplies no comparison at all.
Three estimators, and the diagnostics that go with them
Where the covariates take few discrete values, split the data: work out the contrast inside every cell and average those cell contrasts over how common each cell is.
That is standardisation in its plainest form. When the covariates are many, the propensity score reduces the dimension. It is the probability of treatment given the covariates, and it has a balancing property: conditional on it, treatment is independent of the covariates, so conditioning on one number does the work of conditioning on all of them.
Reweighting is the third route.
With both conditions in force the effect can be written as an expectation of observed outcomes, each divided by the chance that unit had of receiving the arm it actually received, and swapping the expectation for a sample average produces the weighted estimator. The sequence the course asks for is fixed. Model the chance of allocation to treatment.
Plot that fitted chance separately by arm and look at where the two distributions meet. Form the weighted estimate. Prefer a sandwich standard error or resampling for its uncertainty, since the weights were themselves estimated. Then ask what strength of hidden background variable would be needed to overturn the answer.
The limits are examinable in their own right: a hidden common cause can never be excluded, a near-zero fitted chance produces a colossal weight and a wildly variable estimate, and claims drawn this way should be stated cautiously and hedged explicitly.
What this chapter covers
- 01
Why some causal questions cannot be randomised
- 02
A confounding story in which the treatment does nothing
- 03
Three symptoms that precede any causal sentence
- 04
Ignorability, and the positivity condition that travels with it
- 05
Stratification and standardisation for low-dimensional covariates
- 06
The propensity score and its balancing property
- 07
Inverse probability weighting, its diagnostics and its limits
Worked example · free
Recovering a zero effect by stratifying on the confounder
- 2Compute the pooled rate in each arm from the stratum sizes and rates.
- 1Take the naive difference and note its size.
- 2Compute the contrast inside each stratum.
- 2Average the stratum contrasts and explain why the two answers differ.
Key terms
- Confounding
- The distortion of a treatment-outcome comparison by a common cause of both, which can create, hide or reverse an apparent effect.
- Ignorability
- The assumption that, conditional on measured covariates, assignment is independent of both potential outcomes, so treatment is as good as randomised within strata.
- Positivity
- The requirement that every covariate value has a genuine chance of receiving either arm, without which no comparison exists in that region.
- Standardisation
- Estimating the contrast within each covariate stratum and averaging those contrasts over the covariate distribution.
- Propensity Score
- The probability of receiving treatment given the covariates, which balances the covariates between arms when conditioned upon.
- Inverse Probability Weighting
- Estimating a causal effect by weighting each observed outcome by the reciprocal of the probability of the arm that unit received.
- Overlap
- The degree to which both arms are represented across the range of the propensity score; poor overlap makes part of the population uncomparable.
- Sensitivity Analysis
- A calculation of how strong an unmeasured confounder would need to be to overturn the reported conclusion.
Confounding, Propensity Scores and Adjustment FAQ
Why can ignorability never be tested from the data?
Because it is an assertion about the unobserved half of each unit's outcomes. Any test would need the counterfactual outcomes that the design never supplies, so no statistic can confirm or refute it. It is defended by subject-matter argument about what actually drove treatment decisions, supported by a sensitivity analysis, and never by a goodness-of-fit measure.
What goes wrong when some propensity scores are near zero?
The weighted estimator divides by that probability, so a unit with a near-zero score receives an enormous weight and the estimate becomes a noisy average dominated by a handful of observations. The symptom is an extreme weight distribution and a standard error far larger than the sample size suggests. Restrict to the region where both arms appear, say so, and state that the estimate now refers to that region.
Is adjusting for more covariates always safer?
No. Adjusting for a variable that lies on the causal pathway between treatment and outcome removes part of the effect you are trying to measure, and adjusting for a consequence of both can introduce bias rather than remove it. Which covariates belong in the adjustment set is a subject-matter decision about the order in which things happened, not a model-selection exercise.
Exam move
Rehearse the stratification arithmetic on a two-stratum example until you can produce the reversal from memory. It is the clearest single demonstration in the course that a correct calculation can answer the wrong question, and it makes every later adjustment method easy to motivate.
Working through Confounding, Propensity Scores and Adjustment in BIOS6902? Sia is AskSia’s AI Statistics tutor — ask any BIOS6902 Confounding, Propensity Scores and Adjustment question and get a clear, step-by-step explanation grounded in how BIOS6902 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
