BB2025 Chap.3 Validity, Bias and Missing Data
Validity, Bias and Missing Data
Locate the stage where validity was threatened
Selection bias changes who enters or remains in a comparison. Information bias changes how variables are observed. Confounding creates an alternative pathway because a common cause influences exposure and outcome. Attrition can combine selection and missing outcomes after enrolment.
The labels matter only when they identify the process and the point at which a preventive control could operate.
Separate internal and external questions
Internal validity concerns the credibility of the estimated relationship inside the study. External validity concerns transport to another population, setting or time.
A narrow trial can have strong internal validity and limited reach, while a broad observational sample can resemble a population without supporting a causal effect. The two questions should be answered separately.
Missingness requires an assumption
The percentage missing is not enough. Ask which variable is incomplete, when it disappeared and whether the reason depends on observed information or the unseen value.
Complete-case analysis can change the target population. Model-based imputation can propagate uncertainty under stated assumptions, while sensitivity analysis shows how conclusions move when those assumptions are relaxed.
What this chapter covers
- 01
Internal and external validity
- 02
Selection processes and analysis entry
- 03
Measurement error and information bias
- 04
Confounding pathways
- 05
Differential attrition
- 06
Missing-data mechanisms
- 07
Multiple imputation assumptions
- 08
Sensitivity and transport analysis
Worked example · free
Diagnose unequal loss to follow-up
- 3Identify confounding from treatment choice.
- 3Explain the differential attrition mechanism.
- 3Propose analysis and sensitivity checks.
Key terms
- Selection Bias
- Systematic distortion created by how participants enter, remain in or are included in an analysis.
- Information Bias
- Systematic distortion caused by how exposures, outcomes or covariates are measured or classified.
- Attrition
- Loss of enrolled participants or observations before the intended analysis is completed.
- Complete-case Analysis
- Analysis restricted to records with every variable required by the selected model.
- Multiple Imputation
- Creation and combination of several plausible completed datasets to preserve uncertainty about missing values.
- Sensitivity Analysis
- A structured check of how a conclusion changes under alternative plausible assumptions or analytical choices.
Validity, Bias and Missing Data FAQ
What is the difference between bias and random error?
Bias shifts an estimate systematically through selection, measurement, confounding or analysis. Random error makes estimates vary across repeated samples. Larger samples usually reduce random error, but they do not remove a systematic process and can make a biased estimate appear very precise. Diagnose the mechanism before recommending more observations.
Can missing completely at random be tested from the dataset?
The dataset can reveal associations between missingness and observed variables, which may contradict the strongest assumption. It cannot prove that missingness is unrelated to unseen values. The mechanism remains an assumption informed by collection knowledge, observed patterns and sensitivity analysis. State it explicitly rather than treating a software default as evidence.
How should I write a useful limitation?
Name the process, predict how it could alter the estimate and state the control or sensitivity analysis that addresses it. For example, self-selection by baseline mobility may exaggerate an exercise benefit, so random allocation or careful pre-exposure measurement is needed. This is more informative than saying simply that selection bias may be present.
Exam move
Take each study scenario and draw a timeline from recruitment through allocation, measurement and follow-up. Place the validity threat where it enters, then write one preventive control and one analytical response. For missing data, practise the same four questions every time: which variable is absent, at what point, for what recorded reason and under what alternative values the conclusion changes.
Finish each appraisal with a narrowed verb that reflects the remaining uncertainty. Build a direction ledger for common mechanisms: who is selectively included, which values are systematically shifted and whose outcome is more likely to disappear. If the direction cannot be known, list the competing pathways rather than writing an empty warning.
Rehearse one sensitivity statement that explains how much unseen change would be required to reverse the decision. Compare a design fix with an analysis fix for each threat. Prefer prevention when the analytical repair depends on variables that were never measured accurately. Build one missingness table with rows for observed and absent outcomes and columns for recorded predictors.
Mark which relationships can be checked and which remain assumptions. This separates diagnostics from unverifiable claims about the unseen values.
Working through Validity, Bias and Missing Data in BB2025? Sia is AskSia’s AI Statistics tutor — ask any BB2025 Validity, Bias and Missing Data question and get a clear, step-by-step explanation grounded in how BB2025 is taught and assessed. Read this chapter free, then take your hardest questions to Sia.
