91828 Chap.2 Evidence Hierarchy and Study Design
Evidence Hierarchy and Study Design
Evidence hierarchies are useful starting maps, not automatic quality scores. This chapter connects clinical question types with study designs, then separates the design label from execution, precision and applicability.
It shows why a well-conducted cohort study may offer more useful evidence for one question than a poorly conducted randomised trial, and why a systematic review inherits the strengths and limitations of its included studies.
What this chapter covers
- 01
Question and design fit
- 02
Systematic reviews
- 03
Randomised trials
- 04
Cohort studies
- 05
Case-control studies
- 06
Validity and applicability
Worked example · free
Choose a design for a harm question
- +1The question concerns long-latency harm, so experimental exposure is ethically unsuitable.
- +1A case-control design can begin with people who have the rare outcome and compare prior exposure with controls.
- +1Its efficiency for rare outcomes is a strength, while recall, selection and confounding are key threats.
- +1The conclusion should describe an association unless bias and alternative explanations are convincingly addressed.
Key terms
- Evidence Hierarchy
- An evidence hierarchy ranks designs by their usual ability to answer a type of question with lower bias.
- Cohort Study
- A cohort study follows exposed and unexposed groups to compare subsequent outcomes.
- Case-control Study
- A case-control study compares prior exposures among people with and without an outcome.
- Applicability
- Applicability is the extent to which study evidence informs a target patient, setting or decision.
Evidence Hierarchy and Study Design FAQ
Is a systematic review always the strongest evidence?
No. Its usefulness depends on a focused question, complete search, credible included studies and a coherent synthesis. A review of biased or incompatible studies can be precisely misleading. Read its methods and evidence base rather than relying on the label.
Why use observational evidence when RCTs exist?
Observational designs can address rare harms, long-term outcomes, prognosis and exposures that cannot be randomised. They also reveal practice conditions not represented in trials. Their confounding and selection threats need explicit appraisal rather than automatic dismissal.
What does lower-level evidence contribute?
Case reports, qualitative evidence and expert observation can identify new signals, patient experiences or implementation problems. They usually cannot estimate causal treatment effects reliably, but they can refine hypotheses and show where higher-level average evidence does not answer an individual decision.
Can a design label determine causal language?
The label sets a starting expectation but conduct determines whether it survives. Randomisation supports causal inference only when allocation, follow-up, measurement and analysis preserve the comparison. Observational evidence usually supports association language while explicitly addressing confounding and selection rather than implying that rank alone settles causality.
How should conflicting designs be synthesised?
Assign each design a role. A trial may estimate efficacy, a cohort may reveal longer-term harm, and qualitative work may explain acceptability. Compare compatible outcomes and populations, then explain disagreement through bias, intervention delivery, baseline risk or follow-up instead of taking a simple majority vote.
What makes diagnostic evidence different?
Diagnostic studies compare an index test with an appropriate reference standard in a relevant patient spectrum. Check whether all participants receive both tests, whether assessors are blinded and whether thresholds were prespecified. A treatment hierarchy cannot be imported unchanged because the target inference is classification accuracy.
Assessment move
Learn the hierarchy as a question-to-inference map rather than a fixed pyramid. Draw rows for intervention effect, harm, prognosis, diagnosis, prevalence and lived experience, then place feasible designs beside the estimate each can support. For every placement, add the main protection and the remaining threat.
A randomised trial protects an intervention comparison through allocation but can still lose validity through poor concealment, attrition or selective reporting. A cohort establishes temporal order but needs a credible account of confounding and follow-up. A case-control study efficiently investigates a rare outcome but depends on control selection and exposure measurement.
Rehearse these contrasts with short scenarios before looking at a study label. Once the design is identified, complete a three-part judgement: what it can estimate, what feature most changes confidence and how directly its participants and outcomes match the decision. Compare two reports using the same design and force yourself to rank them by execution.
Then compare two different designs addressing different parts of one clinical problem, such as an RCT for benefit and a registry cohort for uncommon harm. This prevents lower-level evidence from being either dismissed or overpromoted. Create a verb bank that preserves inference: describe for an uncontrolled series, associate for most observational comparisons, and cause only when the design and conduct protect that claim.
For review practice, inspect what designs enter a synthesis and whether the conclusion carries their limitations forward. During AT1 planning, assign each evidence source a role instead of asking which one wins. Note whether it informs efficacy, safety, feasibility, patient experience or applicability, then explain conflicts through populations, interventions, outcomes or bias pathways.
Finish by writing one calibrated conclusion from a high-ranking but poorly conducted study and another from a lower-ranking but well-conducted study. The exercise should show that hierarchy establishes starting confidence while appraisal, precision and directness determine the final judgement.
