91828 Chap.3 Systematic Reviews and Meta-analysis
Systematic Reviews and Meta-analysis
A systematic review earns confidence through visible, reproducible decisions. This chapter follows the route from a registered question and comprehensive search through duplicate selection, risk-of-bias appraisal and synthesis.
It distinguishes systematic review from meta-analysis, explains why compatibility must precede pooling, and turns heterogeneity, sensitivity analysis and conclusion wording into a practical review audit for AT1.
What this chapter covers
- 01
Protocol and registration
- 02
Search completeness
- 03
Eligibility and screening
- 04
Risk-of-bias appraisal
- 05
Narrative and quantitative synthesis
- 06
Heterogeneity and sensitivity
Worked example · free
Audit a pooled rehabilitation review
- +2Test search completeness using dates, registers, reference searching, language rules and unpublished evidence.
- +2Compare populations, interventions, outcomes and follow-up times before accepting one pooled effect.
- +2Inspect risk of bias and rewrite the conclusion with the estimate, interval, heterogeneity and applicability limits.
Key terms
- Eligibility Criteria
- Eligibility criteria define which studies can enter a review before their results drive selection.
- Heterogeneity
- Heterogeneity is variation in effects or study characteristics that can make one average misleading.
- Sensitivity Analysis
- Sensitivity analysis tests whether a conclusion changes under defensible alternative assumptions or study sets.
- Publication Bias
- Publication bias occurs when study availability is related to the direction or strength of results.
Systematic Reviews and Meta-analysis FAQ
Can a systematic review omit meta-analysis?
Yes. A review can use systematic methods to find and appraise evidence but provide a structured narrative synthesis when studies differ too much in population, intervention, outcome or timing. Forced pooling is not a sign of greater rigour.
What should I check before pooling?
Compare the clinical question, intervention dose, comparator, outcome definition, follow-up and study design. Then inspect risk of bias and statistical variation. Software can combine numbers that do not represent one clinically coherent decision, so compatibility is an appraisal judgement.
Does a random-effects model solve heterogeneity?
No. It models between-study variation and changes weighting, but it does not explain why effects differ or make incompatible studies comparable. Investigate clinical and methodological causes, prespecified subgroups and sensitivity analyses before interpreting the average.
Why compare the review with its protocol?
A protocol records intended eligibility, outcomes and analyses before results are known. Comparing it with the final report can reveal added outcomes, changed time points or post-hoc subgroups. Changes may be justified, but they should be transparent because data-driven choices can exaggerate certainty.
What makes sensitivity analysis informative?
It should challenge an assumption that could plausibly alter the conclusion, such as excluding high-risk studies, using another effect model or varying missing-data handling. Repeating closely related models offers limited reassurance. Report whether magnitude, precision, direction or clinical meaning changes under the alternative.
How do I judge whether a review is current?
Record the last search date and look for major later trials addressing the same PICO. A methodologically sound review can become incomplete as evidence accumulates. Distinguish this currency problem from weak conduct, then explain whether newer evidence could affect the pooled estimate or applicability.
Assessment move
Choose one systematic review relevant to the team question and reconstruct it from methods to conclusion. Begin with the protocol or registration and compare the planned population, intervention, comparator, outcomes and analysis with the final report. Record every unexplained change because outcome or subgroup switching can make a result look prespecified when it was not.
Build a search audit with databases, coverage dates, trial registers, grey literature, language restrictions and the last search date. For each omission, state what type of evidence might be missed and how that absence could affect the synthesis without claiming a direction you cannot know. Follow the flow diagram and sample exclusions from each stage; test whether the published eligibility criteria explain them.
Next, make a study-characteristics table for population, dose, comparator, outcome definition and follow-up. Colour differences only after deciding whether they change the clinical decision. Separate risk of bias from heterogeneity: the first concerns systematic distortion within studies, while the second concerns variation across estimates or methods.
Read forest plots alongside this table rather than treating a heterogeneity statistic as a complete explanation. Identify which studies carry most weight and conduct a paper sensitivity exercise by imagining removal of the largest or highest-risk study. Ask whether the direction, precision or clinical meaning would change.
If the authors use subgroup analysis, check whether it was prespecified, supported by a direct interaction and biologically credible. Draft an AT1 paragraph with five parts: review question, search strength, included-study credibility, pooling judgement and calibrated conclusion. Replace checklist phrases with mechanisms, such as explaining how an incomplete register search could omit unpublished trials.
Finally compress the review into a single sentence containing population, comparison, pooled effect and interval, the leading validity concern and an applicability boundary. If one clause cannot be supported from the report, return to the methods instead of filling the gap with confidence language.
