Abstract
Objective
Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses.Design
Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews.Main outcome measure
Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision.Results
In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias.Conclusions
A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution.Free full text
Bias in meta-analysis detected by a simple, graphical test.
Abstract
OBJECTIVE: Funnel plots (plots of effect estimates against sample size) may be useful to detect bias in meta-analyses that were later contradicted by large trials. We examined whether a simple test of asymmetry of funnel plots predicts discordance of results when meta-analyses are compared to large trials, and we assessed the prevalence of bias in published meta-analyses. DESIGN: Medline search to identify pairs consisting of a meta-analysis and a single large trial (concordance of results was assumed if effects were in the same direction and the meta-analytic estimate was within 30% of the trial); analysis of funnel plots from 37 meta-analyses identified from a hand search of four leading general medicine journals 1993-6 and 38 meta-analyses from the second 1996 issue of the Cochrane Database of Systematic Reviews. MAIN OUTCOME MEASURE: Degree of funnel plot asymmetry as measured by the intercept from regression of standard normal deviates against precision. RESULTS: In the eight pairs of meta-analysis and large trial that were identified (five from cardiovascular medicine, one from diabetic medicine, one from geriatric medicine, one from perinatal medicine) there were four concordant and four discordant pairs. In all cases discordance was due to meta-analyses showing larger effects. Funnel plot asymmetry was present in three out of four discordant pairs but in none of concordant pairs. In 14 (38%) journal meta-analyses and 5 (13%) Cochrane reviews, funnel plot asymmetry indicated that there was bias. CONCLUSIONS: A simple analysis of funnel plots provides a useful test for the likely presence of bias in meta-analyses, but as the capacity to detect bias will be limited when meta-analyses are based on a limited number of small trials the results from such analyses should be treated with considerable caution.
Full Text
Articles from The BMJ are provided here courtesy of BMJ Publishing Group
Full text links
Read article at publisher's site: https://doi.org/10.1136/bmj.315.7109.629
Read article for free, from open access legal sources, via Unpaywall: https://europepmc.org/articles/pmc2127453?pdf=render
Citations & impact
Impact metrics
Citations of article over time
Alternative metrics
Smart citations by scite.ai
Explore citation contexts and check if this article has been
supported or disputed.
https://scite.ai/reports/10.1136/bmj.315.7109.629
Article citations
Acute Augmentations to Psychological Therapies in Eating Disorders: A Systematic Review and Meta-Analysis.
Curr Psychiatry Rep, 02 Aug 2024
Cited by: 0 articles | PMID: 39093534
Systematic review and meta-analysis on the effects of chronic peri-adolescent cannabinoid exposure on schizophrenia-like behaviour in rodents.
Mol Psychiatry, 02 Aug 2024
Cited by: 0 articles | PMID: 39090371
Review
Mortality and associated factors among patients with TB-HIV co-infection in Ethiopia: a systematic review and meta-analysis.
BMC Infect Dis, 24(1):773, 02 Aug 2024
Cited by: 0 articles | PMID: 39095740 | PMCID: PMC11295522
Review Free full text in Europe PMC
The effects of probiotic supplementation and exercise training on liver enzymes and cardiometabolic markers in patients with non-alcoholic fatty liver disease: a systematic review and meta-analysis of randomized clinical trials.
Nutr Metab (Lond), 21(1):59, 01 Aug 2024
Cited by: 0 articles | PMID: 39090657 | PMCID: PMC11293022
Review Free full text in Europe PMC
Prevalence of intrinsic capacity decline among community-dwelling older adults: a systematic review and meta-analysis.
Aging Clin Exp Res, 36(1):157, 01 Aug 2024
Cited by: 0 articles | PMID: 39088112 | PMCID: PMC11294388
Review Free full text in Europe PMC
Go to all (25,198) article citations
Other citations
Similar Articles
To arrive at the top five similar articles we use a word-weighted algorithm to compare words from the Title and Abstract of each citation.
Bias in meta-analysis and funnel plot asymmetry.
Stud Health Technol Inform, 68:323-328, 01 Jan 1999
Cited by: 51 articles | PMID: 10724898
The future of Cochrane Neonatal.
Early Hum Dev, 150:105191, 12 Sep 2020
Cited by: 5 articles | PMID: 33036834
Graphical augmentations to sample-size-based funnel plot in meta-analysis.
Res Synth Methods, 10(3):376-388, 07 Feb 2019
Cited by: 13 articles | PMID: 30664834 | PMCID: PMC6642847
Low-Dose Aspirin for the Prevention of Morbidity and Mortality From Preeclampsia: A Systematic Evidence Review for the U.S. Preventive Services Task Force
Agency for Healthcare Research and Quality (US), Rockville (MD), 01 May 2014
Cited by: 0 articles | PMID: 24783270
ReviewBooks & documents Free full text in Europe PMC
Behavioral and Pharmacotherapy Weight Loss Interventions to Prevent Obesity-Related Morbidity and Mortality in Adults: An Updated Systematic Review for the U.S. Preventive Services Task Force
Agency for Healthcare Research and Quality (US), Rockville (MD), 26 Oct 2018
Cited by: 0 articles | PMID: 30354042
ReviewBooks & documents Free full text in Europe PMC