Meta-evaluation of meta-analysis: ten appraisal questions for biologists

• Published in: BMC biology Vol. 15; no. 1; p. 18
• Main Authors: Nakagawa, Shinichi, Noble, Daniel W A, Senior, Alistair M, Lagisz, Malgorzata
• Format: Journal Article
• Language: English
• Published: England BioMed Central Ltd 03-03-2017; BioMed Central
• Subjects: Accessibility; Analysis; Bias; Biological evolution; Biologists; Biology; Clinical trials; Coining; Collaboration; Computer programs; Cooperation; Data processing; Documentation; Ecology; Ecosystem biology; Ecosystems; Error analysis; Evaluation; Evolution; Fruits; Glass; Greenhouses; Guidelines; Handbooks; Heterogeneity; Homogeneity; Humans; Information dissemination; Infrared radiation; Knowledge; Mathematical models; Messages; Meta-analysis; Meta-Analysis as Topic; Mice; Models, Theoretical; Practice; Practice guidelines (Medicine); Protocol (computers); Psychologists; Psychotherapy; Publication Bias; Rats; Readers; Regression analysis; Review; Reviews; Searching; Sensitivity analysis; Social sciences; Species richness; Statistical analysis; Statistical significance; Statistical tests; Statistics; Strings; Studies; Synthesis; Transparency; Trends; Wildlife conservation; Meta-regression; Statistical significance; Meta-research; Biological importance; Quantitative synthesis; Non-independence; Effect size; Publication bias; Systematic review; Reporting bias
• ISSN: 1741-7007; 1741-7007
• DOI: 10.1186/s12915-017-0357-7

Meta-analysis is a statistical procedure for analyzing the combined data from different studies, and can be a major source of concise up-to-date information. The overall conclusions of a meta-analysis, however, depend heavily on the quality of the meta-analytic process, and an appropriate evaluation of the quality of meta-analysis (meta-evaluation) can be challenging. We outline ten questions biologists can ask to critically appraise a meta-analysis. These questions could also act as simple and accessible guidelines for the authors of meta-analyses. We focus on meta-analyses using non-human species, which we term 'biological' meta-analysis. Our ten questions are aimed at enabling a biologist to evaluate whether a biological meta-analysis embodies 'mega-enlightenment', a 'mega-mistake', or something in between.