True Reproducibility of UltraSound Techniques (TRUST): systematic review of reliability studies in obstetrics and gynecology

• Published in: Ultrasound in obstetrics & gynecology Vol. 46; no. 1; pp. 14 - 20
• Main Authors: Coelho Neto, M. A., Roncato, P., Nastri, C. O., Martins, W. P.
• Format: Journal Article
• Language: English
• Published: Chichester, UK John Wiley & Sons, Ltd 01-07-2015; Wiley Subscription Services, Inc
• Subjects: Female; gynecology; Gynecology - standards; Humans; obstetrics; Obstetrics - standards; Pregnancy; Reproducibility of Results; Ultrasonography - standards; Ultrasonography, Prenatal - standards; ultrasound; gynecology; reproducibility of results; obstetrics; ultrasound
• ISSN: 0960-7692; 1469-0705
• DOI: 10.1002/uog.14654

ABSTRACT
Objectives
To examine the quality of methods used and the accuracy of the interpretation of agreement in existing studies that examine the reliability of ultrasound measurements and judgments in obstetrics and gynecology.
Methods
A systematic search of MEDLINE was performed on 25 March 2014, looking for studies that examined the reliability of ultrasound measurements and judgments in obstetrics and gynecology with evaluation of concordance (CCC) or intraclass (ICC) correlation coefficients or kappa as a main objective.
Results
Seven hundred and thirty‐three records were examined on the basis of their title and , of which 141 full‐text articles were examined completely for eligibility. We excluded 29 studies because they did not report CCC/ICC/kappa, leaving 112 studies that were included in our analysis. Two studies reported both ICC and kappa and were counted twice, therefore, the number used as the denominator in the analyses was 114. Only 16/114 (14.0%) studies were considered to be well designed (independent acquisition and blinded analysis) and to have interpreted the results properly. Most errors occurring in the studies are likely to overestimate the reliability of the method examined.
Conclusions
The vast majority of published studies examined had
important flaws in design, interpretation and/or reporting. Such limitations are important to identify as they might create false confidence in the existing measurements and judgments, jeopardizing clinical practice and future research. Specific guidelines aimed at improving the quality of reproducibility studies that examine ultrasound methods should be encouraged. Copyright © 2014 ISUOG. Published by John Wiley & Sons Ltd.