Science/Education Portraits IX- Reproducibility and Transparency of Systematic Reviews

Raphael YH Hon¹ and Maurice HT Ling²*
¹Yong Loo Lin School of Medicine, National University of Singapore, Singapore
²HOHY Pte Ltd, Singapore
*Corresponding Author: Maurice HT Ling, HOHY Pte Ltd, Singapore.

Published: July 23, 2024

DOI: 10.55162/MCMS.07.231

View Pdf

Abstract  

The main advantage of systematic review (SR) over other review types, such as narrative review, is its transparent and rigorous approach to minimize bias and ensure future replicability. Reproducibility, comprehensiveness, and transparency are the 3 keys to minimize bias. Reproducibility and transparency are deemed more important than comprehensiveness as missing literature may be identifiable from a reproducible and transparent SR lacking comprehensiveness but missing literature cannot be identified from a comprehensive SR lacking reproducibility and/or transparency. Hence, a SR lacking reproducibility and/or transparency is a narrative review. Several studies on published SRs report poor reproducibility and transparency. Here, we examine a sample of 100 SRs published in 2023 and indexed within PubMed for reproducibility and transparency on 10 criteria - 7 for reproducibility and 3 for transparency. In terms of reproducibility, our results show that only 31 SRs gave sufficient information for reconstructed searches in PubMed to yield search results within 10% of the hits reported. In terms of transparency, none of the SRs provided the full list of search results and only 5 SRs provided sufficient information to determine which study is excluded by which exclusion criterion. Hence, our results support previous studies reporting poor reproducibility and transparency in SRs. We propose that search URLs should be given whenever possible, and the full list of search results with inclusion / exclusion analysis should be given in all cases.

.