#MarksManInsights

Conducting a systematic literature review (SLR) is a detail-oriented, scrupulous process that hinges on the reliability of included studies, particularly randomized controlled trials (RCTs) that are the gold standard for scientific evidence. Therefore, when an RCT included in an SLR is later retracted, it can seem like a foundational crack compromising the entire structure.(1, 2) Sadly, retractions are quite common and so are their consequences. A 2025 JAMA meta-analysis reports 35% of meta-analyses experiencing at least a 10% change in effect estimates after eliminating retracted studies, with some even invalidating conclusions or entirely losing statistical significance.(3) The impact of retracted RCTs goes beyond academic distinction; influencing clinical guidelines, treatment decisions, and eventually patient care, allowing the approval of faulty results long after discreditation.(1-4)

Immediate damage assessment is the first step as soon as a retraction is identified. This can be done by re-running the analysis without the retracted RCT and carefully observing the effect of the exclusion on the results. The 2025 VITALITY Study observes that in 8.4% of meta-analyses, the direction of effects was reversed after the exclusion of retracted trials, while 16% of meta-analyses observed loss of statistical significance.(5) Such shifts may trigger researchers to take quick action, especially if the findings are already published. Journals are becoming increasingly alert, as seen in the Korean Journal of Anesthesiology case, where authors were asked to conduct a full re-analysis when a retraction appeared late in peer review.(6) With transparency being non-negotiable, it is essential for reviewers to clearly document the reason for exclusion, link to the retraction notice, and correct any summary of results accordingly.(1-4)

It is equally crucial to responsibly modify the scientific record. If the SLR has already been published or shared as a preprint, releasing a correction, error, or updated version becomes ethically important. However, this is more like an exception rather than the rule, wherein only 5% of systematic reviews citing later-retracted RCTs actually change their findings. This inaction has real-world impact. As of 2024, 157 clinical guidelines still referenced meta-analyses tarnished by retracted studies.(5) To avoid adding to this cycle, it is ideal to cite retracted studies with “[RETRACTED]” in the references, explaining the reason in-text, including methodological errors, data fabrication, or ethical violations.(1-4)

Reducing future risk safeguards the process of systematic reviews. Retraction screening is necessary at multiple checkpoints, such as during database searches, manuscript writing, submission, and even post publication. Several tools and databases, such as Retraction Watch, PubMed’s “Publication Status” filter, and citation managers like Zotero that mark retracted papers, can facilitate this process.(7) The Cochrane Handbook clearly specifies confirming study status to avoid unintentional inclusion of disputed information.(8) Also, pre-defining sensitivity analyses in the systematic review protocol can help ease the impact of important studies, particularly large or outlier RCTs, on the overall direction and strength of the findings.(1-4)

Furthermore, the presence of “zombie data” lingers with retracted studies still listed in citations, thus uncovering inherent systemic gaps. Nearly 40% of faulty meta-analyses have been observed to include retracted RCTs even after their official withdrawal, which can primarily be attributed to poor screening protocols. Additionally, a 2022 review observes only 6% of post-retraction citations acknowledging the study’s invalid status.(1)

It is the need of the hour for journals and databases to develop dependable defences like automated retraction alerts and obligatory checks at submission, as until then, it is the responsibility of researchers to remain vigilant. Safeguarding the reliability of evidence synthesis is more than just about data management, it includes maintaining trust in science and ensuring flawed studies stop shaping policy or clinical practice decisions.

Become A Certified HEOR Professional – Enrol yourself here!

References

1. Kataoka Y, Banno M, Tsujimoto Y, et al. Retracted randomized controlled trials were cited and not corrected in systematic reviews and clinical practice guidelines. J Clin Epidemiol. 2022 Oct;150:90-97.
2. Kataoka Y, Banno M, Tsujimoto Y, et al. The impact of retracted randomised controlled trials on systematic reviews and clinical practice guidelines: a meta-epidemiological study. Journal of Clinical Epidemiology. 2022.
3. Graña Possamai C, Cabanac G, Perrodeau E, et al. Inclusion of Retracted Studies in Systematic Reviews and Meta-Analyses of Interventions: A Systematic Review and Meta-Analysis. JAMA Intern Med. 2025;185(6):702–709.
4. Wartolowska K. Retracted RCTs and clinical guidelines. February 2019. [Accessed online on 16^th June 2025]. Available at: https://www.bennett.ox.ac.uk/blog/2019/02/retracted-rcts-and-clinical-guidelines/
5. Xu C, Fan S, Tian Y, et al. Investigating the impact of trial retractions on the healthcare evidence ecosystem (VITALITY Study I): retrospective cohort study. BMJ 2025; 389:e082068.
6. Choi GJ, Kang H. On the road to make KJA’s review process robust, transparent, and credible: retracted study in systematic review. Korean J Anesthesiol 2022; 75(3):197-199.
7. Bakker C, Boughton S, Faggion CM, et al. Reducing the residue of retractions in evidence synthesis: ways to minimise inappropriate citation and use of retracted data. BMJ Evid Based Med. 2024; 29(2):121-126.
8. Higgins JPT , Thomas  J , Chandler  Searching for and selecting studies. In: Higgins  JPT , Thomas  J , Chandler  J , eds. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: John Wiley & Sons, Ltd, 2019: 67-107.