#HEORInsights

The bedrock of evidence-based decision-making rests on the integrity of systematic reviews, which rely on the credibility of the studies they encompass. While bias assessment and risk of bias evaluation are crucial steps in ensuring study quality, the need to go beyond these measures has become increasingly evident. Enter the Research Integrity Assessment (RIA) tool, a groundbreaking approach that aims to safeguard the authenticity and reliability of studies included in evidence synthesis.[1-4]

The RIA tool is not a mere supplement to the traditional “Risk of Bias” assessment; it is a distinct and innovative framework designed to establish the integrity and authenticity of studies. RIA meticulously scrutinizes various aspects of study conduct, ranging from retraction notices and prospective trial registration to ethics approval, authorship, and the plausibility of methods and results. By doing so, RIA addresses concerns related to scientific misconduct, poor research practices, and potential biases that may distort evidence synthesis findings.[1, 5]

Timing is crucial when implementing RIA within evidence synthesis. RIA is best employed early in the review process, particularly for randomized controlled trials (RCTs) that have passed the initial PICO (participants, intervention, comparator, and outcomes) eligibility screening. This proactive approach allows for the early exclusion of problematic RCTs, ensuring the integrity of the entire study pool and all subsequent analyses.[5]

The workflow of RIA involves a hierarchical assessment through the six domains: Retracted studies, absence of prospective registration, inadequate ethical approval without informed written consent, discrepancies within the author group and study location, insufficient randomization, and implausible study results should result in the exclusion of an RCT. Should any concerns arise within any domain, the study is categorized as “awaiting classification,” prompting further scrutiny. If no concerns persist across all domains, or if issues are adequately addressed through correspondence with study authors, the RCT meets the criteria for inclusion in the review and can proceed to the next stages. In the context of living systematic reviews, both included RCTs and those labeled as “awaiting classification” need to be re-evaluated for potential retraction notices., culminating in a decision regarding a study’s eligibility.[5]

For the RIA assessment, a collaborative and thorough approach is essential. Each study should be independently evaluated by two review authors, and any discrepancies should be resolved through discussions. A diverse team of researchers with expertise in clinical trial design, systematic review methodology, and clinical content should carry out the RIA assessment.[5]

A hallmark of the RIA tool is its emphasis on transparency and documentation. An Excel-based format of the RIA tool contains critical signaling questions and columns for summarizing conclusions for each domain. The resulting table not only justifies the decision on research integrity and eligibility but also offers an accessible way to document the review authors’ assessments and judgments. This information should be made readily available through publication as a supplement to the systematic review or through online repositories.[5]

The RIA tool represents a significant step towards a standardized approach for identifying and managing problematic studies within evidence synthesis. While the tool’s development was prompted by the challenges posed by the COVID-19 pandemic, its potential extends far beyond this context. As the systematic review landscape continues to evolve, the iterative refinement and validation of RIA offer a promising avenue for enhancing the credibility, reliability, and ethical soundness of evidence synthesis. To our collective commitment, the RIA tool stands as a testament, demonstrating our dedication to maintaining research integrity and furthering the quest for impartial knowledge, all for the advancement of society’s well-being.[5]

In the ever-evolving landscape of evidence synthesis, the Research Integrity Assessment (RIA) tool emerges as a beacon of hope. By providing a systematic and comprehensive framework to assess research integrity, RIA adds a crucial layer of protection against distorted findings and compromised recommendations.

Become A Certified HEOR Professional – Enrol yourself here!

Learn AI Potential For Writing Manuscripts – Enrol yourself here!

References

1. Ioannidis JPA. Hundreds of thousands of zombie randomised trials circulate among us. Anaesthesia. 2021 Apr;76(4):444-447.
2. Soares-Weiser K, Lasserson T, Jorgensen KJ, et al. Policy makers must act on incomplete evidence in responding to COVID-19. Cochrane Database Syst Rev. 2020 Nov 20;11(11):ED000149.
3. Avenell A, Stewart F, Grey A, Gamble G, Bolland M. An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature. BMJ Open. 2019 Oct 30;9(10):e031909.
4. Higgins JP, Altman DG, Gøtzsche PC, et al. Cochrane Bias Methods Group; Cochrane Statistical Methods Group. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011 Oct 18;343:d5928.
5. Weibel S, Popp M, Reis S, et al. Identifying and managing problematic trials: A research integrity assessment tool for randomized controlled trials in evidence synthesis. Res Synth Methods. 2023 May;14(3):357-369.