#HEORInsights

In the realm of medical research, the credibility and accuracy of published articles are paramount. Healthcare professionals rely on these articles to make informed decisions regarding patient care, treatment modalities, and developing clinical guidelines. However, the presence of bias in scientific studies can significantly undermine the validity and trustworthiness of their findings, potentially leading to misguided conclusions and inappropriate healthcare practices. The emergence of artificial intelligence (AI) has sparked considerable interest in utilizing its capabilities to assist in assessing bias in published articles.(1)

Bias can occur at various stages of the research process, including study design, data collection, analysis, interpretation, etc. Identifying and minimizing bias is crucial to ensure that research findings are unbiased, reliable, and can effectively translate into clinical practice. Typically, this assessment involves thoroughly examining various aspects of a study, such as study design, methodology, data collection, analysis, and reporting. Experts evaluate factors that may introduce bias, including conflicts of interest, selective reporting, inadequate blinding or randomization, and other potential sources of bias. This manual process requires expertise and can be time-consuming, especially when analyzing a large number of articles. The introduction of artificial intelligence (AI) in risk of bias assessment offers several advantages over traditional means. By leveraging machine learning algorithms, AI tools can identify patterns and indicators of bias in titles, abstracts, and full-text articles. This technology accelerates the screening process, increases consistency in assessments, and provides additional insights into potential biases.(2)

While AI cannot replace human expertise, it serves as a valuable tool for initial screening and prioritization, enabling researchers and clinicians to focus their attention on articles with a lower risk of bias and facilitating evidence-based decision-making in a more timely and efficient manner. The integration of AI in assessing the risk of bias in published articles signifies a significant advancement, promising enhanced reliability and objectivity in evaluating scientific literature.(3)

AI algorithms can analyze vast amounts of data and identify patterns that might be challenging for humans to detect. In recent years, researchers have developed AI-based tools and techniques to assist in assessing the risk of bias in scientific studies. These tools utilize machine learning algorithms to evaluate published articles based on predefined criteria and indicators of bias.(3)

While AI has shown promise in assessing bias in scientific literature, it is crucial to emphasize the need for collaboration between researchers, clinicians, and AI experts. By combining domain expertise and technical knowledge, interdisciplinary teams can develop more accurate and reliable AI models. Ongoing research and development are necessary to refine AI models, improve their performance in detecting various types of bias, and address the limitations, such as the reliance on limited text information.(4, 5)

The application of AI in assessing the risk of bias (ROB) in scientific articles is not without its challenges. AI models may lack contextual understanding, struggle with interpretation and identifying subtle bias, and have limited adaptability to evolving research practices. They can also perpetuate biases present in training data, raise ethical concerns, and create accountability challenges. Further, despite reasonably accurate predictions, the imperfections of AI models highlight the need for manual verification to ensure comprehensive and reliable assessments.(5)

While AI has showcased promise in assessing bias within the scientific literature, a collaborative approach between researchers, clinicians, and AI experts is vital for further advancement. The fusion of domain expertise and technical acumen within interdisciplinary teams can foster the development of more accurate and reliable AI models. Continued research and development efforts are essential to refine existing models, augment their performance in detecting diverse types of bias, and address inherent limitations, such as the dependence on limited textual information.

Become a Certified HEOR Professional – Enrol yourself here!

References:

1. Jardim PS, Rose CJ, Ames HM, et al. Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system. BMC Medical Research Methodology. 2022 Jun 8;22(1):167.
2. Arno A, Elliott J, Wallace B, Turner T, Thomas J. The views of health guideline developers on the use of automation in health evidence synthesis. Systematic Reviews. 2021 Dec;10:1-0.
3. Soboczenski F, Trikalinos TA, Kuiper J, et al. Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study. BMC medical informatics and decision making. 2019 Dec;19:1-2.
4. Marshall IJ, Kuiper J, Wallace BC. Automating risk of bias assessment for clinical trials. IEEE J Biomed Health Inform. 2015 Jul;19(4):1406-12.
5. Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc. 2016 Jan;23(1):193-201.