#HEORInsights

A systematic literature review (SLR) is considered the highest form of evidence due to its rigorous approach through which every relevant piece of published or unpublished literature that is currently available to address a specific research issue is pooled and analyzed to answer a research question. For this reason, it is essential that the risk of bias (ROB) is assessed for all studies that are included in the SLRs. ROB assessment ensures the transparency of the outcomes, the validity of evidence, confidence, and reproducibility of the SLR process. It is typically done by identifying systematic errors or limitations in the conduct, design, or analysis of each included study in SR. Further, ROB assessment of the included studies is also a requirement for optimal reporting of SLRs, recommended by the PRISMA 2020 statement. (1,2)

The tool for ROB assessment depends on the study design of the included studies. For SLRs including RCTs, different tools are available for ROB assessment; one such tool is the Cochrane risk of bias tool version 2.0 (RoB-2), that was published in 2019 as an upgrade to the previous Cochrane RoB tool. According to the developers, the Cochrane RoB-2 is suitable for assessing ROB in individually-randomized, parallel-group, and cluster- randomized trials. (3) Other tools that are used for assessing ROB of RCTs include the EPOC RoB Tool for complex interventions randomized trials, the Critical Appraisal Skills Programme (CASP) checklist, the Joanna Briggs Institute (JBI) critical appraisal checklist, and the Scottish Intercollegiate Guidelines Network (SIGN) critical appraisal checklists for assessing methodological quality of different study types, including RCT. (4-6) Out of all of these available tools, the choice of the most appropriate tools depends on the research question, domain coverage of the tool, availability of the tool, the type of scoring system used, and any specific regulatory requirement.

For SLRs including non-randomized studies, a frequently used tool is the ROBINS-I (Risk Of Bias In Non-randomized Studies – of Interventions) tool. Apart from this, other popular tools include the JBI critical appraisal checklist for non-randomized experimental studies, the EPOC RoB tool, and the methodological index for non-randomized studies (MINORS) tool. (1,5,6)

There are several available tools for observational studies. To name a few, the CASP cohort study checklist, the SIGN critical appraisal checklists, the NIH quality assessment tool, the Newcastle-Ottawa Scale, and JBI critical appraisal checklist are the recommended tools for cohort study and Case-control studies. The Appraisal tool for Cross-Sectional Studies (AXIS) is another recommended tool for cross-sectional studies. Additionally, MINORS and the JBI Critical Appraisal tool can be used for ROB assessment of case series and case reports. (4-7) If an SLR includes more than one type of study design, the ROB assessment must use multiple tools based on the study design. Alternatively, a recently developed mixed methods appraisal tool (MMAT) can be used in such a situation.(8) Finally, for SLRs of SLRs, tools such as ROBINS and AMSTAR-2 are available, which can assess the risk of bias or methodological quality of the included SLR.(9,10)

While assessing ROB of the included studies in an SLR, it is essential to ensure that the latest version of the appropriate tool is used, and that the tool is validated and reliable. It is also recommended that independent assessment of ROB is carried out, and the results consolidated so that a comprehensive ROB assessment is performed. Finally, the reporting of ROB of the included studies must follow the recommendations of the tool.

While systematic reviews are the gold standard of evidence synthesis, their findings highly depend on the validity of their assessed studies. Therefore, the researchers must ensure that these studies are reliable and free from bias, leading to more accurate conclusions and evidence-based recommendations.

Become A Certified HEOR Professional – Enrol yourself here!

References

1. Zeng X, Zhang Y, Kwong JS, et al. The methodological quality assessment tools for preclinical and clinical studies, systematic review and meta-analysis, and clinical practice guideline: a systematic review. J Evid Based Med. 2015 Feb;8(1):2-10.
2. Jüni P, Altman DG, Egger M. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ. 2001 Jul 7;323(7303):42-6.
3. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019 Aug 28;366:l4898
4. Nadelson S, Nadelson LS. Evidence‐based practice article reviews using CASP tools: a method for teaching EBP. Worldviews on Evidence‐Based Nursing. 2014 Oct;11(5):344-6.
5. Baker A, Young K, Potter J, Madan I. A review of grading systems for evidence-based guidelines produced by medical specialties. Clinical medicine. 2010 Aug;10(4):358.
6. Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. Int J Evid Based Healthc. 2015;13(3):147–153.
7. Downes MJ, Brennan ML, Williams HC, Dean RS. Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open. 2016 Dec 8;6(12):e011458. doi: 10.1136/bmjopen-2016-011458.
8. Hong QN, Gonzalez-Reyes A, Pluye P. Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the Mixed Methods Appraisal Tool (MMAT). J Eval Clin Pract. 2018 Jun;24(3):459-467.
9. Whiting P, Savović J, Higgins JP, Caldwell DM, Reeves BC, Shea B, Davies P, Kleijnen J, Churchill R; ROBIS group. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016 Jan;69:225-34.
10. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, Moher D, Tugwell P, Welch V, Kristjansson E, Henry DA. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017 Sep 21;358:j4008.