Assessing the risk of bias (RoB) in diagnostic studies is crucial for determining the reliability of their findings. Unlike interventional trials that assess treatment outcomes, diagnostic studies explore the ability of diagnostic tests to correctly identify the presence or absence of a disease condition. Bias in such studies can result in both an overestimation or underestimation of the true diagnostic accuracy of a test, which may mislead the clinical practice. To avoid this, reviewers must carefully assess the study design, execution, and reporting to identify any factors that could alter the results.(1-4)

One of the most commonly applied and recommended tools for assessing the RoB in diagnostic studies is the “Quality Assessment of Diagnostic Accuracy Studies” (QUADAS-2), which analytically evaluates RoB across four domains, viz. patient selection, index test, reference standard, and flow and timing. Every single domain characterizes a common point in the study that may potentially lead to bias. Along with bias, the QUADAS-2 also encourages reviewers to factor in the applicability of the study results to the clinical context. The strength of QUADAS-2 lies in its structured approach and adaptability, allowing for adaptation based on the specific review setting.(1, 4)

For studies directly comparing two or more diagnostic tests, the QUADAS-C tool, which is an extension of QUADAS-2, is specifically applicable.(5) Studies on comparative diagnostic accuracy present unique RoB, including differential verification, incorporation bias, or test application discrepancies. QUADAS-C helps address these risks by means of domains that parallel QUADAS-2 but involve signalling questions specific to the comparative context. This enables a more appropriate assessment of methodological accuracy and internal validity in direct test assessments. This tool also upholds transparent, reproducible views about bias like QUADAS-2 and is increasingly implemented in comparative diagnostic systematic reviews.(5)

The selection of participants is among the most common sources of bias. If the patient population in the study lacks representation (for instance, representing healthy volunteers or obvious disease cases), the findings may not apply to real-world clinical settings. Such spectrum bias can alter the estimates of sensitivity and specificity of diagnostic tests. Preferably, diagnostic studies should randomly or consecutively enrol patients, thus avoiding case-control designs unless necessary. Inappropriate exclusions or selective enrolment can also result in partial verification bias, further limiting interpretability.(1, 2, 4)

Conduct and interpretation of the index test under evaluation is another significant domain considered in QUADAS-2. If the test is interpreted with an understanding of the reference standard result, observer bias may impact the assessment; thus necessitating the aspect of blinding. Also, diagnostic limits should be clearly specified prior to the study initiation. Changing these limits post hoc to improve accuracy measures can potentially cause bias, as it customizes results to the sample rather than representing test performance in a neutral population.(2, 4)

The reference standard, an approach that helps establish the true disease status, should ideally be the most accurate and applied consistently. Flawed standard or the standard inferred with prior knowledge of the index test results can cause bias and hamper the accuracy measures. Moreover, using different reference standards for different groups within the study, also known as differential verification, can also mislead the test results. Therefore, both the choice of reference standard and its reliable implementation are crucial to the authenticity of a diagnostic accuracy study.(1, 2, 4)

Another subtle but crucial factor is the timing. A long delay between conducting the index test and performing the reference standard can lead to disease progression or regression, which may change the true status of the disease. In case of loss of follow-up, or not all patients receiving the reference standard, there can be attrition and verification bias. Therefore, maintaining a short, clinically appropriate time interval between tests and considering all enrolled patients in the final analysis are important to help preserve internal validity.(3, 4)

QUADAS-2 leads reviewers through each of these domains by integrating signalling questions, i.e., targeted prompts that direct attention to possible risks or methodological challenges. These questions are customized to the specific context of a review, which necessitate reviewers to make informed decisions on levels of risk bias, such as low, high, or unclear risk. Ideally, this action is performed by two independent reviewers, with an agreement to address inconsistencies and improve objectivity and reproducibility.(4)

In conclusion, evaluating RoB in diagnostic studies is a refinement-oriented and context-focused assessment of study design, execution, and transparency. Tools like QUADAS-2 and QUADAS-C facilitate a structured and systematic method for this assessment, thus helping identification and representation of biases that may concede the accuracy of test performance estimates. Critical appraisal aspects, such as patient selection, test blinding, reference standard reliability, and study timing, can help make better judgements about the integrity of diagnostic evidence and its generalizability to clinical decision-making.

Become A Certified HEOR Professional – Enrol yourself here!

References

1. Hall MK, Kea B, Wang R. Recognising Bias in Studies of Diagnostic Tests Part 1: Patient Selection. Emerg Med J. 2019 Jul;36(7):431-434.
2. Schmidt R, Factor RW. Understanding Sources of Bias in Diagnostic Accuracy Studies. Archives of Pathology & Laboratory Medicine. 2013; 137(4):558-65.
3. Di Girolamo N, Winter A, Meursinge Reynders R. High and unclear risk of bias assessments are predominant in diagnostic accuracy studies included in Cochrane reviews. J Clin Epidemiol. 2018 Sep;101:73-78.
4. Whiting PF, et al; QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011 Oct 18;155(8):529-36.
5. Yang B, Mallett S, Takwoingi Y, et al; QUADAS-C Group. QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies. Ann Intern Med. 2021 Nov;174(11):1592-1599.
6. Bezerra CT, Grande AJ, Galvão VK, et al. Assessment of the strength of recommendation and quality of evidence: GRADE checklist. A descriptive study. Sao Paulo Med J. 2022 Nov-Dec;140(6):829-836.