In real-world evidence (RWE) studies, bias is a constant phenomenon, often driven by methods of patient selection, data capture, and defining outcomes.(1) Unlike randomized controlled trials (RCTs) that are designed to minimize systematic error, RWE studies often compete with the routine clinical practice and administrative data. Therefore, more than identifying bias; quantifying the distorted results is crucial.(1-4)

Quantifying bias is different from simply adjusting for known issues. Quantification enables the estimation of the possible impact of systematic error on effect estimates, providing transparency and a guideline to evaluate the relaibility of findings. Several methods have been developed to support this.(5, 6)

Quantitative Bias Analysis (QBA) is one of the most organized methodologies. It unambiguously mimics the direction and magnitude of bias through assumptions about misclassification, selection, or measurement error. Deterministic QBA offers corrected estimates with fixed scenarios, while probabilistic QBA helps describe the uncertainty in bias parameters through simulations. QBA essentially gives decision-makers a range of possible outcomes reflecting real-world imperfections, rather than giving a single ‘true’ estimate.(5-7)

Another distinct tool is the E-value, which measures the minimum strength for an unmeasured bias, supported with both the exposure and outcome, to fully justify an observed correlation.(4, 5) A higher E-value represents more robust results against an unmeasured bias. E-values, although more commonly used for unmeasured confounding, can be a general standard for result stability across multiple types of bias.(5, 7, 8)

Sensitivity analyses are supplementary to these tools as they assess the variability of results under different assumptions.(9) Whether it’s reconsidering exposure time windows, adjusting definitions of outcomes, or simulating different missing data scenarios, sensitivity analyses help assess the validity of a study’s conclusions under reasonable alternative conditions.(5, 7, 9)

Negative control analyses also serve as an important tool to identify hidden bias. These work by causing unrelated exposures or outcomes for researchers to notice residual bias, which can often be missed by standard models. Spotting a signal where none should exist causes concerns for data validity or procedural flaws.(5-7)

All these quantifying techniques don’t aim to eliminate bias, but rather to make its impact noticeable. For instance, all these parameters might show different levels of biases; yet, together, they move the interpretation from binary “yes/no” inferences to informed judgments about the amount of confidence in a particular evidence.(5-7)

As RWE increasingly becomes an important driver of regulatory and clinical decisions, quantifying bias should become a routine practice. It facilitates transparency, duplicability, as well as more detailed conversations about evidence quality.

Become A Certified HEOR Professional – Enrol yourself here!

References

1. Gokhale M, Stürmer T, Buse JB. Real-world evidence: the devil is in the detail. Diabetologia. 2020; 63:1694-1705.
2. Kim HS, Kim JH. Proceed with Caution When Using Real World Data and Real World Evidence. J Korean Med Sci. 2019 Jan 16;34(4):e28.
3. Maihöfner C, Mallick-Searle T, Vollert J, Kalita P, Sood Sethi V. Review of Challenges in Performing Real-World Evidence Studies for Nonprescription Products. Pragmat Obs Res. 2025; 16:7-18.
4. Bykov K, Patorno E, D’Andrea E, et al. Prevalence of Avoidable and Bias-Inflicting Methodological Pitfalls in Real-World Studies of Medication Safety and Effectiveness. Clin Pharmacol Ther. 2022; 111(1):209-217.
5. Petersen JM, Ranker LR, Barnard-Mayers R, MacLehose RF, Fox MP. A systematic review of quantitative bias analysis applied to epidemiological research. Int J Epidemiol. 2021 Nov 10;50(5):1708-1730.
6. Shi X, Liu Z, Zhang M, Hua W, Li J, Lee JY, Dharmarajan S, Nyhan K, Naimi A, Lash TL, Jeffery MM, Ross JS, Liew Z, Wallach JD. Quantitative bias analysis methods for summary-level epidemiologic data in the peer-reviewed literature: a systematic review. J Clin Epidemiol. 2024 Nov;175:111507.
7. Ramagopalan S, et al. Quantifying Bias in Real-World Studies: A New Hope for RWD Acceptance or Are HTAers Gonna Hate? [Accessed online on 4^th July 2025]. Available at: https://www.ispor.org/docs/default-source/euro2022/ispor-ad-symposium-combined-slides—final-v4.pdf?sfvrsn=9e41ba3_0
8. Barberio J, Ahern TP, MacLehose RF, et al. Assessing Techniques for Quantifying the Impact of Bias Due to an Unmeasured Confounder: An Applied Example. Clin Epidemiol. 2021; 13:627-635.
9. Greenland, S. (2014). Sensitivity Analysis and Bias Analysis. In: Ahrens, W., Pigeot, I. (eds) Handbook of Epidemiology. Springer, New York, NY. https://doi.org/10.1007/978-0-387-09834-0_60