#MarksManInsights

Health Technology Assessment (HTA) plays a critical role in evaluating the value of medical interventions. With the increasing availability of Electronic Health Records (EHRs), the integration of real-world data (RWD) into HTA has become more prevalent. The adoption of EHR data into HTA offers significant opportunities to enhance various aspects of health and medicine, from evaluating medical products and technologies to improving healthcare delivery. As a rich source of RWD generated at the point of care or during daily activities, EHR systems provide detailed insights into patients’ health and care. EHR data have been utilized for clinically relevant research, such as monitoring quality of care and medication adherence, and developing clinical predictive models.[1]

However, the use of EHR data in HTA presents unique challenges. Only a portion of EHR information is structured and ready for statistical analysis, with completeness and accuracy often compromised since EHRs are primarily collected for clinical or administrative purposes and not necessarily for research purposes. This affects the types, detail, and reliability of variables collected, and the timeliness of data availability. Relevant data may be missing due to the problem-focused nature of clinical summaries. Additionally, a single EHR system may not capture a patient’s entire clinical history if they receive care from multiple settings with different EHR systems. Therefore, data may need to be gathered from various healthcare entities and linked with other sources, such as disease registries, pharmacy data, national birth and death registrations, and claims. Time lags between clinician use of EHR information and analyst access to EHR data for decision-making are also common.[2-5]

To address these issues, the Professional Society for Health Economics and Outcomes Research (ISPOR) has developed the SUITABILITY checklist.[1] This checklist is designed to assess the appropriateness and quality of RWD sources, including EHRs, for HTA.[1]

The ISPOR SUITABILITY Checklist contains two main elements: data delineation and data fitness for purpose. Data delineation provides a comprehensive understanding of the data and assesses their trustworthiness by describing data under three headings: data characteristics, provenance, and governance. On the other hand, data fitness for purpose examines two main components: the accuracy and completeness of items (data reliability) and the suitability of the data to answer the particular question at hand (data relevance). Data relevance is assessed by examining whether the data aligns with the research question or HTA objective, considering the population, intervention, comparator, outcomes, and settings of interest. Core issues for data relevance include data content; care settings and the time period of interest; and sample size and follow-up period.[1]

Ensuring relevance, completeness, accuracy, timeliness, and generalizability is crucial for deriving meaningful insights and making informed decisions that impact a wide range of patients. Completeness evaluates whether the EHR data includes all necessary information to comprehensively address the research question, checking for missing data, gaps, and the presence of all relevant variables. Accuracy involves verifying the precision of recorded information to ensure it accurately reflects real-world clinical scenarios, while timeliness assesses whether the data is current and reflects contemporary clinical practices. Generalizability evaluates the extent to which findings from the EHR data can be applied to the broader population, examining the representativeness and diversity of the patient population included.[1,6-8]

The checklist also addresses potential biases and confounding factors, and promotes transparency and reproducibility by providing a standardized framework for evaluating EHR data, which allows for consistent documentation and verification of research methodologies. Ultimately, the checklist supports robust decision-making and policy development by providing reliable evidence for assessing medical technologies, including their safety, efficacy, cost-effectiveness, and overall impact on patient outcomes and healthcare systems.[1,9]

Digital health products offer new ways to manage and monitor care. In regulatory agencies, this demand is driven by the influx of innovative technologies and the need for quicker assessments. In this challenging environment, EHR-derived data hold promise for meeting information needs that traditional RWD platforms struggle to address. The task force’s framework and checklist are expected to evolve as experience with EHR-derived data increases.[5-8]

The ISPOR task force also acknowledge the limitations of the checklist, such as not accounting for national or local EHR data system idiosyncrasies. Secondly, the checklist offers a broad categorization of data suitability components rather than explicit standards for data provenance, reliability, or relevance. Thirdly, the benefits of integrating EHR data with other data types have not been considered. These limitations notwithstanding, it is expected that as experience with EHRs grows, we will have a better understanding of the nature of data sources that are better suited for specific HTA questions. Addressing some components of the SUITABILITY framework may require significant efforts and resources, and some information might not be accessible to end users. With the rapid advancement of AI, reimagining how unstructured EHR information is transformed into EHR-derived data will likely necessitate updates to the framework and checklist.[1,10,11]

The ISPOR SUITABILITY checklist is a crucial resource for effectively utilizing EHR data in HTA. By evaluating critical aspects like relevance, completeness, accuracy, timeliness, and generalizability, the checklist improves the quality, reliability, and validity of EHR data. It fosters transparent and reproducible research, supports evidence-based decision-making, and enhances the integration of RWD in HTA processes. As EHR data usage expands, the SUITABILITY checklist will continue to be vital for conducting thorough and influential HTAs, ultimately contributing to better healthcare outcomes globally.

Become A Certified HEOR Professional – Enrol yourself here!

References:

1. Fleurence RL, Kent S, Adamson B, Bouée-Benhamiche E, García Martí S, Ramsey S. Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force. ISPOR Report. 2024 Jun;27(6):692-701.
2. Califf RM, Robb MA, Bindman AB, et al. Transforming evidence generation to support health and health care decisions. N Engl J Med. 2016;375(24):2395– 2400.
3. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence – what is it and what can it tell us? N Engl J Med. 2016;375(23):2293–2297.
4. Graili P, Guertin JR, Chan KKW, Tadrous M. Integration of real-world evidence from different data sources in health technology assessment. J Pharm Pharm Sci. 2023;26:11460.
5. Shadmi E, Flaks-Manov N, Hoshen M, et al. Predicting 30-day readmissions with preadmission electronic health record data. Med Care. 2015;53(3):283–289.
6. Duke Margolis Center for Health Policy. Determining Real-World Data’s Fitness for Use and the Role of Reliability; 2019:1–54. Available from: https://healthpolicy.duke.edu/ sites/default/files/2019-11/rwd_reliability.pdf
7. US Food and Drug Administration. Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory DecisionMaking for Drug and Biological Products; 2021:1–39. Available from: https://www.fda.gov/ media/152503/download.
8. European Medicines Agency. Data Quality Framework for EU medicines regulation; 2023:1–42. Available from: https://www.ema.europa.eu/en/documents/regulatory-proceduralguideline/data-quality-framework-eu-medicines-regulation_en.pdf.
9. O’Rourke B, Oortwijn W, Schuller T, International Joint Task Group. The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care. 2020;36(3):187–190.
10. Fleurence RL, Shuren J. Advances in the use of real-world evidence for medical devices: an update from the national evaluation system for health technology. Clin Pharmacol Ther. 2019;106(1):30–33.
11. Desai RJ, Matheny ME, Johnson K, et al. Broadening the reach of the FDA Sentinel system: a roadmap for integrating electronic health record data in a causal analysis framework. NPJ Digit Med. 2021;4(1):170.