• Addressing Confounding in RWE studies

    Addressing Confounding in RWE studies
    Addressing Confounding in RWE studies

    Real-world evidence (RWE) studies strongly complement the conventional randomized controlled trials (RCTs), offering crucial insights into the performance of interventions in routine clinical settings. However, one of the most persistent practical concerns in RWE studies is the issue of confounding, which occurs owing to the inherent biases in real-world data (RWD). Unlike RCTs, where randomization factors in both known and unknown covariates amongst study groups, RWE studies are usually based on observational data, where treatment allocation is not arbitrary.[1] This results in confounding variables, i.e. factors that are influenced by both the treatment and the outcome, which can distort the estimated effects of interventions.[2]

    The first step in addressing confounding in RWE studies is a robust study design. Researchers must be cautious while evaluating the data source, conditions for cohort selection, and timing of assessment of covariates to ensure that potential confounders are well-defined. It is essential to recognize a distinct temporal connection between exposure, confounders, and outcome. Mispositioning in these time points can result in biased associations, especially if covariates are influenced by the treatment itself or are quantified post-exposure. Careful design selection can lower this risk before applying any statistical adjustment.[2, 3]

    Statistical methods play a key role in addressing confounding in RWE. Techniques like multivariable regression, inverse probability of treatment weighting (IPTW), instrumental variable analysis, and propensity score matching (PSM) are commonly applied; each method has its set of assumptions and limitations. For example, PSM can compare observed covariates between treatment groups, but they cannot justify unmeasured confounding. Instrumental variable methods, while theoretically strong, need even robust instruments, which may not be widely available in real-world datasets. These techniques seek to improve causal inference by simulating the balance achieved in RCTs. The choice of an appropriate method relies on the nature of data, the credibility of assumptions, and the research question.[2, 4, 5]

    Sensitivity analyses are important tools in assessing the strength and validity of findings in the presence of residual confounding. With variable assumptions, such as the robustness of unmeasured confounding or the model specifics, researchers can evaluate how much their results might be influenced by factors not included directly. Quantitative bias analysis, E-values, and negative control outcomes are some methods that can improve the reliability of study findings. These methods do not remove confounding but help analyse the potential extent of bias.[2, 6]

    Finally, transparent reporting is also essential for addressing confounding in RWE studies. Researchers should precisely define their methods for identifying, measuring, and adjusting for confounders, including the reasoning behind selected techniques and any limitations in the data. Communicating code lists, model specifications, and sensitivity analyses improves robustness and enables others to evaluate the authenticity of the findings. Established reporting guidelines, such as the STRengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement [7] for observational studies, provide a solid basis for transparency of findings. For studies considering routinely collected RWD, the REporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) statement, which is an extension of the STROBE statement, provides additional guidance particularly to RWE complexities.[8] Implementing such frameworks facilitates clearer communication of study design and results, making RWE more reliable and actionable.[6, 8]

    With the growing use of RWE in regulatory, clinical, and policy decision-making, carefully addressing confounding will be vital for ensuring reliable and actionable evidence generation.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Tashkin DP, Amin AN, Kerwin EM. Comparing Randomized Controlled Trials and Real-World Studies in Chronic Obstructive Pulmonary Disease Pharmacotherapy. Int J Chron Obstruct Pulmon Dis. 2020 Jun 2;15:1225-1243.
    2. Wang SV, Schneeweiss S. Assessing and Interpreting Real-World Evidence Studies: Introductory Points for New Reviewers. Clin Pharmacol Ther. 2022 Jan;111(1):145-149.
    3. Laurent T, Lambrelli D, Wakabayashi R, et al. Strategies to Address Current Challenges in Real-World Evidence Generation in Japan. Drugs Real World Outcomes. 2023 Jun;10(2):167-176.
    4. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. Chapter 6: Methods to address bias and confounding. Available at: https://encepp.europa.eu/encepp-toolkit/methodological-guide/chapter-6-methods-address-bias-and-confounding_en
    5. Chandramouli R. Statistical Methodologies in Real-World Evidence (RWE) for Medical Product Development. 2024. Available at: https://www.linkedin.com/pulse/statistical-methodologies-real-world-evidence-rwe-medical-r-bbglc
    6. Assimon MM. Confounding in Observational Studies Evaluating the Safety and Effectiveness of Medical Treatments. Kidney360. 2021 May 14;2(7):1156-1159.
    7. Von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Preventive medicine. 2007;45(4):247–51.
    8. Nicholls SG, Quach P, von Elm E, et al. The REporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) Statement: Methods for Arriving at Consensus and Developing Reporting Guidelines. PLoS One. 2015 May 12;10(5):e0125620.
  • Advancing Health Equity through Health Economics and Outcomes Research

    Advancing Health Equity through Health Economics and Outcomes Research

    In a world where access to healthcare should be a universal right, the persistent disparities in health outcomes starkly remind us that health equity remains an elusive goal. Health equity is defined as the absence of unfair and avoidable differences in health among population groups defined socially, economically, demographically, or geographically. This concept underscores the need to ensure that every individual has a fair opportunity to achieve their highest possible level of health, unimpeded by systemic barriers or socioeconomic disadvantages.[1]

    Health Economics and Outcomes Research (HEOR) examines the cost-effectiveness, value, and real-world outcomes of healthcare interventions, enabling data-driven resource allocation. HEOR is pivotal in advancing health equity by analyzing the economic and clinical outcomes of healthcare interventions. HEOR evaluates the value of medical treatments and services, providing data-driven insights that inform decisions about the allocation of healthcare resources. By intertwining health equity with health economics, HEOR helps identify which interventions deliver the most significant benefits across diverse populations, guiding efforts to reduce disparities and improve overall health outcomes.[2]

    The COVID-19 pandemic showed how social determinants of health, such as income, occupation, and access to technology, impact health equity. Higher-income individuals often had the advantage of working from home, reducing their exposure to the virus and benefiting from better internet connectivity, which enabled access to virtual healthcare. In contrast, essential workers in lower-paying jobs faced greater exposure and limited healthcare access, leading to worse outcomes. These disparities highlight the urgent need to reevaluate healthcare practices and structures to address inequities effectively.[3-5]

    HEOR plays a critical role in this re-evaluation by incorporating cost-effectiveness analysis (CEA) to determine which healthcare interventions provide the most value. CEA can also include equity-weighted analyses that prioritize interventions benefitting disadvantaged populations. CEA compares the costs and outcomes of various strategies, enabling policymakers to allocate resources efficiently, especially in settings with limited healthcare budgets. For example, analyzing cancer screening programs’ cost-effectiveness can identify the most beneficial approach for underserved communities, ensuring that resources are directed where they can achieve the greatest impact.[6]

    Furthermore, HEOR’s focus on real-world evidence (RWE) extends beyond controlled clinical trials to understand how treatments perform in everyday practice. This evidence is essential for addressing health disparities, as it reflects the diverse experiences of different patient populations. For instance, RWE during the pandemic showed how vaccine hesitancy and limited access impacted vaccination rates in underserved communities, guiding targeted outreach campaigns. Studies might show that a particular medication is less effective in certain ethnic groups due to genetic variations or disparities in healthcare access, prompting the development of tailored strategies to improve outcomes for these groups.[7]

    Patient-reported outcomes (PROs) and electronic PROs (ePROs) are vital components of HEOR that enhance health equity by capturing patients’ perspectives on their health status, quality of life, and treatment satisfaction. Incorporating PROs and ePROs ensures that the voices of all patients, including those from marginalized groups, are considered in healthcare decision-making. This approach helps uncover specific challenges faced by different populations and informs the design of more inclusive and effective interventions.[8]

    HEOR also influences policy decisions by providing evidence on the broader social determinants of health, such as education, housing, and employment. Highlighting the impact of these factors on health outcomes allows HEOR to advocate for integrated policies that address these root causes of inequity. For instance, research showing that stable housing reduces emergency room visits has prompted healthcare systems to invest in housing assistance programs. In addition, research might demonstrate that improving access to quality education and stable housing can lead to better health outcomes and reduced healthcare costs, supporting the case for holistic approaches that combine social and health policies.[5]

    The pharmaceutical industry benefits significantly from integrating health equity into HEOR. By including diverse populations in clinical trials and subsequent research, companies can develop treatments that are effective across different demographic groups. Failing to include diverse populations can lead to treatments that are less effective or even harmful for certain groups, underscoring the importance of inclusive research practices. This approach not only improves health outcomes but also enhances market access and ensures compliance with regulatory requirements focused on diversity and inclusion.[7]

    In conclusion, advancing health equity through HEOR is both a moral imperative and a strategic necessity for optimizing healthcare delivery and outcomes. By embedding equity considerations into economic evaluations, RWE, PROs, and policy research, HEOR can guide the development of more inclusive healthcare strategies. This comprehensive approach ensures that all individuals, irrespective of their background, have access to the care they need to lead healthy and fulfilling lives. Moving forward, it is essential for all healthcare stakeholders to prioritize health equity in their research and decision-making processes, paving the way for a fairer and healthier future for all.

    Become A Certified HEOR Professional – Enrol yourself here!

    References:

    1. Health equity. World Health Organisation. Available from: https://www.who.int/health-topics/health-equity#tab=tab_1
    2. Fautrel B. SP0124 Health economics and health equity: two complementary disciplines.2017;76:31.
    3. Closing the Gap in a Generation: Health Equity Through Action on the Social Determinants of Health. Final Report of the Commission on Social Determinants of Health. World Health Organization; 2008:1 -256. Available from: https://apps.who.int/iris/bitstream/handle/10665/43943/9789241563703_eng.pdf;jsessionid=365271ACE2052888542881700EEDCA8B?sequence=1.
    4. Burström B, Tao W. Social determinants of health and inequalities in COVID-19. European journal of public health. 2020 Aug 1;30(4):617-8.
    5. Thomas R, Chalkidou K. Cost–effectiveness analysis. InHealth system efficiency: How to make measurement matter for policy and management. 2016. European Observatory on Health Systems and Policies.
    6. Fendrick AM, ISPOR. Real-World Evidence: Additional Tool to Support Clinical Decision Making. Available from: https://www.ispor.org/docs/default-source/strategic-initiatives/ispor-rwe-byline-article_10-25-21.pdf?sfvrsn=687e4bc8_0
    7. Rosenberg SS, Carson BB, Kang A, Lee TH, Pandey R, Rizzo EJ. The Impact of Digital Health Technologies on Health Equity: Designing Research to Capture Patient-Reported Outcomes. ISPOR value & outcomes spotlight. Available from: https://www.ispor.org/publications/journals/value-outcomes-spotlight/vos-archives/issue/view/addressing-assessment-and-access-issues-for-rare-diseases/the-impact-of-digital-health-technologies-on-health-equity-designing-research-to-capture-patient-reported-outcomes.

  • Enhancing Credibility in Real-World Evidence Generation through SPACE and SPIFD Frameworks

    Enhancing Credibility in Real-World Evidence Generation through SPACE and SPIFD Frameworks
    Enhancing Credibility in Real-World Evidence Generation through SPACE and SPIFD Frameworks

    Over time, real-world evidence (RWE) has transitioned from a supplementary tool to a key driver in healthcare decision-making, bridging the gap between clinical trials and real-world practice. Recognized by regulatory bodies like the FDA, EMA, NICE, and CADTH, RWE offers insights into intervention effectiveness and safety across diverse populations. However, challenges persist, including credibility concerns highlighted by flawed COVID-19 studies, emphasizing the need for transparent, relevant data selection. To address this, frameworks like SPACE and SPIFD provide structured methodologies to enhance the design, data selection, and credibility of RWE.[1,2]

    Since 2021, regulatory and HTA bodies have published new or updated guidelines, necessitating rationalization and transparency of real-world study design and data source selection to ensure fitness-for-purpose in addressing specific research questions. Researchers have, therefore, been exploring tools to meet these standards. Among them is the Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence (SPACE) tool, introduced in 2019.[3,4]

    This SPACE framework provides a step-by-step process for identifying elements of real-world study design, minimal criteria to ensure feasibility and validity of data, and documentation of study design decisions, including the planned analysis. This structured approach significantly mitigates bias in research by ensuring that every aspect of the study design is systematically planned and documented, thereby supporting the initial steps in study design to identify suitable data or draft protocol documents.[3,5]

    The SPACE framework consists of several key steps that guide researchers through designing credible real-world studies. The first step involves formulating a clear research question that focuses on addressing specific healthcare needs. This is followed by identifying relevant study designs that align with the research question to ensure valid comparisons. Next, researchers assess the feasibility and validity of data sources by evaluating whether they can provide reliable information for answering the research question. Finally, all decision-making processes are documented comprehensively to enhance transparency and reproducibility throughout the study design process.

    In 2021, the Structured Process to Identify Fit-for-Purpose Data (SPIFD) was introduced as an extension to the SPACE framework. SPIFD offers a comprehensive, step-by-step process for conducting and documenting systematic data feasibility assessments to ensure data fitness for the research question. By thoroughly assessing the data sources, SPIFD enhances transparency and validity in data selection, crucially reducing bias by ensuring that only the most suitable data sources are used. Together, SPACE and SPIFD frameworks facilitate valid and transparent real-world comparative study design, planned analysis, and data selection, meeting the stringent standards required by regulators, HTAs, and payers.[6]

    The SPIFD framework also follows a structured methodology with specific steps to ensure appropriate data selection for RWE studies. First, researchers conduct systematic assessments of potential data sources to evaluate their suitability for addressing the research question. They then evaluate these sources for relevance, quality, and completeness to ensure they meet necessary standards for generating credible evidence. Finally, alignment with regulatory requirements is ensured so that selected datasets comply with applicable guidelines from regulatory bodies like FDA or EMA.[3,5]

    One of the most significant roles of the SPACE framework in generating credible RWE data is its focus on minimizing bias through meticulous study design. Adopting a target trial approach allows researchers to simulate the conditions of a randomized controlled trial (RCT) in the context of real-world setting. This practice ensures that the selected study design closely replicates the ideal experimental conditions, and helps to identify and address potential sources of bias at early stages of a study design process. Thus, enhancing the validity of results and making the evidence more reliable for decision-making.[4]

    Furthermore, the framework of SPIFD places immense emphasis on systematic assessment of data feasibility. This means that only data fit for the intended purpose shall be used for studies of real-world settings. SPIFD also aids researchers in selecting the most relevant datasets for their studies, by rigorously evaluating candidate data sources for their relevance, quality, and completeness. This thorough vetting process not only enhances transparency in the selection of data but also ensures that the resulting evidence is strong and credible. This alignment with stringent regulatory and HTA standards fosters greater confidence in the use of RWE for critical healthcare decisions.[6]

    To evaluate the effectiveness of studies designed using these frameworks, researchers can rely on several key metrics. For example, tracking the “percentage of relevant data sources identified” during feasibility assessments provides insights into how effectively suitable datasets were pinpointed. Similarly, monitoring the “number of potential biases mitigated” during study design helps gauge how well frameworks like SPACE address methodological challenges early on. These metrics provide tangible measures of success when applying these frameworks to RWE studies while ensuring alignment with regulatory expectations.[6]

    In 2023, the introduction of SPIFD2 marked a significant advancement by consolidating both the design and data aspects of the original SPACE and SPIFD templates. SPIFD2 ensures that users specify the correct real-world data (RWD) study design before assessing the feasibility of candidate data sources. This comprehensive framework captures potential sources of bias that may arise in the real-world emulation of the target trial, providing a robust mechanism for enhancing the credibility of RWE. By addressing both study design and data assessment in a unified framework, SPIFD2 offers a holistic approach to mitigate bias and improve the reliability of real-world studies.[3]

    In conclusion, the SPACE and SPIFD frameworks represent pivotal advancements in the field of real-world evidence generation, offering structured methodologies to tackle the complexities and challenges inherent in RWE studies. Where SPACE facilitates rigorous study design by adopting a target trial approach, the SPIFD framework ensures the selection of fit-for-purpose data through systematic assessment. By guiding study design and data source selection, these frameworks ensure that RWE studies are rigorous, transparent, and aligned with regulatory and HTA standards. The introduction of SPIFD2 in 2023 further enhances these frameworks, aligning them with evolving regulatory and HTA requirements. This unified approach improves the reliability and relevance of RWE, empowering informed healthcare decisions and advancing patient outcomes.

    Become A Certified HEOR Professional – Enrol yourself here!

    References:

    1. White R. Building trust in real world evidence (RWE): moving transparency in RWE towards the randomized controlled trial standard. Current Medical Research and Opinion. 2023 Dec 2;39(12):1737-41.
    2. Winterstein AG, Ehrenstein V, Brown JS, Stürmer T, Smith MY. A road map for peer review of real-world evidence studies on safety and effectiveness of treatments. Diabetes Care. 2023 Aug 1;46(8):1448-54.
    3. Gatto NM, Vititoe SE, Rubinstein E, Reynolds RF, Campbell UB. A structured process to identify fit‐for‐purpose study design and data to generate valid and transparent real‐world evidence for regulatory uses. Clinical Pharmacology & Therapeutics. 2023 Jun;113(6):1235-9.
    4. Gatto NM, Reynolds RF, Campbell UB. A structured preapproval and postapproval comparative study design framework to generate valid and transparent real‐world evidence for regulatory decisions. Clinical Pharmacology & Therapeutics. 2019 Jul;106(1):103-15.
    5. Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. American journal of epidemiology. 2016 Apr 15;183(8):758-64.
    6. Gatto NM, Campbell UB, Rubinstein E, Jaksa A, Mattox P, Mo J, Reynolds RF. The structured process to identify fit‐for‐purpose data: a data feasibility assessment framework. Clinical Pharmacology & Therapeutics. 2022 Jan;111(1):122-34.
  • Exploring Patient Registries: Managing Market Access With Real World Evidence

    Exploring Patient Registries: Managing Market Access With Real World Evidence
    Exploring Patient Registries Managing Market Access With Real World Evidence

    In the increasingly data-driven world of healthcare, patient registries have emerged as powerful tools for managing market access through the application of real-world evidence (RWE). These registries, essentially databases that collect information about patients’ demographics, clinical history, treatment outcomes, and other pertinent details, provide a treasure trove of insights that can bridge the gap between clinical trials and everyday medical practice. The use of patient registries in gathering RWE is transforming how pharmaceutical companies, healthcare providers, and policymakers evaluate and manage the accessibility and effectiveness of medical treatments and interventions.[1]

    Patient registries provide a comprehensive view of how treatments perform outside the controlled environment of clinical trials. Clinical trials, while essential for establishing the efficacy and safety of new therapies, often have limitations, such as restrictive inclusion criteria and relatively short follow-up periods. In contrast, patient registries encompass a broader patient population, including those with comorbidities and varying adherence levels, thus reflecting real-world scenarios more accurately. This inclusivity makes registries a valuable source of RWE, which is increasingly recognized by regulatory agencies and payers as critical for assessing the long-term value and impact of medical interventions. Regulators also view the use of registries as a key tool for generating RWE, supporting decision-making processes across areas such as post-market surveillance, submissions for regulatory approval, and health technology assessments (HTAs).[1]

    One of the primary benefits of utilizing patient registries is the ability to generate evidence on long-term safety and effectiveness of treatments. This ongoing evidence collection improves patient trust in the intervention, thereby increasing patient access. For instance, post-marketing surveillance through patient registries can uncover adverse effects or benefits that may not be apparent during pre-approval clinical trials. This continuous monitoring helps in ensuring that the treatments remain safe and effective once they are in widespread use, supporting regulatory decisions and guiding clinical practice. Consequently, this fosters greater confidence and willingness among patients to adopt new treatments, enhancing their accessibility and uptake in the general population.[2]

    Patient registries also play a pivotal role in market access by demonstrating the value of new treatments to payers and health technology assessment (HTA) bodies. The data derived from these registries can support health economic evaluations, such as cost-effectiveness and budget impact analyses, which are essential for reimbursement decisions. By providing robust evidence on the real-world effectiveness and safety of treatments, registries help build a compelling case for their adoption and coverage. This is particularly important in an era where healthcare budgets are constrained, and there is increasing scrutiny over the allocation of resources.[3]

    Despite their benefits, patient registries face challenges that must be acknowledged to provide a balanced perspective. Issues such as data quality inconsistencies, difficulties in patient recruitment, and regulatory hurdles can hinder their utility. For example, inconsistent data collection processes may impact the reliability of insights, and recruitment barriers can limit the diversity of patient populations within registries. Addressing these challenges requires standardizing methodologies, fostering collaboration among stakeholders, and ensuring compliance with evolving regulatory frameworks. Additionally, emerging trends such as the integration of digital health technologies and artificial intelligence (AI) are redefining the capabilities of patient registries. AI, for instance, can analyze large-scale registry data to identify patterns, predict treatment outcomes, and optimize resource allocation, enhancing the potential of RWE to inform decision-making.[4]

    Collaboration among stakeholders is critical for the success of patient registry initiatives. Engaging a diverse group of participants, including patients, healthcare providers, researchers, and industry partners, ensures that registries are comprehensive and relevant. Leveraging technologies such as electronic health records (EHRs) and mobile health applications can streamline data collection, improve accuracy, and enhance data utility. These collaborative and technological advancements position registries as powerful tools for addressing unmet medical needs, optimizing healthcare delivery, and achieving equitable access to treatments.[4,5]

    In conclusion, patient registries stand as pillars in the contemporary healthcare landscape, offering a bridge between clinical trials and real-world practice. Their role in managing market access through the application of RWE is indispensable, shaping how pharmaceuticals are evaluated, accessed, and integrated into healthcare systems. With their ability to capture the intricacies of patient experiences over time, these registries empower stakeholders to make informed decisions, driving advancements in healthcare delivery and ensuring that groundbreaking therapies reach those in need. By addressing challenges, embracing innovation, and fostering collaboration, patient registries will continue to play a transformative role in optimizing patient care and healthcare resource utilization.

    Become A Certified HEOR Professional – Enrol yourself here!

    References:

    1. Trotter JP. Patient registries: a new gold standard for “real world” research. Ochsner Journal. 2002 Sep 21;4(4):211-4.
    2. Reid CM. The role of clinical registries in monitoring drug safety and efficacy. Heart, Lung and Circulation. 2015 Nov 1;24(11):1049-52.
    3. Blommestein HM, Franken MG, Uyl-de Groot CA. A practical guide for using registry data to inform decisions about the cost effectiveness of new cancer drugs: lessons learned from the PHAROS registry. Pharmacoeconomics. 2015 Jun;33:551-60.
    4. Gliklich RE, Dreyer NA, Leavy MB. Patient registries. InRegistries for Evaluating Patient Outcomes: A User’s Guide [Internet]. 3rd edition 2014 Apr. Agency for Healthcare Research and Quality (US).
    5. Daugherty SE, Lee SB, Nowell B, Peay H, Solomon D, Valbrun TG, Velentgas P, Whicher D. The increasing focus on the patient in patient registries. In21st Century Patient Registries: Registries for Evaluating Patient Outcomes: A User’s Guide: 3rd Edition, Addendum 2018 Mar. Agency for Healthcare Research and Quality (US).
  • When is the Ideal Moment to Start HEOR Activities Within Product Development?

    When is the Ideal Moment to Start HEOR Activities Within Product Development?
    When is the Ideal Moment to Start HEOR Activities Within Product Development?

    Health Economics and Outcomes Research (HEOR) has grown into a vital domain in the health and care industry. It plays a key role by offering vital insights into how drugs in the market affect both health and finances. With the means of data driven and evidence-based insights, it plays a critical role in the development and dissemination of new healthcare products by providing important findings regarding clinical development strategies, informing regulatory decisions, and supporting market access. Medical affairs teams in pharmaceutical companies are often concerned about the ideal time for beginning HEOR activities within the product development cycle. The simple answer to this question is: the earlier, the better.[1]

    One can consider engaging in HEOR activities as early as the preclinical or discovery phases of product development. This way, HEOR insights can contribute significantly to the foundational understanding of the economic and clinical landscape. It has been observed that early HEOR involvement can result in more informed decision-making, and influence stakeholder engagement and investment decisions. HEOR helps identify and address potential market access barriers. By understanding the economic landscape and the outcomes that matter most to payers and patients, developers can tailor their products to meet these needs.[2]

    HEOR activities can further strengthen evidence generation in Phase I and II clinical trials. This period is considered critical for gathering preliminary data on safety, efficacy, and health-related quality of life (HRQoL) outcomes. Complementing this data with observations from HEOR studies can help refine economic models and identify the most relevant clinical endpoints. Additionally, HEOR data in early phases can be very useful to inform trial design in Phase III studies, ensuring that all necessary economic and QOL endpoints are captured, to support future health technology assessments (HTAs).[3]

    Advancing to Phase III clinical trials marks a very important milestone in the development of a new pharmaceutical product. At this stage, HEOR has a critical role of providing comprehensive data and evidence needed to confirm the product’s value proposition in the market. This is the phase where the most extensive and rigorous data on efficacy, safety, and outcomes are collected. HEOR activities in Phase III should define clinical endpoints, finalize economic models, undertake full cost-effectiveness analyses, and prepare dossiers required for regulatory submission and reimbursement applications. Embedding HEOR insights into Phase III activities can help predict and prevent potential concerns from regulatory agencies and payers. It showcases the value of the product not only through its clinical efficacy but also through the economic and patient-centered outcomes.[4,5]

    Following Phase III and the acquisition of marketing authorization, the product is launched in the market and becomes available for prescription to patients in the real-world setting. Throughout this stage, the accumulating real-world evidence (RWE) further helps in refining the value of the product by evaluating the efficacy and performance of the product in the diverse patient population. This evidence further substantiates the ongoing research by providing invaluable and robust insights into long-term outcomes, adherence patterns, and overall economic impact of the product. Continuous HEOR efforts post-launch help to support ongoing market access, reimbursement renewals, and potential label expansions. They also provide feedback that can be used to optimize the product’s use and improve patient outcomes over time.[5]

    In summary, HEOR is one continuous process that evolves over the whole cycle of product development. Early initiation of HEOR activities during the development process embeds economic and patient outcome considerations within the strategy of the product from the very beginning. This method not only increases the chances of a drug receiving the appropriate regulatory and reimbursement approval but also guarantees that new therapies are beneficial to both the patient and the medical system. Such proactive HEOR strategy, will put the pharmaceutical companies in a position to better navigate the complexities associated with market access, and to ensure that innovative therapies reach the patients who deserve them most.

    Become A Certified HEOR Professional – Enrol yourself here!

    References:

    1. Holtorf AP, Brixner D, Bellows B, Keskinaslan A, Dye J, Oderda G. Current and future use of HEOR data in healthcare decision-making in the United States and in emerging markets. American health & drug benefits. 2012 Nov;5(7):428.
    2. Van Nooten F, Holmstrom S, Green J, Wiklund I, Odeyemi IA, Wilcox TK. Health economics and outcomes research within drug development: challenges and opportunities for reimbursement and market access within biopharma research. Drug discovery today. 2012 Jun 1;17(11-12):615-22.
    3. Zou KH, Baker CL, Cappelleri JC, Chambers RB. Data Sources for Health Economics and Outcomes Research. InStatistical Topics in Health Economics and Outcomes Research 2017 Nov 22 (pp. 1-13). Chapman and Hall/CRC.
    4. Garrison Jr LP, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real‐world data for coverage and payment decisions: the ISPOR real‐world data task force report. Value in health. 2007 Sep;10(5):326-35.
    5. Chou JW, Portelli A, Cournoyer A. De-risking Market Launch: Leveraging HEOR Evidence Planning to Support Informed Payer Decisions. J Clin Pathways. 2022;8(8):63-65.
  • The Integration of Artificial Intelligence Solutions in Medical Affairs

    The Integration of Artificial Intelligence Solutions in Medical Affairs

    In the era of digital transformation, artificial intelligence (AI) acts as a catalyst to revolutionize the landscape of medical affairs. AI has the potential to disrupt the way a medical affairs department in a pharmaceutical company functions, by virtue of its capabilities in functions as diverse as data handling, data analysis, literature review, information retrieval, and cleaning. In fact, AI can also help medical affairs professionals focus their KOL engagement activities.

    Given the ability AI has to process vast amounts of data, uncover hidden patterns, and automate complex tasks, it is not surprising that AI has already started to transform the clinical trial data analytic landscape. Further, by the capacity to handle and analyze big data, AI has made it possible to assimilate vast amounts of real-world data (RWD) from various sources, including claims data, electronic health records (EHRs), registries, and social media as well, and to generate real-world evidence (RWE) through the analysis of RWD. This capability of handling big data has empowered medical affairs teams to discern elusive patterns and trends, thereby elevating decision-making capabilities.[1,2]

    Next, clinical trials have been known to face challenges such as prolonged recruitment times and suboptimal design. Here, AI algorithms prove instrumental, facilitating the identification of suitable patient populations, predicting enrolment rates, and optimizing protocols with efficiency and ethical considerations. This not only expedites trial completion but also fast-tracks the development of life-saving medications. [3]

    AI also plays a pivotal role in publication planning and medical writing. AI algorithms, in this context, conduct an automated literature review to identify gaps and opportunities for new publications. This ensures the informed strategic planning of publications that contribute meaningfully to scientific discourse. In medical writing, AI-powered tools enhance efficiency and contribute to the overall quality and value of publications by analyzing language patterns to meet both scientific and regulatory standards.[3]

    AI can also contribute to regulatory affairs by identifying and facilitating essential documentation. Further, it can also facilitate communication with relevant stakeholders, thereby ensuring smooth interaction between different departments leading to regulatory submission. This can ensure improved adherence to evolving guidelines and freeing resources for strategic efforts.[4]

    Moving on, AI also has an important role to play in enhancing KOL (key opinion leader) engagement strategies by helping in the refinement of communication and fostering effective dialogue between pharmaceutical companies and stakeholders. Through efficient social network analysis, AI identifies relevant KOLs, ensuring focused efforts for successful communication and market access. By using AI, it is also possible to craft targeted communication campaigns aligned with KOL expertise. AI-powered chatbots can enhance KOL engagement by providing them with on-demand access to essential information. AI can also facilitate planning and executing CMEs, matching the interest and expertise of KOLs with the CME topics. AI can further enhance the effectiveness of CMEs by aligning content with individual learning styles and preferences, ultimately contributing to the professional development of healthcare professionals and strengthening connections with influential leaders in the field.[5,6]

    Analysis of complex datasets leading to predictive modeling using AI can enable a thorough assessment of market dynamics, allowing pharmaceutical companies to make strategic decisions that optimize their market presence. AI also plays a crucial role in pricing strategies, dynamically adjusting them based on the intricate dynamics of healthcare systems, ensuring competitiveness and responsible access to innovative healthcare solutions.[7]

    Health Economics and Outcomes Research (HEOR) significantly benefits from AI’s capabilities. AI-powered systematic literature reviews can significantly shorten the time required for completion of the review, thereby enhancing the speed of market access. AI-driven models facilitate the assessment of cost-effectiveness and budget impact, empowering decision-makers with crucial insights that shape market access and reimbursement strategies.[7]

    Patient support programs are also revolutionized by AI, offering sophisticated and responsive personalized support. AI’s continuous monitoring contributes to enhanced patient adherence and outcomes, fostering a proactive approach to healthcare management for improved patient experiences and overall health outcomes.[8]

    In competitive intelligence, AI stands as a game-changer, continuously monitoring competitor activities and providing nuanced insights based on the competitive landscape. This empowers organizations to anticipate market shifts, identify strategic opportunities, and position themselves effectively within the dynamic healthcare ecosystem. Continuous monitoring of competitor activities, insights gained, and a proactive approach shaped by AI-driven analysis position pharmaceutical companies strategically in the marketplace.[8]

    AI has a role in social media analysis as well. By scanning social media for sentiment analysis, AI identifies emerging trends and unmet needs. Armed with these insights, medical affairs teams can deliver highly relevant information through tailored communication strategies, fostering deeper relationships and trust with both healthcare professionals and patients alike. Additionally, AI plays a pivotal role in forecasting drug demand, ensuring consistent supply chain logistics. This predictive capability minimizes stockouts and overstocks, contributing to a seamless healthcare system where patients have reliable access to the medications they need. Furthermore, AI facilitates communication through telemedicine platforms, breaking down geographical barriers and improving patient access to care.[9]

    Beyond the above-mentioned use cases, AI has been evolving and finding new applications to enhance the efficiency of a medical affairs department. Implementing AI solutions in medical affairs is not merely an upgrade but a strategic leap toward optimized performance and enhanced patient care. While challenges exist, careful planning, a focus on ethical considerations, and proactive change management can pave the way for successful integration and unlock the transformative potential of AI in this critical domain. By leveraging the power of AI, medical affairs teams can gain deeper insights, make data-driven decisions, and ultimately contribute to a more efficient, personalized, and effective healthcare system for all.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Bedenkov A, Moreno C, Agustin L, et al. Customer centricity in medical affairs needs human-centric artificial intelligence. Pharmaceutical Medicine. 2021 Jan;35(1):21-9.
    2. Yang H, Khatry DB. Reinventing Medical Affairs in the Era of Big Data and Analytics. InData Science, I, and Machine Learning in Drug Development 2022 Oct 3 (pp. 245-263). Chapman and Hall/CRC.
    3. Mayorga-Ruiz I, Jiménez-Pastor A, Fos-Guarinos B, et al. The role of AI in clinical trials. Artificial Intelligence in Medical Imaging: Opportunities, applications and risks. 2019:231-43.
    4. Khalifa AA, Ibrahim MA. Artificial intelligence (AI) and ChatGPT involvement in scientific and medical writing, a new concern for researchers. A scoping review. Arab Gulf Journal of Scientific Research. 2024 Jan 4.
    5. Patil RS, Kulkarni SB, Gaikwad VL. Artificial intelligence in pharmaceutical regulatory affairs. Drug Discovery Today. 2023 Jul 12:103700.
    6. Bedenkov A, Rajadhyaksha V, Beekman M, et al. Developing medical affairs leaders who create the future. Pharmaceutical medicine. 2020 Oct;34(5):301-7.
    7. Farah L, Borget I, Martelli N. International Market Access Strategies for Artificial Intelligence–Based Medical Devices: Can We Standardize the Process to Faster Patient Access? Mayo Clinic Proceedings: Digital Health. 2023 Sep 1;1(3):406-12.
    8. Hoffman FP, Freyn SL. The future of competitive intelligence in an AI-enabled world. International Journal of Value Chain Management. 2019;10(4):275-89.
    9. Roski J, Gillingham BL, Just E, et al. Implementing and scaling artificial intelligence solutions: considerations for policy makers and decision makers. Health Affairs Forefront. 2018.
  • Innovation in Rare Disease Research through Integration of Real-World Evidence in HTA

    Innovation in Rare Disease Research through Integration of Real-World Evidence in HTA

    For individuals grappling with rare diseases, finding effective treatment options can be an overwhelming challenge. The conventional pathways of drug development, tailored for larger patient cohorts, encounter numerous obstacles in the context of rare diseases. This leads to delays in innovation and impedes access to potentially life-changing therapies. Fortunately, the emergence of real-world evidence (RWE) offers a promising solution to overcome these challenges and reshape the landscape of innovation in rare diseases.[1]

    The term “rare disease” encompasses a diverse array of conditions affecting a limited number of individuals. This small patient population presents a substantial hurdle for traditional clinical trial design, which relies on large cohorts to establish statistically significant efficacy and safety. Recruiting an adequate number of participants for rare disease trials is a time-consuming and expensive process, often necessitating international collaboration and specialized expertise. Additionally, the heterogeneity of rare diseases, with their varied presentations and disease trajectories, further complicates the development process.[1]

    The traditional HTA process relies heavily on data generated from controlled clinical trials, which may not adequately capture the complexities and variabilities associated with rare diseases. Moreover, the rarity of these conditions often means that traditional clinical trials include a limited number of patients, making it challenging to generate statistically significant results. This limitation not only hampers the regulatory approval of new drugs but also affects the subsequent HTA evaluations.[2]

    RWE offers a powerful tool to address these challenges and accelerate innovation in rare disease research and development. RWE encompasses data collected outside of traditional clinical trials, including electronic health records, claims databases, patient registries, and wearable devices. RWE empowers regulatory agencies and HTA bodies, offering the potential to streamline the approval and reimbursement processes for innovative drugs in rare diseases.[2,3]

    RWE can be used to quantify the unmet medical need by providing valuable insights into the prevalence, burden, and impact of rare diseases. This data not only highlights the necessity for new treatment options but also informs cost-effectiveness analyses, estimating the costs associated with rare disease management and potential cost savings from innovative treatments. Additionally, RWE contributes to evaluating the long-term value of therapies, offering insights into their impact on patient outcomes and healthcare utilization. This comprehensive approach aids HTA bodies in assessing the overall value proposition of new therapies, ultimately expediting access to effective treatments for patients in need.[3]

    RWE significantly drives innovation for rare diseases, offering diverse benefits. Firstly, RWE can be used to identify and recruit patients who meet specific inclusion criteria, even for geographically dispersed populations, leading to faster and more efficient trial completion. Additionally, RWE supports adaptive pathways that align with personalized medicine. This approach allows continuous learning and adaptation based on real-world experiences, tailoring treatments to individual patient characteristics and needs.[3,4]

    The integration of RWE into HTA processes brings forth a range of advantages, yet it is not without its challenges. Notably, the quality and standardization of real-world data emerge as critical considerations, demanding continuous efforts to establish common standards and enhance data quality for credible HTA evaluations. Additionally, the reliance on patient data from real-world settings in RWE necessitates a delicate balance between data access and patient confidentiality, highlighting the ongoing challenge of addressing privacy and ethical concerns.[3]

    Education and adoption present another layer of complexity, with stakeholders such as regulatory agencies, HTA bodies, healthcare professionals, and pharmaceutical companies requiring comprehensive awareness of the benefits and limitations of RWE. This underscores the need for active promotion and facilitation of RWE adoption in decision-making processes. In the context of rare diseases, these challenges are amplified, prompting innovative approaches to evidence generation.[3]

    To fully harness the potential of RWE in rare disease research and HTA, the establishment of a robust infrastructure is imperative. This involves standardizing and harmonizing data collection, analysis, and reporting across different sources to ensure reliable and comparable evidence. Investment in technologies that facilitate data sharing and integration from diverse sources is essential for enhanced data capture. Building trust in RWE necessitates transparency in data sources, methodologies, and limitations, requiring open dialogue with patients, researchers, and HTA bodies. Additionally, clear regulatory guidelines and frameworks for utilizing RWE in HTA decisions are crucial, providing developers and researchers with the necessary clarity and fostering greater use of RWE.[4,5]

    The integration of RWE into HTA has the potential to reshape the landscape of rare disease innovation. By providing a more comprehensive understanding of treatment effectiveness, safety, and cost-effectiveness in real-world settings, RWE can address the limitations of traditional clinical trials and expedite the development and access to innovative therapies for rare diseases.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Dang A. Real-World Evidence: A Primer. Pharmaceut Med. 2023 Jan;37(1):25-36.
    2. Graili P, Guertin JR, Chan KKW, Tadrous M. Integration of real-world evidence from different data sources in health technology assessment. J Pharm Pharm Sci. 2023 Jul 17;26:11460.
    3. Hampson G, Towse A, Dreitlein WB, et al. Real-world evidence for coverage decisions: opportunities and challenges. Journal of comparative effectiveness research. 2018 Dec;7(12):1133-43.
    4. Field MJ, Boat TF. Development of new therapeutic drugs and biologics for rare diseases. InRare Diseases and Orphan Products: Accelerating Research and Development 2010. National Academies Press (US).
    5. Boat TF, Field MJ, editors. Rare diseases and orphan products: accelerating research and development. National Academies Press; 2011 Apr 3.
  • Single Arm Trials in HTA: Navigating Assessment Pathways

    Single Arm Trials in HTA: Navigating Assessment Pathways

    In the evolving landscape of drug development and regulatory approval, the traditional stronghold of Randomized Controlled Trials (RCTs) is being challenged. The surge in the focus on rare diseases and highly targeted patient populations has catalyzed a paradigm shift toward non-RCT designs, such as Single Arm Trials (SATs), switchover studies, and real-world studies. This shift is a response to the recognition by regulatory bodies, including the European Medicines Agency (EMA) and the United States Food and Drug Administration (FDA), of the need for faster access to treatments for underserved patient populations.[1]

    SATs have emerged as a key player in drug development, especially in scenarios where traditional RCTs face ethical or practical challenges. SATs, characterized by the absence of a separate control group offer a valuable avenue for assessing the safety and potential efficacy of novel interventions, particularly in early-phase clinical trials.[1,2]

    Traditionally, Health Technology Assessment (HTA) bodies favor evidence from RCTs to evaluate the benefits of a new product compared to standard care. However, in situations where direct comparisons through RCTs are not possible between two interventions of interest, indirect treatment comparison (ITC) using methodologies such as network meta-analyses (NMAs), matching-adjusted indirect comparisons (MAIC), and simulated treatment comparisons (STC) come into play. Incorporation of SATs into the indirect treatment comparisons becomes essential because of the valuable clinical evidence contained in SATs pertaining to the condition of interest, and several innovative methodologies have been conceptualized to achieve this, such as using historical controls and using simulations such as Bayesian hierarchical models.[1]

    Despite the advantages of SATs, they come with inherent limitations. The absence of internal control arms and randomization makes establishing causal interpretations challenging. To address these challenges, meticulous consideration of biases, including ascertainment and attrition biases, is paramount. External comparators play a pivotal role in mitigating these limitations. Strategies aimed at minimizing biases, such as careful pre-planning to avoid missing data through study design, are critical for ensuring the robustness of SAT submissions.[2]

    Finding appropriate comparator cohorts for SATs poses a significant challenge. The standards of patient care can evolve over time and vary between countries. Careful evaluation of past HTA decisions in the target indication(s) and early input from HTA stakeholders is critical. Before designing an external comparator study, a strategic approach is recommended. This involves establishing the need for an external comparator, identifying the factors that will drive success, determining the right external comparator group, collecting external comparator data, and ensuring the most appropriate methodology for comparison to study data. This strategic groundwork is essential to enhance the credibility and relevance of SAT submissions in the HTA landscape.[2, 3]

    A significant development in the understanding and regulation of SATs is the EMA reflection paper. This marks the first guidance document by an international medicine regulator specifically addressing key concepts and challenges associated with SATs submitted as pivotal evidence for marketing authorization applications. This initiative aims to enhance the design and conduct of SATs, providing a structured framework for stakeholders.[2-4]

    Looking ahead, the future direction of SAT-based submissions calls for a continued emphasis on refining evaluation methodologies, ensuring transparency, and adapting to the evolving nature of drug development. In conclusion, SATs present both challenges and opportunities, underscoring the imperative for a well-considered strategy in navigating this distinctive terrain.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Evans SR. Clinical trial structures. J Exp Stroke Transl Med. 2010 Feb 9;3(1):8-18. doi: 10.6030/1939-067x-3.1.8.
    2. Patel D, Grimson F, Mihaylova E, et al. Use of external comparators for health technology assessment submissions based on single-arm trials. Value in Health. 2021 Aug 1;24(8):1118-25.
    3. Dijkhuis S, Patel D, Foster S, et al. HTA77 The Use and Acceptability of External Comparator Studies to Support Hemato-Oncology Single-Arm Trial Submissions to Health Technology Assessment Bodies. Value in Health. 2022 Dec 1;25(12):S311.
    4. Single-arm trials as pivotal evidence for the authorisation of medicines in the EU | European Medicines Agency. Available from: https://www.ema.europa.eu/en/news/single-arm-trials-pivotal-evidence-authorisation-medicines-eu
  • Generating Regulatory-Grade Real-World Evidence: Tools and Frameworks

    Generating Regulatory-Grade Real-World Evidence: Tools and Frameworks
    Generating Regulatory-Grade Real-World Evidence Tools and Frameworks

    The integration of real-world evidence (RWE) into the regulatory landscape presents a pivotal shift in the evaluation and assessment of healthcare interventions. Acknowledging the multifaceted challenges in generating high-quality RWE, comprehensive tools, and frameworks have been developed to ensure the credibility, transparency, and reliability of real-world data (RWD). Understanding the significance of these tools and frameworks in generating regulatory-grade RWE is paramount to fostering evidence-based healthcare practices and informed decision-making.[1]

    RWE stands as a crucial asset in supplementing traditional data derived from randomized clinical trials (RCTs), offering insights into the long-term effectiveness, safety, and comparative outcomes of healthcare interventions in diverse patient populations and in real-world settings. RWE thus has a significant potential to bridge the gap between clinical research and practical healthcare applications, and thus, it becomes essential that the RWE be generated in a manner that has a high level of robustness and trustworthiness. The challenges associated with RWE generation include data quality and integrity, regulatory compliance and standards, ethical and privacy concerns, and limitations of traditional research methods. Strategies that have been developed to tackle these challenges include the enhancement of data validation, standardization, and quality assurance to foster the credibility and utility of RWE.[2]

    The deployment of advanced data collection from diverse RWD sources, including electronic health records (EHRs), patient registries, claims data, pharmacy data, and other sources with unstructured patient data, has significantly streamlined the process of RWE generation. Development of secure storage and transmission of RWD via data management systems and data warehouses have further enhanced the reliability and completeness of RWE for regulatory decision-making and policy formulations.[3]

    Next, the integration of advanced analytics and AI solutions has transformed the analysis and interpretation of complex RWE datasets. Embracing machine learning algorithms, natural language processing techniques, and predictive modeling tools has bolstered the efficiency and accuracy of RWE analysis, fostering evidence-based healthcare practices and informed decision-making.[3]

    Ensuring the security and integrity of patient data remains a paramount concern in RWE generation. The integration of blockchain technology, cryptographic protocols, and data encryption mechanisms has emerged as a pivotal strategy in safeguarding patient confidentiality and preventing data breaches. Upholding data transparency and security has bolstered the acceptance and credibility of RWE in healthcare interventions and regulatory decision-making.[4]

    Robust frameworks elucidated in guidelines and regulations published by global regulatory agencies of repute, such as the USFDA and the EMA, have been pivotal in steering the trajectory of RWE towards greater transparency, reliability, and standardization. The stringent regulations set up by these regulatory agencies for RWD collection and RWE generation have set a precedent for the meticulous documentation and dissemination of RWD, thereby enhancing the credibility and validity of RWE-based regulatory decisions. These frameworks serve as crucial cornerstones, ensuring that the generation of RWE aligns seamlessly with established regulatory protocols. This has instilled additional trust in the reliability and relevance of RWE among all stakeholders in the healthcare and pharmaceutical sectors.[3-5]

    The adoption of international standards for data generation has contributed to the harmonization of RWE practices across diverse geographical regions. By adhering to globally recognized standards, the process of RWE documentation and dissemination has transcended geographical boundaries, fostering a cohesive and collaborative approach to evidence-based decision-making. This emphasis on international standards has not only facilitated a more holistic understanding of global healthcare trends and outcomes but has also encouraged the exchange of best practices and methodologies, propelling the evolution of RWE on a global scale.[4,5]

    There have been other significant steps taken to enhance the generation of regulatory-grade RWE. For instance, the implementation of best practices for data validation and quality assurance has reinforced the credibility and trustworthiness of RWE-based insights.[4,5] Furthermore, the establishment of interoperability and data-sharing frameworks has facilitated seamless data exchange and collaboration among various stakeholders within the healthcare ecosystem.[1,4,5]

    The development and validation of frameworks exclusively for RWE generation have paved the way for enhancing standardization in methodologies for generating regulatory-grade RWE. Some of these frameworks include the Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real‐world Evidence (SPACE), the Structured Process to Identify Fit-For-Purpose Data (SPIFD), the Structured Template and Reporting Tool for Real World Evidence (STaRT-RWE), and the HARmonized Protocol Template to Enhance Reproducibility (HARPER). These frameworks ensure standardized approaches to data assessment, documentation, and study protocols, thereby enhancing the validity, transparency, and reproducibility of RWE studies and fostering their acceptance and integration into contemporary healthcare practices and regulatory decision-making processes.[4-7]

    The systematic implementation of these tools and frameworks underscores the transformative potential of RWE in addressing evidentiary gaps and enhancing healthcare decision-making. Acknowledging the significance of reliable and credible RWE in regulatory decisions has laid the groundwork for fostering evidence-based healthcare practices and informed policy formulations. As the healthcare landscape continues to evolve, the seamless integration of RWE tools and frameworks will play a pivotal role in shaping the future of evidence-based healthcare interventions and regulatory decision-making.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Berger M, Daniel G, Frank K, et al. A framework for regulatory use of real-world evidence. White paper prepared by the Duke Margolis Center for Health Policy. 2017 Sep 13;6.
    2. Liu F, Panagiotakos D. Real-world data: a brief review of the methods, applications, challenges and opportunities. BMC Medical Research Methodology. 2022 Nov 5;22(1):287.
    3. Miksad RA, Abernethy AP. Harnessing the power of real‐world evidence (RWE): a checklist to ensure regulatory‐grade data quality. Clinical Pharmacology & Therapeutics. 2018 Feb;103(2):202-5.
    4. Burns L, Le Roux N, Kalesnik-Orszulak R, et al Real-world evidence for regulatory decision-making: guidance from around the world. Clinical Therapeutics. 2022 Mar 1;44(3):420-37.
    5. Xia AD, Schaefer CP, Szende A, et al. RWE Framework: An Interactive Visual Tool to Support a Real-World Evidence Study Design. Drugs Real World Outcomes. 2019 Dec;6(4):193-203.
    6. Jaksa A, Wu J, Jónsson P, et al. Organized structure of real-world evidence best practices: moving from fragmented recommendations to comprehensive guidance. Journal of comparative effectiveness research. 2021 Jun;10(9):711-31.
    7. Gatto NM, Campbell UB, Rubinstein E, et al. The structured process to identify fit‐for‐purpose data: a data feasibility assessment framework. Clinical Pharmacology & Therapeutics. 2022 Jan;111(1):122-34.