Contact Us

Home

About us

Services

AI Excellence

case studies

publications

HEOR Certificationheor Certification

AI for Medical Writing

Contact

  • Using Individual Participant Data for Network Meta-Analysis: Advantages and Challenges

    Using Individual Participant Data for Network Meta-Analysis: Advantages and Challenges

    The domain of healthcare research has witnessed remarkable advancements in the evaluation of various treatment modalities for specific diseases. One such pioneering approach, the Network Meta-Analysis (NMA), has proven instrumental in comparing multiple treatments assessed in randomized trials. Conventionally, NMAs primarily rely on Aggregate Data (AD) extracted from study publications, limiting the scope of analysis to the available summarized data. However, an innovative alternative that has gained traction in recent years is the integration of Individual Participant Data (IPD) into the NMA framework. This approach involves procuring raw information recorded for each participant in a study, allowing for a more comprehensive and nuanced evaluation of treatment effects.[1,2]

    A pivotal strength of incorporating IPD into NMA lies in the enhanced quality and scope of information available for each trial. Unlike AD, IPD circumvents the limitations posed by the published data, thereby facilitating a comprehensive analysis. This comprehensive approach fosters the standardization of outcome and covariate definitions across trials, reducing the potential for reporting biases. By harmonizing diverse definitions of outcomes, IPD enables a more robust analysis of continuous scales, yielding more precise and reliable conclusions. Additionally, the collaborative nature of IPD NMA projects, involving direct engagement with trial investigators, facilitates a more thorough identification of relevant trials, thereby reinforcing the reliability of risk assessments and data analyses.[2-4]

    Another compelling advantage of employing IPD for NMAs is the ability to examine and plot distributions of covariates across trials. This granular approach empowers researchers to discern systematic differences in variables that might influence measures of relative treatment effect. With access to a broader set of recorded covariates, IPD enables a more detailed comparison of how covariate distributions vary across trials, thus aiding in the assessment of their potential impact on network heterogeneity and inconsistency. Notably, this approach contributes significantly to the evaluation of the consistency assumption, a critical factor in ensuring the reliability of NMA results.[4]

    Furthermore, IPD allows for the standardization and improvement of the analysis of each trial, enabling researchers to define their own estimands of interest. By standardizing effect measures across trials, IPD facilitates the utilization of advanced analytical approaches, particularly in the modeling of time-to-event data and the handling of missing data. The utilization of a multivariate NMA approach becomes feasible, enabling the simultaneous comparison of treatments across multiple outcomes, while accounting for correlations among these outcomes. This multifaceted analysis not only enhances the precision of inferences but can also alter the ranking of treatments, thereby providing a more comprehensive perspective for clinical decision-making.[4]

    Using IPD in NMA can also allow adjusting for prognostic factors in the analysis of each trial. This aspect is instrumental in improving the power to detect treatment effects, especially in scenarios where stratified randomization has been employed. Moreover, the inclusion of treatment-covariate interactions facilitated by IPD, allows for a more comprehensive understanding of how treatment effects may vary across different patient populations. This nuanced analysis paves the way for tailored treatment recommendations, aligning with the evolving paradigm of personalized medicine and patient-centric care.[4]

    However, the integration of IPD into NMA projects is not without its share of challenges. The substantial time and effort required to procure, validate, harmonize, and synthesize IPD often pose significant logistical hurdles. Negotiating data-sharing agreements and maintaining effective collaboration with trial investigators demand meticulous planning and clear communication to ensure the smooth execution of the research. Additionally, the inclusion of data from diverse sources may introduce data heterogeneity. Performance of IPD NMA would necessitate expertise to handle both the IPD as well as perform NMA: thus, IPD NMA requires higher and complicated statistical analytic techniques, adding to the resource-intensiveness of the technique. Mitigating concerns regarding availability bias, stemming from the unavailability of IPD from certain studies, remains a persistent challenge that requires careful consideration and methodological adjustments. Furthermore, data privacy, confidentiality, and ethical considerations arise due to the transmission of sensitive information contained in the IPD, adding another layer of complexity.[4]

    To address the challenges discussed above, various innovative methodologies have emerged, such as multilevel network meta-regression, which extends the IPD NMA framework to incorporate both IPD and AD, ensuring robust and accurate estimates across diverse target populations. Additionally, matching-adjusted indirect comparison, simulated treatment comparison, and predictive-adjusted indirect comparison have been proposed as alternative approaches to account for limited IPD, although their applicability remains confined to specific study designs and populations.[4-7]

    In conclusion, the integration of IPD into NMA represents a paradigm shift in evidence synthesis, offering unprecedented insights into treatment effects and patient outcomes. Despite the challenges posed by the complex nature of IPD meta-analyses, the benefits of utilizing this approach far outweigh the obstacles, fostering a more holistic and informed approach to clinical decision-making and healthcare policy formulation. As the healthcare landscape continues to evolve, the judicious utilization of IPD in NMA is poised to redefine the trajectory of evidence-based medicine, leading to more effective and tailored healthcare interventions for diverse patient populations.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Dias S, Ades AE, Welton NJ, et al. Network meta-analysis for decision-making. John Wiley & Sons; 2018 Jan 8.
    2. Riley RD, Tierney JF, Stewart LA, editors. Individual participant data meta-analysis: a handbook for healthcare research. John Wiley & Sons; 2021 May 24.
    3. Riley RD, Phillippo DM, Dias S. Network Meta‐Analysis Using IPD. Individual Participant Data Meta‐Analysis: A Handbook for Healthcare Research. 2021 Apr 22:347-73.
    4. Riley RD, Dias S, Donegan S, et al. Using individual participant data to improve network meta-analysis projects. BMJ evidence-based medicine. 2023 Jun 1;28(3):197-203.
    5. Chaimani A. Conduct and reporting of individual participant data network meta-analyses need improvement. BMC medicine. 2020 Jun 2;18(1):156.
    6. Phillippo DM, Dias S, Ades AE, et al. Multilevel network meta-regression for population-adjusted treatment comparisons. Journal of the Royal Statistical Society Series A: Statistics in Society. 2020 Jun;183(3):1189-210.
    7. Phillippo D, Ades T, Dias S, et al. NICE DSU technical support document 18: methods for population-adjusted indirect comparisons in submissions to NICE.

    MarksMan Healthcare

    , , ,

  • Challenges of Single-Arm Trials in Network Meta-Analyses (NMAs)

    Challenges of Single-Arm Trials in Network Meta-Analyses (NMAs)
    Challenges of Single-Arm Trials in NMA

    Network Meta-Analysis (NMA) is a sophisticated statistical technique that allows researchers to simultaneously compare multiple treatments using a combination of direct and indirect evidence. While NMAs have revolutionized evidence synthesis, the inclusion of single-arm trials (SATs) within these analyses has sparked discussions and raised important questions about the potential challenges they bring.[1]

    NMAs typically rely on a wealth of data, including randomized controlled trials (RCTs) and observational studies. However, there are instances where patient-level data from published studies might be inadequate, leading researchers to explore alternative avenues for incorporating evidence. This is where SATs come into play, offering a unique approach to addressing data gaps in NMA. SATs can function as mutual control groups based on the presence of complete covariate profiles and differing interventions. These trials primarily provide insights into the safety and efficacy of a treatment within a specific population. They often play a crucial role in early-phase drug development, helping researchers understand the potential benefits and risks of a novel intervention.[2-4]

    The absence of a comparator group, which is the hallmark of SATs, is perhaps the greatest challenge of using these studies in NMAs because this precludes the construction of a connected network of treatments. NMAs depend on direct comparisons between treatments to establish relationships and estimate treatment effects accurately. The absence of head-to-head comparisons from SATs disrupts the seamless flow of evidence within the network. Moreover, the inclusion of SATs can potentially introduce bias and confounding. Without a control group, the observed outcomes in these trials might be influenced by factors beyond the intervention itself, leading to skewed estimates of treatment effects. This bias can jeopardize the validity and reliability of the overall NMA results.[5]

    SATs can also exacerbate heterogeneity, which is an inherent challenge in evidence synthesis through NMAs. The diversity in study design, patient populations, and outcomes measured across these trials can contribute to significant heterogeneity within the NMA, undermining the assumption of treatment effect consistency that NMAs rely upon. The precision and confidence in NMA results can also be compromised when single-arm trials with limited sample sizes are included. Since these trials contribute a smaller amount of data relative to comparative trials, their impact on the overall estimates can be disproportionate. This can lead to wider confidence intervals and reduced statistical power, which in turn affects the reliability of the conclusions drawn from the NMA.[5]

    To address these challenges, several strategies can be employed when considering the inclusion of SATs in NMAs. The incorporation of external evidence, such as historical controls, real-world data, or findings from related studies, can enrich the analysis by providing additional context and comparators, thereby enhancing the understanding of treatment efficacy and safety. Secondly, indirect comparisons can be leveraged, wherein treatments are linked through a common comparator, enabling the assessment of relative effects even in the absence of direct trial comparisons. Furthermore, the inclusion of matching controls entails identifying studies with analogous participant characteristics or baseline factors to the SAT, effectively generating a pseudo-comparator group that aligns with the trial’s attributes, thus mitigating potential bias. Moreover, the development of synthetic control groups by amalgamating data from multiple SATs that share comparable patient profiles and outcomes can be facilitated through statistical methods like propensity score matching or Bayesian approaches, enhancing the robustness of the analysis.[6-8]

    Thus, SATs can offer valuable information for conducting NMAs, but their inclusion in the NMA is associated with some genuine challenges. By implementing strategies such as sensitivity analyses, cautious interpretation, and judicious inclusion, researchers can navigate these challenges and make well-informed decisions about including SATs in NMAs.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Annals of internal medicine. 2013 Jul 16;159(2):130-7.
    2. Evans SR. Clinical trial structures. J Exp Stroke Transl Med. 2010 Feb 9;3(1):8-18.
    3. Wang Z, Lin L, Murray T, Hodges JS, Chu H. Bridging randomized controlled trials and single-arm trials using commensurate priors in arm-based network meta-analysis. Ann Appl Stat. 2021 Dec;15(4):1767-1787.
    4. Schmitz S, Maguire Á, Morris J, et al. The use of single armed observational data to closing the gap in otherwise disconnected evidence networks: a network meta-analysis in multiple myeloma. BMC Med Res Methodol. 2018 Jun 28;18(1):66.
    5. Sachdev M. Using Single Arm Trials for HTA Submissions: Dr. Heeg and Maria Rizzo.
    6. Seeger JD, Davis KJ, Iannacone MR, et al. Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials. Pharmacoepidemiol Drug Saf. 2020 Nov;29(11):1382-1392.
    7. Burcu M, Dreyer NA, Franklin JM, et al. Real-world evidence to support regulatory decision-making for medicines: Considerations for external control arms. Pharmacoepidemiol Drug Saf. 2020 Oct;29(10):1228-1235.
    8. Zhang J, Ko CW, Nie L, Chen Y, Tiwari R. Bayesian hierarchical methods for meta-analysis combining randomized-controlled and single-arm studies. Statistical methods in medical research. 2019 May;28(5):1293-310.

    MarksMan Healthcare

    , , ,

  • An Overview of Methods for Indirect Treatment Comparisons in Healthcare Decision-making

    An Overview of Methods for Indirect Treatment Comparisons in Healthcare Decision-making

    Meta-analyses summarize data from head-to-head trials to evaluate pairs of treatments that have been directly compared.[1] However, in certain circumstances, multiple therapies are of interest, and no data is available on their direct comparison. In such cases, indirect treatment comparison (ITC) is performed for synthesizing evidence surrounding treatments of interest.[2] ITC assumes that the studies are similar and homogenous regarding the administered therapies, patient characteristics, and observed effects, and works best when the inconsistency between indirect and direct evidence is minimal or absent. 

    Various methods of ITC have been developed depending on the availability of individual patient data (IPD) and summary level data (SLD). Some of these methods include naïve ITC, network meta-analysis (NMA), population-adjusted indirect comparisons (PAIC), simulated treatment comparisons (STCs), and matching-adjusted indirect comparisons (MAICs). The choice of these methods depends on the study design, the number of comparators available, and the degree to which the outcomes are measured. In addition, the extent of assumptions employed, methodological limitations, and inherent biases associated with each method also determine the choice of the ITC method.

    Naïve ITC is based on SLD and is used when the treatments cannot be connected by a common comparator. It does not account for heterogeneity and excludes information from the placebo arms when comparing treatments, thereby introducing bias. Hence, this method is mainly avoided to preserve the randomization in trials during the analyses. 

    Network Meta-Analysis (NMA) is perhaps the most popular of the ITC methods. It works with SLD, and compares treatments by combining indirect and direct evidence connected by a network of studies.[3] NMA is considered as the gold standard for ITCs. It offers more an exact estimate of the relative effects of treatments in the network than a single direct or indirect estimate. It also enables for the assessment of intervention ranking and hierarchy. To some extent, this bias can be reduced by using meta-regression, that addresses heterogeneity in treatment effects. It can assess how the effect of treatment changes with a covariate (a patient or methodological attribute). Unfortunately, the usage of meta-regression with NMAs becomes questionable in cases where the number of studies in a network is limited. Furthermore, this approach can only be used when there is a variance study or comparison with only minor variations in impact modifiers.[4,5] Moreover, covariate correction in aggregate-level data may result in ecological bias, which limits the interpretation of estimated results for subgroups. In such cases, Individual Patient Data (IPD) provides adjustments for covariates that cause inconsistencies (e.g., prognostic factors, effect modifiers, etc.). Hence, NMA that leverages IPD can be put to use for conducting analyses that can provide adjustments to reduce such inconsistencies.[6] 

    The application of NMAs and their associated methods are often limited by insufficient evidence networks and heterogeneity across trials. This is resolved to certain extent through population-adjusted indirect comparison (PAIC), which is a targeted approach to enhance ITC.[7] It allows to overcome the challenges faced by NMAs by carrying out a targeted comparison between outcomes for specific treatments and factors. It includes two methods: simulated treatment comparisons (STCs) and matching-adjusted indirect comparisons (MAICs). These methods can help reduce the ambiguity in the comparisons with statistical adjustment. STCs do this by applying predictive equations, whereas MAIC relies on patient reweighting. 

    STCs or MAICs can be used to conduct either “anchored” indirect comparison, in which each trial has a common comparator arm, or “unanchored” indirect comparison, in which the treatment network is disconnected (single-arm investigations). An anchored approach relies on “conditional constancy of relative effects”. In contrast, an unanchored approach works on a stringent assumption of “conditional constancy of absolute effects”. which is more demanding than the former and is not a widely accepted approach [8]  STCs are often appropriate in analyses where numerous comparators are available for a small set of outcomes, whereas MAICs are often suitable in cases with only one comparator but multiple outcomes. The precision of the equations in STC and the effective matching of populations in MAIC determine the dependability of the studies.

    A task force report released in 2011 by the Professional Society for Health Economics and Outcomes Research (ISPOR) defines the fundamentals of conducting ITCs and assessing these studies for informed and efficient decision-making.[4,5] Though the methodological aspects of NMAs have received much attention from researchers, the other ITC methods are yet to be refined to a similar extent. The standardization of these methods is vital to increase their reliability and application.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    [1] Ahn E, Kang H. Introduction to systematic review and meta-analysis. Korean J Anesthesiol. 2018 Apr;71(2):103-112. doi: 10.4097/kjae.2018.71.2.103.  [2] Veroniki AA, Straus SE, Soobiah C, et al. A scoping review of indirect comparison methods and applications using individual patient data. BMC Med Res Methodol. 2016 Apr 27;16:47. doi: 10.1186/s12874-016-0146-y.  [3] Tonin FS, Rotta I, Mendes AM, Pontarolo R. Network meta-analysis: a technique to gather evidence from direct and indirect comparisons. Pharm Pract (Granada). 2017 Jan-Mar;15(1):943. doi: 10.18549/PharmPract.2017.01.943 [4] Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011 Jun;14(4):417-28. doi: 10.1016/j.jval.2011.04.002.  [5] Hoaglin DC, Hawkins N, Jansen JP, et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health. 2011 Jun;14(4):429-37. doi: 10.1016/j.jval.2011.01.011.  [6] Riley RD, Dias S, Donegan S, et al. Using individual participant data to improve network meta-analysis projects. BMJ Evid Based Med. 2022 Aug 10:bmjebm-2022-111931. doi: 10.1136/bmjebm-2022-111931. Epub ahead of print.  [7] Phillippo DM, Dias S, Elsada A, et al. Population Adjustment Methods for Indirect Comparisons: A Review of National Institute for Health and Care Excellence Technology Appraisals. Int J Technol Assess Health Care. 2019 Jan;35(3):221-228. doi: 10.1017/S0266462319000333. Epub 2019 Jun 13.  [8] Jiang Y, Ni W. Performance of unanchored matching-adjusted indirect comparison (MAIC) for the evidence synthesis of single-arm trials with time-to-event outcomes. BMC Med Res Methodol. 2020 Sep 29;20(1):241. doi: 10.1186/s12874-020-01124-6.

    MarksMan Healthcare

    , , , ,

  • What Are The Pros & Cons of Network Meta-Analysis (NMA)?

    What Are The Pros & Cons of Network Meta-Analysis (NMA)?

    Evidence-based medicine (EBM) is gaining wide acceptance from researchers globally as it thoroughly optimizes the latest available evidence to make informed care decisions. This involves evaluating the quality of the clinical data by critically assessing methodologies reported in publications. Moreover, EBM incorporates both clinical expertise as well as patient values. Meta-analyses of RCTs often make it among the top of the evidence hierarchy, since it’s regarded as the most valid clinical proof. Indeed, meta-analysis is a validated method to analyse and summarize knowledge by increasing the number of patients, and thus also the effective statistical power. However, there are several limitations associated with meta-analysis, which considers only pairwise comparisons. Unfortunately, head-to-head comparisons are not always available in the literature or they fail to answer a specific clinical question. This can be overcome with the help of network meta-analysis (NMA), which helps providing a global estimate of efficacy or safety of numerous experimental treatments that have not before been directly compared with adequate precision, or at all. Network meta-analysis integrates both direct and indirect effects from the entire set of evidence. Additionally, it ranks the treatments as the best or worst on the basis of valid statistical inference methods. (1)

    Network meta-analysis is preferable over conventional pair-wise meta-analysis, since it uses indirect evidence to justify comparisons amongst all treatments, thus enabling estimation of comparative effects that have not been investigated as precisely in RCTs. Thus, NMA is increasingly getting popular with clinicians, guideline developers, and HTA agencies as the ever growing new evidence needs to be placed in the context of all available evidence for appraisals. (2)

    Furthermore, NMA is increasingly becoming essential to formulating recommendations on reimbursements as well as clinical guidelines by healthcare agencies around the world. It has recently been adopted by Cochrane, as 10% (23/230) of their systematic reviews since the year 2015 have used NMA. In 2015, GRADE working groups published guidance on using GRADE in conjunction with NMA. Furthermore, the National Institute for Health and Clinical Excellence (NICE, UK) also approves the application of NMA within its clinical guidelines manual. (3)

    In addition, the ability of NMA to quantitatively assess interventions that have not been directly compared in studies aids the process of developing guidelines. This is because, in the absence of head-to-head evidence, guideline development groups will bank more strongly on expert opinion. Also, collective analysis of both direct and indirect evidence strengthens the evidence base. Moreover, while NMA determines the comparative effectiveness of drugs, the approach can be applied more broadly. For instance, the latest revision of the WHO HIV guidelines represents how the use of NMA to evaluate interventions in improving adherence to ART significantly contributed in the formulation of recommendations. (4)

    Although NMA is a powerful tool for comparative effectiveness research, it is more complex than pair-wise meta-analysis. Also, the assumption of transitivity is stringent, which is essential to consider throughout the entire process of NMA. Supplementary analyses like network meta-regression are often necessary, which further increase the complexity of the analysis. What’s more, NMA is very resource-demanding. As NMAs generally cater to broader questions, they usually involve more studies at each step of the systematic review, right from screening to analysis, than conventional meta-analysis. Therefore, it is crucial to schedule the time and resource commitment before actually conducting an NMA. (5)

    Network meta-analysis allows for indirect comparisons, incorporating more data in the analysis, thus tackling the bigger picture; while a single pairwise meta-analysis offers a fragmented picture. However, NMA should be done carefully. There needs to be a clinical question developed with input from both a subject area clinical expert and a statistician. Assessments of transitivity and consistency are fundamental for ensuring the validity of NMA. Finally, as mentioned earlier, the time and resource commitments required to produce a high-quality NMA should be appropriately taken into consideration. (5)

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Greco T, et al. The attractiveness of network meta-analysis: a comprehensive systematic and narrative review. Heart, Lung and Vessels 2015; 7(2):133-142.
    2. Ward P. Network Meta-Analysis Can Play Decisive Role. June, 2013.
    3. Kanters S, et al. Use of network meta-analysis in clinical guidelines. Bulletin of the World Health Organization 2016; 94:782-784. 
    4. Consolidated guidelines on HIV prevention, diagnosis, treatment and care for key populations. 2016 update. Geneva: World Health Organization; 2016. 
    5. Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Intern Emerg Med 2016; 12(1):103-111.

    Written by – Ms. Tanvi Laghate

    MarksMan Healthcare

    , , ,