Contact Us

Home

About us

Services

AI Excellence

case studies

publications

HEOR Certificationheor Certification

AI for Medical Writing

Contact

  • Navigating Complexity in Meta-Analysis: How the DECiMAL Guide Makes a Difference

    Navigating Complexity in Meta-Analysis: How the DECiMAL Guide Makes a Difference

    Meta-analysis is a cornerstone of evidence-based research, offering a systematic approach to combine and synthesize data from multiple studies. However, as research questions become more nuanced and datasets more diverse, the complexity of meta-analyses increases significantly. This is where the Data Extraction for Complex Meta-Analysis (DECiMAL) guide comes into play, providing a structured framework to navigate these complexities.[1]

    Extracting data for meta-analysis can be a complex task, especially when dealing with diverse study designs, outcomes, and data formats. Traditional data extraction methods may not be sufficient to handle the complexities of modern research, leading to potential biases and inconsistencies in the analysis. The DECiMAL guide addresses these challenges by offering a detailed methodology for data extraction, ensuring all relevant information is captured and analyzed consistently. This promotes standardization, reduces bias, and enhances transparency in the meta-analysis process, ultimately leading to more reliable and informative results. DECiMAL covers a wide range of data types, including continuous, binary, and time-to-event outcomes, as well as more complex data structures, such as multiple treatment arms and correlated outcomes. By addressing these complexities, DECiMAL helps researchers conduct rigorous and reproducible meta-analyses.[1-3]

    The DECiMAL guide comprises several core components designed to tackle the complexities of meta-analyses. First, it stresses the importance of a clearly defined research question, utilizing the Population, Intervention, Comparison, and Outcome (PICO) criteria to guide the data extraction process. A comprehensive literature search is essential, and DECiMAL advocates for a systematic approach across various databases to capture all relevant studies while minimizing publication bias, with meticulous documentation of the search strategy. The guide’s detailed data extraction template captures a broad range of data points, ensuring consistency and completeness. Addressing heterogeneity is another key aspect, with DECiMAL offering guidance on statistical methods like subgroup analyses and meta-regression to understand variability between studies. For data synthesis and analysis, DECiMAL provides best practices, including the use of fixed-effect and random-effects models and a multivariate approach for diagnostic accuracy studies. It also emphasizes the assessment of bias using standardized tools and advocates for transparent reporting according to guidelines, such as PRISMA, which supports replication and enhances research credibility.[4]

    The DECiMAL guide provides a detailed methodology for extracting various types of data for meta-analysis. For time-to-event data, such as cancer recurrence, hazard ratios and their uncertainties should be collected, and it should be noted if Kaplan-Meier plots or life tables are reported. For rate data, like migraine episode frequency, the total number of person-years at risk should be collected. If this information is not available, the average length of follow-up and the total number of patients at study end can be used to approximate person-years. Binary and categorical variables should use numerical coding for responses and additional coding for other responses, while both numbers of patients randomized and those completing the trial should be extracted. Continuous and ordinal variables should be consistently reported in chosen units, with both final values and changes from the baseline being combined if baselines are equal. The guide ensures comprehensive data collection and helps identify and address potential issues early on, enhancing the consistency and accuracy of complex meta-analyses.[4]

    While DECiMAL provides a comprehensive framework for data extraction, it has certain limitations. For instance, it does not delve into specific statistical techniques for handling missing data or converting summary statistics. Additionally, while DECiMAL is primarily designed for aggregate data meta-analyses, it may not be directly applicable to individual patient data meta-analyses. The guide primarily addresses considerations related to data extraction for subsequent meta-analyses but provides limited information on the practical and technical aspects of data extraction itself. Furthermore, DECiMAL is designed specifically for data extraction in aggregate data meta-analyses, and its methods do not apply to individual patient data meta-analyses.[4]

    The DECiMAL guide marks significant progress in meta-analysis, especially for handling complex datasets. By standardizing data extraction, addressing heterogeneity, and enhancing transparency, DECiMAL ensures that meta-analytical results are robust and reliable. With the increasing volume and complexity of research data, adopting comprehensive tools like DECiMAL will be essential for preserving the integrity and effectiveness of meta-analyses. For researchers undertaking complex meta-analyses, DECiMAL provides a structured approach to navigating the challenges of data extraction and analysis. Following its guidelines can improve the quality and impact of research findings, offering valuable contributions to the scientific community.

    Become A Certified HEOR Professional – Enrol yourself here!

    References:

    1. Brown SA, Upchurch SL, Acton GJ. A framework for developing a coding scheme for meta-analysis. West J of Nurs Res. 2003;25:205–22.
    2. Centre for Evidence-Based Medicine. Data Extraction Tips: Meta-Analysis [Internet]. Oxford: University of Oxford. 2023; Available from: https://www.cebm.ox.ac.uk/resources/data-extraction-tips-meta-analysis.
    3. Effective Practice and Organisation of Care (EPOC). Data collection form. EPOC Resources for review authors. Norwegian Knowledge Centre for the Health Services. 2013;Available from: http://epoc.cochrane.org/epoc-specific-resources-review-authors.
    4. Pedder H, Sarri G, Keeney E, Nunes V, Dias S. Data extraction for complex meta-analysis (DECiMAL) guide. Syst Rev. 2016 Dec 13;5(1):212. 5. Afifi M, Stryhn H, Sanchez J. Data extraction and comparison for complex systematic reviews: a step-by-step guideline and an implementation example using open-source software. Syst Rev. 2023 Dec 1;12(1):226.

    MarksMan Healthcare

    , , ,

  • Matching-Adjusted Indirect Comparisons (MAICs): What, Why, and How?

    Matching-Adjusted Indirect Comparisons (MAICs): What, Why, and How?

    Meta-analysis is crucial in evidence-based medicine as it combines data from multiple studies for more precise treatment effect estimates. However, when head-to-head clinical trials directly comparing treatments are scarce, Indirect Treatment Comparisons (ITC) become valuable by offering insights through common comparators. The conventional approaches to ITCs hinge on aggregate data, assuming a uniform distribution of effect-modifying variables across trials. The emergence of the Matching-Adjusted Indirect Comparison (MAIC) methodology, which challenges these assumptions, is gaining momentum, particularly in submissions to reimbursement organizations.[1-3]

    MAICs are an extension of the traditional ITC method, developed with the aim of addressing some of the limitations of traditional ITCs, particularly the issue of confounding by patient characteristics. MAICs attempt to make the compared treatment groups more comparable by adjusting for patient-level characteristics that may influence treatment outcomes. MAICs offer a unique vantage point within Health Technology Assessment (HTA) submissions, amalgamating unadjusted ITC outcomes, even when relative treatment efficacy appears modest. This method aims to minimize bias, facilitating a fair and nuanced comparison of therapies akin to real-world scenarios.[4-6]

    MAICs are grounded in individual-level patient data (IPD) from an intervention trial (e.g., manufacturer’s product) and published aggregate data from the comparator’s trial, and seek equilibrium by reweighting IPD patient characteristics. Techniques such as propensity scores derived from moment methods or entropy balancing play a pivotal role in this equilibrium, ensuring the reweighted IPD outcomes are juxtaposed against published aggregate data to discern relative impact.[7]

    MAICs predominantly operate within an “anchored” framework, often relying on a shared comparator (e.g., placebo) to ground comparisons. This approach, common in connected networks that account for randomization, shields estimations from the sway of imbalanced prognostic factors. Nonetheless, empirical evidence or clinical insight must substantiate effect modification claims. Conversely, the “unanchored” MAIC takes center stage in disconnected networks lacking a common comparator, directly juxtaposing reweighted IPD outcomes and published aggregate data. Rigorous estimates of absolute effects and vigilant control of prognostic and effect-modifying factors are prerequisites for unanchored comparisons, while lurking unobserved confounding remains challenging due to a lack of randomization. Fundamentally, anchored MAICs illuminate treatment impact, whereas unanchored variants scrutinize outcomes across trials.[6,7]

    MAICs often have a lower risk of confounding because of the matching of patients based on key characteristics; for the same reason, potential bias from differences between the treatment groups in the original trials is also lower with MAICs. Further, since MAIC creates a more balanced comparison by aligning patient characteristics, treatment estimates are often more robust and reliable than conventional ITCs. However, MAICs also have certain limitations pertaining to the availability of suitable IPD, the potential of selection bias of patient data, quality and completeness of IPD, and challenges related to assumptions and extrapolations. While MAIC employs individual-level patient data (IPD) to mitigate observed differences, unobserved disparities can lead to residual confounding. Even when placebo-arm outcomes are balanced, unobserved factors affecting treatment outcomes but not placebo outcomes can bias comparisons. Practical challenges include the need for matched outcome definitions and inclusion/exclusion criteria and the inability to fit or calibrate propensity score models using aggregate data. Balancing multiple baseline factors relies on an adequate number of patients with IPD, which can reduce the adequate sample size. MAIC may be utilized for single-arm trials, but the absence of a common comparator limits validation. Irreconcilable differences in trial design or patient characteristics might exclude trials from analysis, necessitating a trade-off between evidence inclusion and reducing heterogeneity. Sensitivity analyses are crucial to assessing the impact of trial inclusion/exclusion on results.[8,9]

    In a landscape where clinical decision-making hinges on robust evidence, MAIC is a valuable tool, offering unique perspectives and cautionary lessons. As researchers, practitioners, and evaluators continue to explore the horizons of evidence synthesis, the pursuit of accuracy, transparency, and informed choices remains paramount. By embracing the insights and addressing the limitations of MAIC, we inch closer to a comprehensive understanding of treatment landscapes and forge a path toward more informed and patient-centered healthcare decisions.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Ahn E, Kang H. Introduction to systematic review and meta-analysis. Korean J Anesthesiol. 2018 Apr;71(2):103-112.
    2. Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011 Jun;14(4):417-28.
    3. Veroniki AA, Straus SE, Soobiah C, et al. A scoping review of indirect comparison methods and applications using individual patient data. BMC Med Res Methodol. 2016 Apr 27;16:47.
    4. Phillippo DM, Ades AE, Dias S, et al. Methods for Population-Adjusted Indirect Comparisons in Health Technology Appraisal. Med Decis Making. 2018 Feb;38(2):200-211.
    5. Phillippo DM, Dias S, Elsada A, et al. Population Adjustment Methods for Indirect Comparisons: A Review of National Institute for Health and Care Excellence Technology Appraisals. Int J Technol Assess Health Care. 2019 Jan;35(3):221-228.
    6. Thom H, Jugl SM, Palaka E, Jawla S. Matching adjusted indirect comparisons to assess comparative effectiveness of therapies: usage in scientific literature and health technology appraisals. Value in Health. 2016 May 1;19(3):A100-1.
    7. Petto H, Kadziola Z, Brnabic A, et al. Alternative Weighting Approaches for Anchored Matching-Adjusted Indirect Comparisons via a Common Comparator. Value Health. 2019 Jan;22(1):85-91.
    8. Signorovitch JE, Sikirica V, Erder MH, et al. Matching-adjusted indirect comparisons: a new tool for timely comparative effectiveness research. Value in Health. 2012 Sep 1;15(6):940-7.
    9. Jiang Y, Ni W. Performance of unanchored matching-adjusted indirect comparison (MAIC) for the evidence synthesis of single-arm trials with time-to-event outcomes. BMC Med Res Methodol. 2020 Sep 29;20(1):241.

    MarksMan Healthcare

    , , , ,

  • An Overview of Evidence Maps: a New Approach for Evidence Review Process

    An Overview of Evidence Maps: a New Approach for Evidence Review Process

    Systematic reviews (SRs) are the embodiment of the evidence-based approaches, and they have reformed clinical decision-making in almost all therapy areas. The approach of SRs was essentially developed to fulfil the need of the medical practitioners to obtain precise and consistent information about the efficacy and safety of a clinical intervention, diagnostic procedure, or a prognostic marker from a pool of evidence, which is apparently full of contradiction, heterogeneity and bias.(1) Although SRs and meta-analyses are robust and detail-oriented, they’re both resource intense, with a limited scope of outcomes.(2) In order to cater to a range of needs from the stakeholders, the SR approach has branched within the realm of evidence synthesis. For instance, rapid reviews provide for more urgent deadlines but may not follow all the methods of an SR,(3) scoping reviews include larger bodies of evidence, not requiring a detailed synthesis,(4) and realist reviews focus on the assessment of the functions of complex interventions, often comprising of evidence excluded from classic SRs.(5, 6)

    Given the resource intense nature of the SRs, it is essential to recognize the most informative research questions in order to maximize their value and efficiency in clinical and regulatory decision-making. It can be inefficient to invest resources in SRs barely as a means of addressing specific research questions, if data available to answer those questions is lacking. Therefore, decision-makers need to monitor and understand the evidence base as a whole, so as to quickly determine the emerging trends or issues of potential concern. This can, in turn, facilitate the development of proactive research questions by relevant stakeholders for SRs to answer.(1)

    Evidence mapping is a new approach for the evidence review process. This approach can potentially expedite evidence surveillance in a clear and reproducible manner, thus offering a broader understanding of the existing evidence base through interactive yields.(1) Evidence maps and evidence visualizations are systematic evidence synthesis approaches, which work by displaying visually the gaps in evidence or study characteristics, and, at times, summarize study quality or synthesized evidence from multiple studies. Such an interactive and visual representation provides a quick overview of the existing evidence base, thereby helping stakeholders and researchers to immediately understand research priorities.(7) For these reasons, evidence maps are excellent tools that help in guiding clinical investigators to set the agenda for future research.(8)

    Being a rather new concept, there has been no uniform definition of, or methodology for conducting, evidence maps yet. Mainly, evidence maps are referred to as tools of systematic organisation and illustration of evidence base with the intent to characterize the breadth, depth and methodology of relevant evidence, identifying gaps.(9) Another definition of evidence map is “an approach to providing a visual representation and critical assessment of evidence landscape for a particular topic or question”.(10) A more recent definition is developed from the published evidence maps, which turned out to be a systematic search of a broad field identifying gaps in knowledge and the needs for future research.(6) The last one thus takes evidence maps to be a user-friendly representation of evidence bases visually in a figure or graph, a table or a searchable database.(8)

    Nonetheless, due to the lack of a uniform definition, the stakeholders may not essentially know what to expect while warranting an evidence map or identifying existing maps. Moreover, lack of a repository for evidence maps makes them difficult to locate, thus making it less likely for authors to develop existing approaches further.(11)

    Essentially, evidence maps are primarily prepared by the relevant stakeholders (researchers, policy-makers, funders, and, most importantly, patients) by identifying the most important clinical questions to their context, and researching on the body of evidence that is available already. Next, the quality of the available evidence is assessed and conveyed to stakeholders. The final step includes the visual depiction of the most relevant data elements to the stakeholder; for e.g., focusing on the size of the body of evidence, comparisons made versus those avoided, populations studied versus those avoided, and risk of bias, among other factors.(8) At the end of this process, the gaps in the available evidence in the context of the original research question starts to become apparent, and this can be used to plan further research activities.

    In conclusion, evidence maps offer a robust and transparent methodological framework with which to assess the evidence landscape in a detailed manner, and aid clinical and regulatory decision-making. The broad scope of evidence maps, through efficient use of resources, can substantially streamline evidence synthesis by preventing unnecessary duplication of work. Additionally, future text mining and machine learning advancements will further possibly reduce the resource intensity of the methodology.(8)

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Wolffe TAM, Whaley P, Halsall C, et al. Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int. 2019; 130:104871.
    2. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010; 7(9):e1000326.
    3. Tricco AC, Antony J, Zarin W, et al. A scoping review of rapid review methods. BMC Med. 2015; 13(1):224.
    4. Colquhoun HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014; 67(12):1291–4.
    5. Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005; 10 suppl 1:21–34.
    6. Miake-Lye IM, Hempel S, Shanman R, et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016; 5:28.
    7. Evidence Maps and Evidence Visualizations. Patient-centered Outcomes Research Institute. Available at: https://www.pcori.org/impact/evidence-maps-and-evidence-visualizations
    8. Alahdab F, Murad MH. BMJ Evidence-Based Medicine. 2019.
    9. Katz DL, Williams AL, Girard C, et al. The evidence base for complementary and alternative medicine: methods of Evidence Mapping with application to CAM. Altern Ther Health Med 2003; 9:22–30.
    10. Bethan C, O’Leary PW, Kaiser MJ, et al. Evidence maps and evidence gaps: evidence review mapping as a method for collating and appraising evidence reviews to inform research and policy. Environmental Evidence 2017; 6.
    11. Evidence and gap maps: A comparison of different approaches. Oslo, Norway: The Campbell Collaboration. Retrieved from: campbellcollaboration.org/ DOI: https://doi.org/10.4073/cmdp.2018.2

    MarksMan Healthcare

    , ,

  • The Importance of Grey Literature Search in Systematic Literature Reviews

    The Importance of Grey Literature Search in Systematic Literature Reviews

    During the course of any research, most of the relevant literature pertaining to a research question is retrieved though searching of recognised databases. However, in addition to this, searching of grey literature can add value to the depth of the research by providing information from varied sources. Grey literature search is an important, but often ignored, part of systematic literature review and data synthesis, especially in the medical research.

    Grey Literature: Definition, Types, and Sources

    The Grey Literature International Steering Committee (GLISC) defines grey literature as “Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body”.[1] Grey literature is often self-published, and the sources of grey literature can include Government agencies, research institutions, organizations, companies, and associations.[2]

    Grey literature can be classified under various categories: [2,3]

    • Regulatory Information: this includes information available from archives of regulatory bodies such as the USFDA, CDSCO, EMA, and NICE. Regulatory information can also be sourced from stakeholder organizations and pharmaceutical companies, and include product information leaflets, white papers, internal documentations, SOPs, and procedure briefs. Other examples include company and industry wide repositories and financially driven investor service websites.
    • Government data: these include information available from government resources, such as notifications, guidelines, gazette notifications, judicial information, patent databases, policy briefs, etc.
    • Unpublished material from clinical trials: these include prospectively registered clinical trial protocols in repositories such as ClinicalTrials.gov in the USA and the Clinical Trial Registry of India (CTRI). Pre-prints which are not ultimately published due to various reasons, unpublished dissertations and theses,
    • Conference proceedings: abstracts, scientific sessions, and other conference proceedings provide a brief snapshot of contemporary research, which might not get published due to various reasons
    • Internet resources: With the increasing presence of social media, newer forms of grey literature have also surfaced, such as blogs, internet forums, wikis, video lectures, lecture slides and lecture notes, educational videos, personal websites, and information posted on the omnipresent social media.

    Importance of Including Grey Literature

    Commercial publishers are guided by interests and priorities, and all information which do not conform to these are often ignored and not published. This unpublished information forms the bulk of grey literature.

    A research which focuses solely on published material has a risk of missing out a comprehensive view of the topic under research. Grey literature provides valuable information about emerging or less popular research areas which are not published. Including a grey literature is also found to be useful in validating the results of a research-based literature search.[4]

    Grey literature bypasses the time-consuming peer-review process. Also, because of a quicker publication, the time delay between research and its formal publication is also bypassed. As a result, the information in grey literature can be more recent and up-to-date than in a formal publication. This is especially important in situations of public health emergency, such as the COVID-19 pandemic.

    There is a growing interest in using grey literature in systematic reviews and meta-analysis.[5] A study conducted by McCauley et al. analysed that 33% of meta‐analysis included some form of grey literature, accounting for 4.5% to 75% of studies in the meta‐analyses, and contributed significantly to the estimates of the intervention effects. The authors concluded that excluding grey literature from meta-analyses can result in a falsely exaggerated estimates of the effectiveness of intervention.[6] In several cases, information, and data disclosed at conference presentations is never published.[7] These observations have led to an opinion that the confounding effect of publication bias (where negative findings are more often not published) can be mitigated to a significant extent by including grey literature.[5-7]

    Challenges in Grey Literature Searching

    It is certain that grey literature search adds value to a research; however, it is also not possible to ascertain whether such a grey literature search has been done comprehensively or not. This is because grey literature is not well organized. In other words, there is no way to ‘define’ a proper grey literature search. It might be possible that a researcher has done grey literature search and included only data that is favourable, while excluding unfavourable data. This is in stark contrast to the published literature, which is often found to be well-organized. Thus, while it is possible to duplicate a database search strategy to verify if the search has been performed properly or not, such a luxury is not available with the quite unorganized grey literature. This is also the reason why inclusion of grey literature is more often than not entirely dependent on the choice of a researcher. Further, grey literature is not ensured to be peer-reviewed, which brings in an inherent bias.

    Conclusion

    The importance of transparency in research findings cannot be over emphasized. Publication bias still continues to be a huge problem in medical research. Grey literature search has a unique potential to improve transparency in medical research as well as offer a solution for publication bias, by including the unpublished information, thereby improving the comprehensiveness of research. With the increased ease of access of the internet, the access to grey literature has also become easier. Considering the amount of new information that grey literature can bring to research, all researchers must consider including grey literature search in their works. Efforts are required to organize the grey literature so that the credibility and validity of grey literature search can improve.

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Grey Literature International Steering Committee. Guidelines for the production of scientific and technical reports: how to write and distribute grey literature. Available from: http://eprints.rclis.org/7469/1/nancy.pdf. Accessed on Jun 20th 2020
    2. Royal Roads University. Grey literature: what is it?: What is grey literature. Available from: https://libguides.royalroads.ca/greylit/what. Accessed on Aug 12th
    3. Citrome L. Beyond PubMed: Searching the “Grey Literature” for Clinical Trial Results. Innov Clin Neurosci. 2014 Jul;11(7-8):42-6.
    4. Benzies KM, Premji S, Hayden KA, et al. State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews Evid Based Nurs. 2006;3(2):55-61.
    5. Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synth Methods. 2014;5(3):221-34.
    6. McAuley L, Pham B, Tugwell P, et al. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228-31.
    7. Hopewell S, McDonald S, Clarke M, et al. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007(2):MR000010.

    MarksMan Healthcare

    , , ,

  • What Are The Pros & Cons of Network Meta-Analysis (NMA)?

    What Are The Pros & Cons of Network Meta-Analysis (NMA)?

    Evidence-based medicine (EBM) is gaining wide acceptance from researchers globally as it thoroughly optimizes the latest available evidence to make informed care decisions. This involves evaluating the quality of the clinical data by critically assessing methodologies reported in publications. Moreover, EBM incorporates both clinical expertise as well as patient values. Meta-analyses of RCTs often make it among the top of the evidence hierarchy, since it’s regarded as the most valid clinical proof. Indeed, meta-analysis is a validated method to analyse and summarize knowledge by increasing the number of patients, and thus also the effective statistical power. However, there are several limitations associated with meta-analysis, which considers only pairwise comparisons. Unfortunately, head-to-head comparisons are not always available in the literature or they fail to answer a specific clinical question. This can be overcome with the help of network meta-analysis (NMA), which helps providing a global estimate of efficacy or safety of numerous experimental treatments that have not before been directly compared with adequate precision, or at all. Network meta-analysis integrates both direct and indirect effects from the entire set of evidence. Additionally, it ranks the treatments as the best or worst on the basis of valid statistical inference methods. (1)

    Network meta-analysis is preferable over conventional pair-wise meta-analysis, since it uses indirect evidence to justify comparisons amongst all treatments, thus enabling estimation of comparative effects that have not been investigated as precisely in RCTs. Thus, NMA is increasingly getting popular with clinicians, guideline developers, and HTA agencies as the ever growing new evidence needs to be placed in the context of all available evidence for appraisals. (2)

    Furthermore, NMA is increasingly becoming essential to formulating recommendations on reimbursements as well as clinical guidelines by healthcare agencies around the world. It has recently been adopted by Cochrane, as 10% (23/230) of their systematic reviews since the year 2015 have used NMA. In 2015, GRADE working groups published guidance on using GRADE in conjunction with NMA. Furthermore, the National Institute for Health and Clinical Excellence (NICE, UK) also approves the application of NMA within its clinical guidelines manual. (3)

    In addition, the ability of NMA to quantitatively assess interventions that have not been directly compared in studies aids the process of developing guidelines. This is because, in the absence of head-to-head evidence, guideline development groups will bank more strongly on expert opinion. Also, collective analysis of both direct and indirect evidence strengthens the evidence base. Moreover, while NMA determines the comparative effectiveness of drugs, the approach can be applied more broadly. For instance, the latest revision of the WHO HIV guidelines represents how the use of NMA to evaluate interventions in improving adherence to ART significantly contributed in the formulation of recommendations. (4)

    Although NMA is a powerful tool for comparative effectiveness research, it is more complex than pair-wise meta-analysis. Also, the assumption of transitivity is stringent, which is essential to consider throughout the entire process of NMA. Supplementary analyses like network meta-regression are often necessary, which further increase the complexity of the analysis. What’s more, NMA is very resource-demanding. As NMAs generally cater to broader questions, they usually involve more studies at each step of the systematic review, right from screening to analysis, than conventional meta-analysis. Therefore, it is crucial to schedule the time and resource commitment before actually conducting an NMA. (5)

    Network meta-analysis allows for indirect comparisons, incorporating more data in the analysis, thus tackling the bigger picture; while a single pairwise meta-analysis offers a fragmented picture. However, NMA should be done carefully. There needs to be a clinical question developed with input from both a subject area clinical expert and a statistician. Assessments of transitivity and consistency are fundamental for ensuring the validity of NMA. Finally, as mentioned earlier, the time and resource commitments required to produce a high-quality NMA should be appropriately taken into consideration. (5)

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Greco T, et al. The attractiveness of network meta-analysis: a comprehensive systematic and narrative review. Heart, Lung and Vessels 2015; 7(2):133-142.
    2. Ward P. Network Meta-Analysis Can Play Decisive Role. June, 2013.
    3. Kanters S, et al. Use of network meta-analysis in clinical guidelines. Bulletin of the World Health Organization 2016; 94:782-784. 
    4. Consolidated guidelines on HIV prevention, diagnosis, treatment and care for key populations. 2016 update. Geneva: World Health Organization; 2016. 
    5. Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Intern Emerg Med 2016; 12(1):103-111.

    Written by – Ms. Tanvi Laghate

    MarksMan Healthcare

    , , ,

  • How NMAs are Helping in Taking Informed Clinical Decisions?

    How NMAs are Helping in Taking Informed Clinical Decisions?

    Network meta-analysis (NMA) is a type of meta-analysis that adds an additional variable to a meta-analysis, and instead of a simple summation of trials that have evaluated the same treatment, several different treatments are compared by statistical inference.1 NMA is also referred to as mixed treatments comparison or multiple treatments comparison meta-analysis.2,3,4

    It was recognised at the National Institute for Clinical Excellence (NICE) that there is an increasing need for technology appraisals and clinical guidelines to be informed by integrated analyses, because of the lack of sufficient head to head comparisons of new treatments to inform clinical practice.5 Literature suggests that NMA is a feasible option to inform clinical practice decisions, particularly in cases where several treatments are examined.6,7

    NMA includes a combination of direct evidence within the trials and indirect evidence across the trials, thereby providing estimates of relative efficacy between all the relevant interventions, even in cases where there has never been a head to head comparison.1,2,3 In essence, the treatment effects are calculated for all treatments or interventions using all the available evidence in one simultaneous analysis. 6,8

    NMA relies on two main assumptions; homogeneity of compared trials and consistency in direct and indirect evidence.7,9 A simple example of a NMA would be as follows. A trial compares drug A to drug B and another trial, including the same target patient population, compares drug B to drug C.  Assuming that drug A is superior to drug B in the first trial, and assuming drug B is equivalent to drug C in a second trial, the NMA allows a potential inference that statistically drug A is also superior to drug C for this particular target population.1,4,8 Therefore; one can say that if drug A is more effective than drug B, and drug B is equivalent to drug C, then drug A is also more effective drug C. 1,4,7

    The main advantage of NMA over traditional or pairwise meta-analysis is that it enables some certainty about all treatment comparisons based on the strength of indirect evidence, and it further allows an estimation of the comparative effects, which would not have been examined in parallel group randomized clinical trials.2,4 Overall, NMA potentially enable an assessment of the benefits and harms for more than two interventions for the same clinical condition.

    In terms of limitations with NMA, this type of meta-analysis is more likely to be valid when analysing sufficiently homogenous studies that include very similar patient populations.  As NMA increases the number and type of studies being compared and combined, there is more likelihood of studies getting combined, which are heterogeneous.1,3,9  In addition, the various overlapping meta-analyses with heterogeneous findings may potentially confound the readers and decision makers. Further, NMA from a practical point of view is more complex than the conventional pair-wise meta-analysis, and requires more time and resources. The various assumptions underlying conventional pairwise meta-analyses are well researched and understood; however, the assumptions related to NMA are seen to be more complex, leading to misinterpretations.

    The methodological work to address the limitations of NMAs is an on-going work, and in light of this fact, researchers and end-users should be cautious when interpreting results from NMAs, as inappropriate combination of studies may result in overestimation of treatment effects and therefore misleading results, with some uncertainty in improving patient outcomes! Nevertheless, NMAs are seen as useful tools that are increasingly becoming attractive because they provide a comprehensive framework for decision-making.

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013 Jul 16; 159(2):130-7.
    2. Mills EJ, Thorlund K, Ioannidis JP. Demystifying trial networks and network meta-analysis. BMJ. 2013 May 14; 346: f2914.
    3. National Institute for Health and Care Excellence. Guide to the Methods of Technology Appraisal 2013 [Internet]. London: National Institute for Health and Care Excellence (NICE); 2013 Apr. Process and Methods Guides No. 9. NICE Process and Methods Guides. [Viewed on 02/08/2018]
    4. Li T, Puhan MA, Vedula SS, Singh S, Dickersin K; Ad Hoc Network Meta-analysis Methods Meeting Working Group. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med. 2011 Jun 27; 9:79.
    5. Rawlins MD. In pursuit of quality: the National Institute for Clinical Excellence. Lancet. 1999; 353:1079–82.
    6. Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005; 331(7521):897–900.
    7. Tu YK, Faggion CM Jr. A primer on network meta-analysis for dental research. ISRN Dent. 2012; 2012:276520.
    8. Sutton A, Ades AE, Cooper N, Abrams K. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics. 2008; 26(9):753–767.
    9. Donegan S, Williamson P, D’Alessandro U, Tudur Smith C. Assessing key assumptions of network meta-analysis: a review of methods. Res Synth Methods. 2013 Dec; 4(4):291-323.

    Written By – Dr. Sandeep Moola (Research Fellow, The University of Adelaide, Australia)

    MarksMan Healthcare

    , , ,