Contact Us

Home

About us

Services

AI Excellence

case studies

publications

HEOR Certificationheor Certification

AI for Medical Writing

Contact

  • An Overview of Evidence Maps: a New Approach for Evidence Review Process

    An Overview of Evidence Maps: a New Approach for Evidence Review Process

    Systematic reviews (SRs) are the embodiment of the evidence-based approaches, and they have reformed clinical decision-making in almost all therapy areas. The approach of SRs was essentially developed to fulfil the need of the medical practitioners to obtain precise and consistent information about the efficacy and safety of a clinical intervention, diagnostic procedure, or a prognostic marker from a pool of evidence, which is apparently full of contradiction, heterogeneity and bias.(1) Although SRs and meta-analyses are robust and detail-oriented, they’re both resource intense, with a limited scope of outcomes.(2) In order to cater to a range of needs from the stakeholders, the SR approach has branched within the realm of evidence synthesis. For instance, rapid reviews provide for more urgent deadlines but may not follow all the methods of an SR,(3) scoping reviews include larger bodies of evidence, not requiring a detailed synthesis,(4) and realist reviews focus on the assessment of the functions of complex interventions, often comprising of evidence excluded from classic SRs.(5, 6)

    Given the resource intense nature of the SRs, it is essential to recognize the most informative research questions in order to maximize their value and efficiency in clinical and regulatory decision-making. It can be inefficient to invest resources in SRs barely as a means of addressing specific research questions, if data available to answer those questions is lacking. Therefore, decision-makers need to monitor and understand the evidence base as a whole, so as to quickly determine the emerging trends or issues of potential concern. This can, in turn, facilitate the development of proactive research questions by relevant stakeholders for SRs to answer.(1)

    Evidence mapping is a new approach for the evidence review process. This approach can potentially expedite evidence surveillance in a clear and reproducible manner, thus offering a broader understanding of the existing evidence base through interactive yields.(1) Evidence maps and evidence visualizations are systematic evidence synthesis approaches, which work by displaying visually the gaps in evidence or study characteristics, and, at times, summarize study quality or synthesized evidence from multiple studies. Such an interactive and visual representation provides a quick overview of the existing evidence base, thereby helping stakeholders and researchers to immediately understand research priorities.(7) For these reasons, evidence maps are excellent tools that help in guiding clinical investigators to set the agenda for future research.(8)

    Being a rather new concept, there has been no uniform definition of, or methodology for conducting, evidence maps yet. Mainly, evidence maps are referred to as tools of systematic organisation and illustration of evidence base with the intent to characterize the breadth, depth and methodology of relevant evidence, identifying gaps.(9) Another definition of evidence map is “an approach to providing a visual representation and critical assessment of evidence landscape for a particular topic or question”.(10) A more recent definition is developed from the published evidence maps, which turned out to be a systematic search of a broad field identifying gaps in knowledge and the needs for future research.(6) The last one thus takes evidence maps to be a user-friendly representation of evidence bases visually in a figure or graph, a table or a searchable database.(8)

    Nonetheless, due to the lack of a uniform definition, the stakeholders may not essentially know what to expect while warranting an evidence map or identifying existing maps. Moreover, lack of a repository for evidence maps makes them difficult to locate, thus making it less likely for authors to develop existing approaches further.(11)

    Essentially, evidence maps are primarily prepared by the relevant stakeholders (researchers, policy-makers, funders, and, most importantly, patients) by identifying the most important clinical questions to their context, and researching on the body of evidence that is available already. Next, the quality of the available evidence is assessed and conveyed to stakeholders. The final step includes the visual depiction of the most relevant data elements to the stakeholder; for e.g., focusing on the size of the body of evidence, comparisons made versus those avoided, populations studied versus those avoided, and risk of bias, among other factors.(8) At the end of this process, the gaps in the available evidence in the context of the original research question starts to become apparent, and this can be used to plan further research activities.

    In conclusion, evidence maps offer a robust and transparent methodological framework with which to assess the evidence landscape in a detailed manner, and aid clinical and regulatory decision-making. The broad scope of evidence maps, through efficient use of resources, can substantially streamline evidence synthesis by preventing unnecessary duplication of work. Additionally, future text mining and machine learning advancements will further possibly reduce the resource intensity of the methodology.(8)

    Become A Certified HEOR Professional – Enrol yourself here!

    References

    1. Wolffe TAM, Whaley P, Halsall C, et al. Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environ Int. 2019; 130:104871.
    2. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010; 7(9):e1000326.
    3. Tricco AC, Antony J, Zarin W, et al. A scoping review of rapid review methods. BMC Med. 2015; 13(1):224.
    4. Colquhoun HL, Levac D, O’Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014; 67(12):1291–4.
    5. Pawson R, Greenhalgh T, Harvey G, et al. Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005; 10 suppl 1:21–34.
    6. Miake-Lye IM, Hempel S, Shanman R, et al. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Syst Rev. 2016; 5:28.
    7. Evidence Maps and Evidence Visualizations. Patient-centered Outcomes Research Institute. Available at: https://www.pcori.org/impact/evidence-maps-and-evidence-visualizations
    8. Alahdab F, Murad MH. BMJ Evidence-Based Medicine. 2019.
    9. Katz DL, Williams AL, Girard C, et al. The evidence base for complementary and alternative medicine: methods of Evidence Mapping with application to CAM. Altern Ther Health Med 2003; 9:22–30.
    10. Bethan C, O’Leary PW, Kaiser MJ, et al. Evidence maps and evidence gaps: evidence review mapping as a method for collating and appraising evidence reviews to inform research and policy. Environmental Evidence 2017; 6.
    11. Evidence and gap maps: A comparison of different approaches. Oslo, Norway: The Campbell Collaboration. Retrieved from: campbellcollaboration.org/ DOI: https://doi.org/10.4073/cmdp.2018.2

    MarksMan Healthcare

    , ,

  • Publication Bias, Favourable Results And Publicly Funded Studies

    Publication Bias, Favourable Results And Publicly Funded Studies

    Clinical practice should ideally rely on robust scientific evidence, the standard for which are systematic reviews and meta-analyses of all randomised controlled trials (RCTs). (1,2) Therapeutic decisions in healthcare must be informed by clinical research findings, and patients and prescribers must be able to trust the presented research evidence. However, this evidence can be considered valid only if the studies included in reviews and meta-analyses signify the complete publications, without any bias. (2) Recently, the reliability of much of the evidence base for several popular therapeutic and preventive interventions has been challenged due to the publication bias. (3)

    What is ‘publication bias’? It refers to data distortion in scientific journals mainly due to the increased likeliness of publications of those studies with significant and positive results compared to those with unfavourable or negative or insignificant results. (2)  Publication bias, in essence, is a threat to the core principle of evidence-based medicine that is based on systematic reviews of published evidence providing accurate assessments of an intervention’s actual safety and efficacy data. (4) Moreover, the methods used to conduct the systematic review represent their validity. The presence of a systematic bias of favouring the publication of studies with statistically significant or positive findings substantially threatens the validity of the conclusions of a systematic review. (5) Study selection based on their status of publication (submitted or accepted), duplicate, undetected publications, and selective reporting are some of the factors that lead to publication bias. In addition, the bias also occurs when the publication selectively depends on the nature and direction of the study findings, (6) which will then be thoroughly different from those of unpublished studies.(2)

    Evidence from literature confirms that the probability of the publication of studies with positive and favourable results is more than that of studies with negative or unfavourable results.(1,2,7) For instance, a systematic review and meta-analysis of 85 cohorts assessed their likelihood of publication based on different variables, such as favourability of results, statistical significance, study sponsorship, number of study centers, study phases, study design, study size, and country of origin; with the outcome of interest being complete publication in a peer-reviewed journal.(4) The authors found that favourability of results, statistical significance, and multicenter status were all significantly impacted the probability of publication. Moreover, the likeliness of publication of the studies presented as abstracts was significantly higher if a sponsor funded the studies. Therefore, the likeliness of publication for favourable study findings was significantly influenced by the study’s funding status.(3) This only goes on to suggest a strong association of differential publication due to the favourability of study findings and the status of sponsorship from the industry.(4)

    The decision-making in clinical and medical practice should depend on the entirety of research evidence and not on a sample that is biased by selective publication only of studies showing favourable findings.(6) To overcome the hurdle of bias and support the complete, unbiased publication of studies, researchers are suggesting a mandate for clinical trials to register before recruiting patients so that the authors of systematic reviews know about all potentially eligible studies, notwithstanding their findings. It will also help if authors of systematic reviews ensured the assessment of the potential problems of publication bias in their review, thus considering methods for addressing this issue and confirming an inclusive search for both published and unpublished trials. (5) Finally, precise and robust measures, if taken by the scientific and medical organizations, ethical committees, regulatory bodies, journal editors, and the industry itself, will ensure that commercial interests of pharmaceutical companies do not weaken the knowledge of scientifically correct study planning, study execution, and publication.(2)

    Become A Certified HEOR Professional – Enrol yourself here!

    References  

    1. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine: selective reporting from studies sponsored by pharmaceutical industry; review of studies in new drug applications. BMJ. 2003; 326:1171-3.
    2. Kerekovska A, Galunska B. Publication bias in clinical research sponsored by pharmaceutical industry. Scripta Scientifica Pharmaceutica. 2014; 1:7-13.
    3. Jefferson T. Sponsorship bias in clinical trials: growing menace or dawning realisation? Journal of the Royal Society of Medicine. 2020; 113(4):148-157.
    4. Canestaro WJ, Hendrix N, Bansal A, et al. Favorable and publicly funded studies are more likely to be published: a systematic review and meta-analysis. J Clin Epidemiol. 2017; 92:58-68.
    5. Hopewell S, Loudon K, Clarke MJ, et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: MR000006.
    6. Song F, Eastwood AJ, Gilbody S, et al. Publication and related biases. Health Technology Assessment. 2000; 4(10):1-105.
    7. McGauran N, Wieseler B, Kreis J, et al. Reporting bias in medical research – a narrative review. BioMed Central Trials. 2010; 11:37.

    MarksMan Healthcare

    ,

  • What Are The Pros & Cons of Network Meta-Analysis (NMA)?

    What Are The Pros & Cons of Network Meta-Analysis (NMA)?

    Evidence-based medicine (EBM) is gaining wide acceptance from researchers globally as it thoroughly optimizes the latest available evidence to make informed care decisions. This involves evaluating the quality of the clinical data by critically assessing methodologies reported in publications. Moreover, EBM incorporates both clinical expertise as well as patient values. Meta-analyses of RCTs often make it among the top of the evidence hierarchy, since it’s regarded as the most valid clinical proof. Indeed, meta-analysis is a validated method to analyse and summarize knowledge by increasing the number of patients, and thus also the effective statistical power. However, there are several limitations associated with meta-analysis, which considers only pairwise comparisons. Unfortunately, head-to-head comparisons are not always available in the literature or they fail to answer a specific clinical question. This can be overcome with the help of network meta-analysis (NMA), which helps providing a global estimate of efficacy or safety of numerous experimental treatments that have not before been directly compared with adequate precision, or at all. Network meta-analysis integrates both direct and indirect effects from the entire set of evidence. Additionally, it ranks the treatments as the best or worst on the basis of valid statistical inference methods. (1)

    Network meta-analysis is preferable over conventional pair-wise meta-analysis, since it uses indirect evidence to justify comparisons amongst all treatments, thus enabling estimation of comparative effects that have not been investigated as precisely in RCTs. Thus, NMA is increasingly getting popular with clinicians, guideline developers, and HTA agencies as the ever growing new evidence needs to be placed in the context of all available evidence for appraisals. (2)

    Furthermore, NMA is increasingly becoming essential to formulating recommendations on reimbursements as well as clinical guidelines by healthcare agencies around the world. It has recently been adopted by Cochrane, as 10% (23/230) of their systematic reviews since the year 2015 have used NMA. In 2015, GRADE working groups published guidance on using GRADE in conjunction with NMA. Furthermore, the National Institute for Health and Clinical Excellence (NICE, UK) also approves the application of NMA within its clinical guidelines manual. (3)

    In addition, the ability of NMA to quantitatively assess interventions that have not been directly compared in studies aids the process of developing guidelines. This is because, in the absence of head-to-head evidence, guideline development groups will bank more strongly on expert opinion. Also, collective analysis of both direct and indirect evidence strengthens the evidence base. Moreover, while NMA determines the comparative effectiveness of drugs, the approach can be applied more broadly. For instance, the latest revision of the WHO HIV guidelines represents how the use of NMA to evaluate interventions in improving adherence to ART significantly contributed in the formulation of recommendations. (4)

    Although NMA is a powerful tool for comparative effectiveness research, it is more complex than pair-wise meta-analysis. Also, the assumption of transitivity is stringent, which is essential to consider throughout the entire process of NMA. Supplementary analyses like network meta-regression are often necessary, which further increase the complexity of the analysis. What’s more, NMA is very resource-demanding. As NMAs generally cater to broader questions, they usually involve more studies at each step of the systematic review, right from screening to analysis, than conventional meta-analysis. Therefore, it is crucial to schedule the time and resource commitment before actually conducting an NMA. (5)

    Network meta-analysis allows for indirect comparisons, incorporating more data in the analysis, thus tackling the bigger picture; while a single pairwise meta-analysis offers a fragmented picture. However, NMA should be done carefully. There needs to be a clinical question developed with input from both a subject area clinical expert and a statistician. Assessments of transitivity and consistency are fundamental for ensuring the validity of NMA. Finally, as mentioned earlier, the time and resource commitments required to produce a high-quality NMA should be appropriately taken into consideration. (5)

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Greco T, et al. The attractiveness of network meta-analysis: a comprehensive systematic and narrative review. Heart, Lung and Vessels 2015; 7(2):133-142.
    2. Ward P. Network Meta-Analysis Can Play Decisive Role. June, 2013.
    3. Kanters S, et al. Use of network meta-analysis in clinical guidelines. Bulletin of the World Health Organization 2016; 94:782-784. 
    4. Consolidated guidelines on HIV prevention, diagnosis, treatment and care for key populations. 2016 update. Geneva: World Health Organization; 2016. 
    5. Rouse B, Chaimani A, Li T. Network meta-analysis: an introduction for clinicians. Intern Emerg Med 2016; 12(1):103-111.

    Written by – Ms. Tanvi Laghate

    MarksMan Healthcare

    , , ,

  • What All You Need to Know About PROBAST?

    What All You Need to Know About PROBAST?

    Today’s era of risk based, precision and personalized medicine demands clinical prediction models. Prediction modelling studies focus on two kinds of outcomes, viz. diagnosis (probability of a condition that is undetected) and prognosis (probability of developing a certain outcome in the future). (1,2) These studies develop, validate, or update a multivariable prediction model, wherein multiple predictors are used in combination to estimate probabilities to inform and often guide individual care. Moreover, evidence from literature shows both prognostic as well as diagnostic models being widely used in various medical domains and settings, (3) such as cancer, (4) neurology, (5) and cardiovascular disease. (6) Increasingly common competing prediction models can exist for the same outcome or target population, which necessitate the systematic reviews of these prediction model studies; since their coexistence may facilitate misperceptions amongst health care providers, guideline developers, and policymakers about which model to use or recommend, and in which persons or settings. (1,7)

    Quality assessment is vital while conducting any systematic review, for which several tools are in place that enable the assessment of the risk of bias (ROB). (8) For example, the QUIPS (Quality In Prognosis Studies) tool evaluates the ROB in predictor finding (prognostic factor) studies. (9) Similarly, the revised Cochrane ROB Tool (ROB 2.0) (10) investigates the methodological quality of prediction model impact studies, that use a randomized comparative design, or ROBINS-I (Risk of Bias in Nonrandomized Studies of Interventions) for those incorporating a non-randomized comparative design. (11) Today, prediction model studies as well as their systematic reviews are often being used as evidence for clinical guidance and decision making, which warrants a tool that would facilitate quality assessment for individual prediction model studies. For this purpose, PROBAST (Prediction model Risk Of Bias ASsessment Tool) has been recently introduced. PROBAST came into existence owing to the lack of appropriate tool that would evaluate the ROB for systematic reviews of diagnostic and prognostic prediction model studies. (7,8,12)

    Bias is nothing but a systematic error in a study that leads to inaccurate results, thus inhibiting the study’s internal validity. (8) Similarly, inadequacies of the study design, conduct and analysis may often lead to the distorted estimates of model predictive performance, thus facilitating the ROB to occur. Moreover, different populations, predictors, or outcomes of the study than those specified in the review question may give rise to the concerns regarding the applicability of a primary study. PROBAST has been, therefore, developed to address the lack of suitable tools designed specifically to assess ROB and applicability of primary prediction model studies.

    Development of PROBAST:

    A 4-stage approach for developing health research reporting guidelines was implemented in developing PROBAST. This approach consisted of following stages: 1) defining the scope, 2) reviewing the evidence base, 3) using a Web-based Delphi procedure, and 4) refining the tool through piloting. (8,13) PROBAST was designed mainly to assess primary studies included in a systematic review and not predictor finding or prediction model impact studies. The steering group of 9 experts in prediction model studies and development of quality assessment tools agreed that PROBAST would assess both ROB as well as the concerns regarding applicability of a study evaluating a multivariable prediction model to be used for individualized diagnosis or prognosis. For the first stage, a domain-based structure was adopted to define the scope of PROBAST, similar to that used in other ROB tools, such as ROB 2.0, ROBINS-I, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2), and ROBIS. A total of 3 approaches were used to build an evidence base as part of the second stage, wherein relevant methodological reviews were identified in the area of prediction model research, which was followed by identification of relevant methodological studies by members of the steering group, and lastly, additional evidence was identified with the help of applying the Delphi procedure in a wider group. All this evidence produced an initial list of signalling questions to consider for inclusion in PROBAST. In the third stage, a modified Delphi process, by means of web-based surveys, was used to gain structured feedback and agreement on the scope, structure, and content of PROBAST through 7 rounds. The 38-member Delphi group included methodological experts in prediction model research and development of quality assessment tools, experienced systematic reviewers, commissioners, and representatives of reimbursement agencies. The inclusion of various stakeholders ensured fair representation of the views of end users, methodological experts, and decision makers. In the fourth stage, the then-current version of PROBAST was piloted at multiple workshops at consecutive Cochrane Colloquia as well as numerous workshops with MSc and PhD students. The feedback received was used to further refine the content and structure of PROBAST, wording of the signalling questions, and content of the guidance documents. (7,8)

    PROBAST consists of 4 steps, viz. 1) specifying the systematic review question, 2) classifying the type of prediction model,  3) assessing ROB and applicability and 4) the overall judgement. PROBAST is the first comprehensively developed tool designed explicitly to assess the quality of prediction model studies for development, validation, or updating of both diagnostic and prognostic models, notwithstanding the medical domain, type of outcome, predictors, or statistical technique used. (7,8) PROBAST was introduced earlier this month in two parts; the first publication by Wolf et al.(8) highlights the development and scope of PROBAST, while the second publication by Moons et al. (8) explicitly describes the applications of PROBAST and how to judge ROB and applicability.

    Organizations that support decision making (such as the National Institute for Health and Care Excellence and the Institute for Quality and Efficiency in Health Care); researchers and/or clinicians interested in evidence-based medicine or involved in guideline development; and journal editors, manuscript reviewers are the potential users for PROBAST. (8)

    Become an Certified HEOR Professional – Enrol yourself here!

    References 

    1. Bouwmeester W, Zuithoff NP, Mallett S, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med 2012; 9:1-12.
    2. Steyerberg EW, Moons KG, van der Windt DA, et al; PROGRESS Group. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med 2013; 10:e1001381.
    3. Collins GS, Mallett S, Omar O, et al. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med 2011; 9:103.
    4. Altman DG. Prognostic models: a methodological framework and review of models for breast cancer. Cancer Invest 2009; 27:235-43.
    5. Counsell C, Dennis M. Systematic review of prognostic models in patients with acute stroke. Cerebrovasc Dis 2001; 12:159-70.
    6. Damen JA, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 2016; 353:i2416.
    7. Moons KGM, Wolf RF, Riley RD, et al. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Ann Intern Med 2019; 170:W1-W33.
    8. Wolf RF, Moons KGM, Riley RD, et al; for the PROBAST Group. PROBAST: A Tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019; 170:51-58.
    9. Hayden JA, van der Windt DA, Cartwright JL, et al. Assessing bias in studies of prognostic factors. Ann Intern Med 2013; 158:280-6.
    10. Higgins JPT, Savovic´ J, Page MJ, et al. ROB2 Development Group. A revised tool for assessing risk of bias in randomized trials. In: Chandler J, McKenzie J, Boutron I, Welch V, eds. Cochrane Methods. London: Cochrane; 2018:1-69.
    11. Sterne JA, Herna´n MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355:i4919.
    12. PROBAST. Available at: http://www.probast.org/ABOUT
    13. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med 2010; 7:e1000217.

    Written by: Ms. Tanvi Laghate

    MarksMan Healthcare

    , , ,

  • How NMAs are Helping in Taking Informed Clinical Decisions?

    How NMAs are Helping in Taking Informed Clinical Decisions?

    Network meta-analysis (NMA) is a type of meta-analysis that adds an additional variable to a meta-analysis, and instead of a simple summation of trials that have evaluated the same treatment, several different treatments are compared by statistical inference.1 NMA is also referred to as mixed treatments comparison or multiple treatments comparison meta-analysis.2,3,4

    It was recognised at the National Institute for Clinical Excellence (NICE) that there is an increasing need for technology appraisals and clinical guidelines to be informed by integrated analyses, because of the lack of sufficient head to head comparisons of new treatments to inform clinical practice.5 Literature suggests that NMA is a feasible option to inform clinical practice decisions, particularly in cases where several treatments are examined.6,7

    NMA includes a combination of direct evidence within the trials and indirect evidence across the trials, thereby providing estimates of relative efficacy between all the relevant interventions, even in cases where there has never been a head to head comparison.1,2,3 In essence, the treatment effects are calculated for all treatments or interventions using all the available evidence in one simultaneous analysis. 6,8

    NMA relies on two main assumptions; homogeneity of compared trials and consistency in direct and indirect evidence.7,9 A simple example of a NMA would be as follows. A trial compares drug A to drug B and another trial, including the same target patient population, compares drug B to drug C.  Assuming that drug A is superior to drug B in the first trial, and assuming drug B is equivalent to drug C in a second trial, the NMA allows a potential inference that statistically drug A is also superior to drug C for this particular target population.1,4,8 Therefore; one can say that if drug A is more effective than drug B, and drug B is equivalent to drug C, then drug A is also more effective drug C. 1,4,7

    The main advantage of NMA over traditional or pairwise meta-analysis is that it enables some certainty about all treatment comparisons based on the strength of indirect evidence, and it further allows an estimation of the comparative effects, which would not have been examined in parallel group randomized clinical trials.2,4 Overall, NMA potentially enable an assessment of the benefits and harms for more than two interventions for the same clinical condition.

    In terms of limitations with NMA, this type of meta-analysis is more likely to be valid when analysing sufficiently homogenous studies that include very similar patient populations.  As NMA increases the number and type of studies being compared and combined, there is more likelihood of studies getting combined, which are heterogeneous.1,3,9  In addition, the various overlapping meta-analyses with heterogeneous findings may potentially confound the readers and decision makers. Further, NMA from a practical point of view is more complex than the conventional pair-wise meta-analysis, and requires more time and resources. The various assumptions underlying conventional pairwise meta-analyses are well researched and understood; however, the assumptions related to NMA are seen to be more complex, leading to misinterpretations.

    The methodological work to address the limitations of NMAs is an on-going work, and in light of this fact, researchers and end-users should be cautious when interpreting results from NMAs, as inappropriate combination of studies may result in overestimation of treatment effects and therefore misleading results, with some uncertainty in improving patient outcomes! Nevertheless, NMAs are seen as useful tools that are increasingly becoming attractive because they provide a comprehensive framework for decision-making.

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013 Jul 16; 159(2):130-7.
    2. Mills EJ, Thorlund K, Ioannidis JP. Demystifying trial networks and network meta-analysis. BMJ. 2013 May 14; 346: f2914.
    3. National Institute for Health and Care Excellence. Guide to the Methods of Technology Appraisal 2013 [Internet]. London: National Institute for Health and Care Excellence (NICE); 2013 Apr. Process and Methods Guides No. 9. NICE Process and Methods Guides. [Viewed on 02/08/2018]
    4. Li T, Puhan MA, Vedula SS, Singh S, Dickersin K; Ad Hoc Network Meta-analysis Methods Meeting Working Group. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med. 2011 Jun 27; 9:79.
    5. Rawlins MD. In pursuit of quality: the National Institute for Clinical Excellence. Lancet. 1999; 353:1079–82.
    6. Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005; 331(7521):897–900.
    7. Tu YK, Faggion CM Jr. A primer on network meta-analysis for dental research. ISRN Dent. 2012; 2012:276520.
    8. Sutton A, Ades AE, Cooper N, Abrams K. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics. 2008; 26(9):753–767.
    9. Donegan S, Williamson P, D’Alessandro U, Tudur Smith C. Assessing key assumptions of network meta-analysis: a review of methods. Res Synth Methods. 2013 Dec; 4(4):291-323.

    Written By – Dr. Sandeep Moola (Research Fellow, The University of Adelaide, Australia)

    MarksMan Healthcare

    , , ,

  • Generating Scientific Evidence (Efficacy/Safety/Cost Data) from India

    Generating Scientific Evidence (Efficacy/Safety/Cost Data) from India

    Every country exercises strict control on medicines’ market access. Typically, this requires successful completion and adequate presentation of results from phase I through phase III clinical trials, bringing forward the findings of medicine’s safety and efficacy. The USFDA approves approximately 40 new medicines for the US market each year through this process. (1) In India, this number is more than 100 new medicines annually; however, there is not enough published evidence on submitted applications or summaries of approved medicines. Therefore, concerns are being raised about the safety and efficacy around medicine approvals in India in the absence of appropriate clinical trials. (2,3)

    For instance, a recent study, which evaluated the clinical evidence on the safety and efficacy of the most common metformin fixed dose combinations (FDCs) for T2DM in India, has highlighted the growing national and international concerns about the Indian drug regulatory system. Findings from this study further show high numbers of unapproved medicines and their irrational combinations floating in the market. This study has assessed the basis of efficacy and safety of top-selling metformin FDCs in India against four WHO criteria from clinical trial guidelines for the approval of FDCs. In India, only five FDCs have been approved by the Central Drugs Standard Control Organization (CDSCO); while, in reality, the Indian FDC-diabetes market contributes to the two-third of all diabetes medicine sales. (4) Furthermore, evaluation of published and unpublished clinical trials of these approved FDCs seemed to show underpowered and poor quality evidence of safety and efficacy for the treatment of T2DM. (5)

    The overall lack of available India-specific evidence heightens the need for its generation by publishing the unpublished trial results with Indian patients. India has in place the only required registration with Clinical Trials Registry – India, the national clinical trials database, since 2009. Moreover, the unpublished trials listed in this registry merely provide basic trial information with no results or outcomes reported. The lack of trials on Indian patients, in particular, is of concern, considering CDSCO’s guidelines for drug approvals acknowledge the importance of conducting trials on the Indian population to determine safety and efficacy.4

    Additionally, the Government, with an aim to achieve Universal Health Coverage (UHC) in order to reduce huge out-of-pocket (OOP) health expenditure and ensure affordable access to essential health care for the entire population, has identified a key priority of ensuring value for money in the health budget. This requires a systematic process for generating policy-relevant evidence that can inform policy decisions regarding health resource allocation, i.e. clinical effectiveness studies, cost-effectiveness studies, budget impact studies, along with ethical, social and political feasibility studies. (6) Needless to say that the healthcare payers, regulatory authorities, and health technology assessment (HTA) agencies also make decisions on relative efficacy of the new products based on evidence generated from clinical trials. (7)

    In most western countries along with the United States, the consumer rarely pays for the product—the payer is generally a third-party private or governmental insurer. Before approving a new medical entity (medicines/medical technologies) for reimbursement, private and governmental payers analyze clinical and economic data to determine the clinical value and cost-effectiveness of the new product as compared with currently available treatments. (8) Indian health system, on the other hand, is characterized by a vast but under-utilized public health infrastructure and a largely unregulated private market catering to greater need for curative action; where high OOP health expenditures hinder access to healthcare. (9)

    We believe it is high time even insurance companies start asking for robust evidence in order to provide reimbursement of better healthcare technologies and easier access to care. India needs to bring about a major reform in its health insurance policies, wherein a keen eye for detail is given to the published trial data on safety and efficacy of a drug or relevant evidence about a medical technology.

    Become an Certified HEOR Professional – Enrol yourself here!

    References:

    1. U.S. Food and Drug Administation. New Drugs at FDA: CDER’s New Molecular Entities and New Therapeutic Biological Products.
    2. Ministry of Health and Family Welfare, Department of Health and Family Welfare. Gazette of India, 10 March 2016. New Delhi, 2016.
    3. McGettigan P, et al. Regulatory upheaval and irrational medicines in India: a study of fixed-dose combination drugs. PLoS Med 2015; 12:e1001826.
    4. Evans V, et al. Adequacy of clinical trial evidence of metformin fixed-dose combinations for the treatment of type 2 diabetes mellitus in India. BMJ Glob Health 2018; 3:e000263.
    5. Shimpi RD, et al. Comparison of effect of metformin in combination with glimepiride and glibenclamide on glycaemic control in patient with Type 2 diabetes mellitus. Int J PharmTech Res 2009; 1:50–61
    6. Prinja S, et al. Health Technology Assessment for Policy Making in India: Current Scenario and Way Forward. Pharmacoecon Open 2018; 2(1):1-3. 
    7. Dang A, et al. Real world evidence: An Indian perspective. Perspect Clin Res 2016; 7:156:160.
    8. Gold M. Getting reimbursement for your product in the United States. June, 2003. 
    9. Prinja S, et al. Universal Health Insurance in India: Ensuring Equity, Efficiency, and Quality. Indian Journal of Community Medicine : Official Publication of Indian Association of Preventive & Social Medicine. 2012; 37(3):142-149.

    MarksMan Healthcare

    , , , ,

  • Scoping Reviews: An Evolving Concept in EBM

    Scoping Reviews: An Evolving Concept in EBM

    Scoping reviews are exploratory projects that systematically map the literature on a topic, identifying key concepts, theories and sources of evidence. Scoping reviews are often conducted before full syntheses, and undertaken when feasibility of the research is considered to be a challenge, either because the relevant literature is thought to be vast and diverse (varying by methods, theoretical orientations and disciplines) and/or it is thought that little literature exists. In the scoping review, the same systematic, rigorous methodologies used by the systematic review are used to find studies and extract data. Analyses and syntheses are part of every scoping review but the depth and type of analysis are different.

    A scoping review (also scoping study) usually refers to a rapid gathering of literature in a given policy of clinical area where the aims are to accumulate as much evidence as possible and map the results. Scoping reviews provide an overview of the type, extent and quantity of research available on a given topic. By ‘mapping’ existing research, a scoping review can identify potential research gaps and future research needs, and do so by using systematic and transparent methods.

    In 2005, Arksey and O’Malley published the first methodological framework for conducting scoping studies. The term ‘scoping review’ does not seem to have a commonly-accepted definition but several researchers have attempted definitions. Scoping reviews can be an efficient way of indentifying themes and trends in high-volume areas of scientific enquiry. Generally, a scoping review is an interactive process whereby existing literatures identified, examined and conceptually mapped, and where gaps are identified. Thus, a scoping review could be considered as a first step in doing a systematic review or large study.

    In simple words, researchers can undertake a scoping study to examine the extent, range, and nature of research activity, determine the value of undertaking a full systematic review, summarize and disseminate research findings, in addition to identifying the gaps in the existing literature. As such, researchers can use scoping studies to clarify a complex concept and refine subsequent research inquiries. Scoping studies may be particularly relevant to disciplines with emerging evidence, such as rehabilitation science, in which the paucity of randomized controlled trials makes it difficult for researchers to undertake systematic reviews. In these situations, scoping studies are ideal because researchers can incorporate a range of study designs in both published and grey literature, address questions beyond those related to intervention effectiveness, and generate findings that can complement the findings of clinical trials.

    The literature search in a scoping review should be as extensive as possible, and include a range of relevant databases, hand searches and attempts to identify unpublished literature. Often, the underlying aim of a scoping review is to explore the literature as opposed to answering specific questions. The scoping review should also include locating organizations and individuals that are relevant to the domain and what those groups have published. In the social sciences, scoping studies are performed at an initial stage of doing research (i.e. program, project, process, or grant). Scoping reviews are used in some research areas to justify further investigation, time and resources.

    In evidence-based practice, scoping studies are undertaken as distinct research projects, and as precursors to other types of research. However, a scoping study may be requested as a search prior to the systematic review or preparatory to costing research projects. The interpretation, methodology and expectations of scoping reviews are variable and suggest that conceptually, scoping is not well-understood or defined. The distinction between scoping as an integral preliminary process in the development of a research proposal or a formative, methodologically rigorous activity in its own right has not been examined. Scoping studies in medicine are slowly evolving; their strength lies in their ability to summarize a body of evidence for quick but accurate synthesis. As with other approaches to evidence synthesis a standardized approach is always welcome. Full literature searching aimed at retrieving a maximum number of relevant studies or articles in a given discipline starts with a scope of a topic.

    Become a Certified HEOR Professional – Enrol yourself here!

    MarksMan Healthcare

    , ,

  • EBM and HTA for Healthcare Decision Making – The Time has Come!

    EBM and HTA for Healthcare Decision Making – The Time has Come!

    Health systems have developed at different speeds, and with differing degrees of complexity throughout the twentieth century, reflecting the diverse political and social conditions in each country. Notwithstanding their diversity, all systems, however, share a common reason for their existence, namely the improvement of health for their entire populations. To attain this goal a health system undertakes a series of functions, most notably, the financing and delivering of health services.

    Since available resources are limited, delivering health services involves making decisions. Decisions are required on what interventions should be offered, the way the health system is organized, and how the interventions should be provided in order to achieve an optimal health gain with available resources, while, at the same time, respecting people’s expectations. Decision-makers thus need information about the available options and their potential consequences. It is now clear that interventions once thought to be beneficial have, in the light of more careful evaluation, turned out to be at best of no benefit or, at worst, harmful to the individual and counterproductive to the system. This recognition has led to the emergence of a concept known as “evidence-based medicine” (EBM), which argues that the information used by policymakers should be based on rigorous research to the fullest extent possible.

    Health technology assessment (HTA) increasingly plays an important role in informing reimbursement and pricing decisions and providing clinical guidance on the use of medical technologies across the world. In addition to safety and efficacy information, health economic and outcomes research (HEOR) data are also receiving expanded attention in these assessments in many countries, due to payers seeking better value for money spent on treatments. HTA is now commonly viewed as a tool to assist evidence-based health-care decisions.

    EBM has been defined as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”. The origin of this evidence-based approach can be seen in the application of clinical medicine delivered at an individual level. Pressure to base decisions on evidence has, however, been extended to other areas of health care, such as public health interventions and health care policy-making. In this context, evidence is understood as the product of systematic observation or experiment. It is inseparable from the notion of data collection. The evidence-based approach relies mainly on research, that is, on systematically collected and rigorously analyzed data following a pre-established plan.

    There are exciting new developments in basic science that could lead to targeted, highly effective and curative treatments. Health systems are improving their electronic records and recording health outcomes, which can be analyzed using structured, sophisticated analyses in real-time. There are also new collaborative approaches between healthcare providers and technology developers to enable evaluation of technologies in the health system before adoption or early in adoption to optimize use. There is a need and an opportunity to harness these developments and improve the effectiveness and efficiency of evidence production for new health technologies to input to HTA and inform decision making. Clinicians, managers, patients, and technology developers need to be involved to ensure that the process to a coverage decision is not only efficient but that it is also effective. To be effective, health services need to be organized to enable rapid and appropriate introduction of effective technologies and disinvestment of ineffective technologies. This suggests an additional responsibility for HTA and it would involve helping technology developers understand clinical and patient needs, evidence generation requirements, and limitations and helping health systems understand the potential and implications of new technologies and possible challenges of implementation.

    Therefore, to sum everything up, the evidence should be both efficient as well as effective in order to develop more agile and adaptive processes that help to broker alignment among technology developers and health systems (including healthcare professionals and patients). This suggests that HTA needs to innovate and be prepared to play a more active role to influence evidence production and help facilitate dialogue among stakeholders to optimize technology development and use.

    Become a Certified HEOR Professional – Enrol yourself here!

    MarksMan Healthcare

    , , ,
    , , , ,