Contact Us

Home

About us

Services

AI Excellence

case studies

publications

HEOR Certificationheor Certification

AI for Medical Writing

Contact

  • Impact of Information From ClinicalTrials.gov in the Conduct of SLRs?

    Impact of Information From ClinicalTrials.gov in the Conduct of SLRs?

    Systematic literature reviews (SLRs) are classically conducted with an aim of mapping the complete evidence base of a particular healthcare intervention, which enables an impartial evaluation of the evidence and lays the foundation for strong recommendations. The initial step in achieving this aim, while carrying out an SLR, is to conduct an extensive literature search in bibliographic databases, such as PubMed and EMBASE. However, just this step may not be enough, since these databases usually contain only articles published in scientific journals, except occasional abstracts from conferences. (1,2)

    Systematic literature reviews are of great importance when it comes to the level and quality of evidence, which is why they are extensively used by clinical policy-makers, granting health agencies, and journal editors alike. (3,4) Identifying all relevant randomized controlled trials (RCTs), irrespective of their publication status, poses a great challenge in the conduct of SLRs. (5,6) To state the fact, findings from half of these RCTs never get published, which may affect the publication status and  direction of results. This may further cause bias in the systematic review results. (7)

    Trial registries, set up for a hypothetical registration of trials, have proven to be an effective tool to reduce their selective publication. (80 Since the initial computerized registries in the US in 1960s, various national and international, public and private registries have been generated. However, just the registration of a trial may not yield any information about a particular healthcare intervention, since a complete information about methodology and results of a trial can be achieved only through its unbiased assessment. (1) Registration of clinical trials in public trial registers (for e.g. ClinicalTrials.gov) has become mandatory since July 2005, as per the recommendations from International Committee of Medical Journal Editors (ICMJE). (9) Additionally, the ‘Amendments Act’ (2007) by the USFDA recommends posting of clinical trial results on ClinicalTrials.gov within one year of final data collection for the pre-defined primary outcome, for all phase II to IV trials of drugs, biological treatments as well as devices. (4,10,11)

    The specifications warranted by authorities like USFDA, ICJME as well as National Institutes of Health Policies during registration of trials are – trial type, name of the intervention, trial phase, sponsors, outcomes, and types of data among other variable parameters. There also exist explanations about the information to be included in ClinicalTrials.gov in order to ensure compliance and well-timed submission of appropriate data. The regulators believe that the role of registry platforms, such as ClinicalTrials.gov, would facilitate propagation of aggregated results amongst researchers, clinicians as well as study participants. Registry platforms improve transparency by means of a list of studies either in progress or have been completed. (12)

    Today, searching trial registries is regarded as an essential tool while conducting an SLR. Evidence from literature also shows that the addition of data from unpublished trials logged in the registries, such as ClinicalTrials.gov, may change the magnitude of the effect size or, in some cases, the statistical importance of SLRs as well as meta-analyses. It may also help in achieving more precision. Findings of a recent systematic review, estimating the effect of under-reporting of adverse events (AEs) in SLRs, report that the information from such unpublished trials may lessen the inaccuracy of pooled effect estimates while reporting of AEs. (4) Furthermore, information from ClinicalTrials.gov can also aid planning an SLR and offer valuable updates, since the registries enlist not only ongoing, but also soon to be completed trials. (1)

    While searching ClinicalTrials.gov can be helpful in obtaining precise information on unpublished trials, it is not observed to be implemented thoroughly. (4) Information from trial registries can significantly encourage value-addition in SLRs through identification of additional trials. This search should be promoted and applied; while listing of trials on these registries should also be encouraged.

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Potthast R, Vervolgyi V, McGauran N, et al. Impact of Inclusion of Industry Trial Results Registries as an Information Source for Systematic Reviews ncbi. PLoS One 2014; 9(4):e92067. 
    2. Hopewell S, McDonald S, Clarke M, et al. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev 2007; MR000010.
    3. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 2010; 356:e1000326. 
    4. Baudard M, Yavchitz A, Ravaud P, et al. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ 2017; 356:j448.
    5. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009; 356:86-9.
    6. Chan AW, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014; 356:257-66. 
    7. Schmucker C, Schell LK, Portalupi S, et al. OPEN consortium. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One 2014; 356:e114023.
    8. Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol 1986; 4:1529–1541.
    9. De Angelis C, Drazen JM, Frizelle FA, et al. International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med 2004 Sep 16; 351(12):1250–1.
    10. United States Congress. (2007) Food and Drug Administration Amendments Act (FDAAA) of 2007: public law no 110-85.
    11. Groves T. Mandatory disclosure of trial results for drugs and devices. BMJ 2008; 356:170.
    12. Adam GP, Springs S, Trikalinos T, et al. Does information from ClinicalTrials.gov increase transparency and reduce bias? Results from a five-report case series. Syst Rev 2018; 7(1):59.

    Written By: Ms. Tanvi Laghate

    MarksMan Healthcare

    ,

  • What All You Need to Know About PROBAST?

    What All You Need to Know About PROBAST?

    Today’s era of risk based, precision and personalized medicine demands clinical prediction models. Prediction modelling studies focus on two kinds of outcomes, viz. diagnosis (probability of a condition that is undetected) and prognosis (probability of developing a certain outcome in the future). (1,2) These studies develop, validate, or update a multivariable prediction model, wherein multiple predictors are used in combination to estimate probabilities to inform and often guide individual care. Moreover, evidence from literature shows both prognostic as well as diagnostic models being widely used in various medical domains and settings, (3) such as cancer, (4) neurology, (5) and cardiovascular disease. (6) Increasingly common competing prediction models can exist for the same outcome or target population, which necessitate the systematic reviews of these prediction model studies; since their coexistence may facilitate misperceptions amongst health care providers, guideline developers, and policymakers about which model to use or recommend, and in which persons or settings. (1,7)

    Quality assessment is vital while conducting any systematic review, for which several tools are in place that enable the assessment of the risk of bias (ROB). (8) For example, the QUIPS (Quality In Prognosis Studies) tool evaluates the ROB in predictor finding (prognostic factor) studies. (9) Similarly, the revised Cochrane ROB Tool (ROB 2.0) (10) investigates the methodological quality of prediction model impact studies, that use a randomized comparative design, or ROBINS-I (Risk of Bias in Nonrandomized Studies of Interventions) for those incorporating a non-randomized comparative design. (11) Today, prediction model studies as well as their systematic reviews are often being used as evidence for clinical guidance and decision making, which warrants a tool that would facilitate quality assessment for individual prediction model studies. For this purpose, PROBAST (Prediction model Risk Of Bias ASsessment Tool) has been recently introduced. PROBAST came into existence owing to the lack of appropriate tool that would evaluate the ROB for systematic reviews of diagnostic and prognostic prediction model studies. (7,8,12)

    Bias is nothing but a systematic error in a study that leads to inaccurate results, thus inhibiting the study’s internal validity. (8) Similarly, inadequacies of the study design, conduct and analysis may often lead to the distorted estimates of model predictive performance, thus facilitating the ROB to occur. Moreover, different populations, predictors, or outcomes of the study than those specified in the review question may give rise to the concerns regarding the applicability of a primary study. PROBAST has been, therefore, developed to address the lack of suitable tools designed specifically to assess ROB and applicability of primary prediction model studies.

    Development of PROBAST:

    A 4-stage approach for developing health research reporting guidelines was implemented in developing PROBAST. This approach consisted of following stages: 1) defining the scope, 2) reviewing the evidence base, 3) using a Web-based Delphi procedure, and 4) refining the tool through piloting. (8,13) PROBAST was designed mainly to assess primary studies included in a systematic review and not predictor finding or prediction model impact studies. The steering group of 9 experts in prediction model studies and development of quality assessment tools agreed that PROBAST would assess both ROB as well as the concerns regarding applicability of a study evaluating a multivariable prediction model to be used for individualized diagnosis or prognosis. For the first stage, a domain-based structure was adopted to define the scope of PROBAST, similar to that used in other ROB tools, such as ROB 2.0, ROBINS-I, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2), and ROBIS. A total of 3 approaches were used to build an evidence base as part of the second stage, wherein relevant methodological reviews were identified in the area of prediction model research, which was followed by identification of relevant methodological studies by members of the steering group, and lastly, additional evidence was identified with the help of applying the Delphi procedure in a wider group. All this evidence produced an initial list of signalling questions to consider for inclusion in PROBAST. In the third stage, a modified Delphi process, by means of web-based surveys, was used to gain structured feedback and agreement on the scope, structure, and content of PROBAST through 7 rounds. The 38-member Delphi group included methodological experts in prediction model research and development of quality assessment tools, experienced systematic reviewers, commissioners, and representatives of reimbursement agencies. The inclusion of various stakeholders ensured fair representation of the views of end users, methodological experts, and decision makers. In the fourth stage, the then-current version of PROBAST was piloted at multiple workshops at consecutive Cochrane Colloquia as well as numerous workshops with MSc and PhD students. The feedback received was used to further refine the content and structure of PROBAST, wording of the signalling questions, and content of the guidance documents. (7,8)

    PROBAST consists of 4 steps, viz. 1) specifying the systematic review question, 2) classifying the type of prediction model,  3) assessing ROB and applicability and 4) the overall judgement. PROBAST is the first comprehensively developed tool designed explicitly to assess the quality of prediction model studies for development, validation, or updating of both diagnostic and prognostic models, notwithstanding the medical domain, type of outcome, predictors, or statistical technique used. (7,8) PROBAST was introduced earlier this month in two parts; the first publication by Wolf et al.(8) highlights the development and scope of PROBAST, while the second publication by Moons et al. (8) explicitly describes the applications of PROBAST and how to judge ROB and applicability.

    Organizations that support decision making (such as the National Institute for Health and Care Excellence and the Institute for Quality and Efficiency in Health Care); researchers and/or clinicians interested in evidence-based medicine or involved in guideline development; and journal editors, manuscript reviewers are the potential users for PROBAST. (8)

    Become an Certified HEOR Professional – Enrol yourself here!

    References 

    1. Bouwmeester W, Zuithoff NP, Mallett S, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med 2012; 9:1-12.
    2. Steyerberg EW, Moons KG, van der Windt DA, et al; PROGRESS Group. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med 2013; 10:e1001381.
    3. Collins GS, Mallett S, Omar O, et al. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med 2011; 9:103.
    4. Altman DG. Prognostic models: a methodological framework and review of models for breast cancer. Cancer Invest 2009; 27:235-43.
    5. Counsell C, Dennis M. Systematic review of prognostic models in patients with acute stroke. Cerebrovasc Dis 2001; 12:159-70.
    6. Damen JA, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ 2016; 353:i2416.
    7. Moons KGM, Wolf RF, Riley RD, et al. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Ann Intern Med 2019; 170:W1-W33.
    8. Wolf RF, Moons KGM, Riley RD, et al; for the PROBAST Group. PROBAST: A Tool to assess the risk of bias and applicability of prediction model studies. Ann Intern Med 2019; 170:51-58.
    9. Hayden JA, van der Windt DA, Cartwright JL, et al. Assessing bias in studies of prognostic factors. Ann Intern Med 2013; 158:280-6.
    10. Higgins JPT, Savovic´ J, Page MJ, et al. ROB2 Development Group. A revised tool for assessing risk of bias in randomized trials. In: Chandler J, McKenzie J, Boutron I, Welch V, eds. Cochrane Methods. London: Cochrane; 2018:1-69.
    11. Sterne JA, Herna´n MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355:i4919.
    12. PROBAST. Available at: http://www.probast.org/ABOUT
    13. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med 2010; 7:e1000217.

    Written by: Ms. Tanvi Laghate

    MarksMan Healthcare

    , , ,

  • How NMAs are Helping in Taking Informed Clinical Decisions?

    How NMAs are Helping in Taking Informed Clinical Decisions?

    Network meta-analysis (NMA) is a type of meta-analysis that adds an additional variable to a meta-analysis, and instead of a simple summation of trials that have evaluated the same treatment, several different treatments are compared by statistical inference.1 NMA is also referred to as mixed treatments comparison or multiple treatments comparison meta-analysis.2,3,4

    It was recognised at the National Institute for Clinical Excellence (NICE) that there is an increasing need for technology appraisals and clinical guidelines to be informed by integrated analyses, because of the lack of sufficient head to head comparisons of new treatments to inform clinical practice.5 Literature suggests that NMA is a feasible option to inform clinical practice decisions, particularly in cases where several treatments are examined.6,7

    NMA includes a combination of direct evidence within the trials and indirect evidence across the trials, thereby providing estimates of relative efficacy between all the relevant interventions, even in cases where there has never been a head to head comparison.1,2,3 In essence, the treatment effects are calculated for all treatments or interventions using all the available evidence in one simultaneous analysis. 6,8

    NMA relies on two main assumptions; homogeneity of compared trials and consistency in direct and indirect evidence.7,9 A simple example of a NMA would be as follows. A trial compares drug A to drug B and another trial, including the same target patient population, compares drug B to drug C.  Assuming that drug A is superior to drug B in the first trial, and assuming drug B is equivalent to drug C in a second trial, the NMA allows a potential inference that statistically drug A is also superior to drug C for this particular target population.1,4,8 Therefore; one can say that if drug A is more effective than drug B, and drug B is equivalent to drug C, then drug A is also more effective drug C. 1,4,7

    The main advantage of NMA over traditional or pairwise meta-analysis is that it enables some certainty about all treatment comparisons based on the strength of indirect evidence, and it further allows an estimation of the comparative effects, which would not have been examined in parallel group randomized clinical trials.2,4 Overall, NMA potentially enable an assessment of the benefits and harms for more than two interventions for the same clinical condition.

    In terms of limitations with NMA, this type of meta-analysis is more likely to be valid when analysing sufficiently homogenous studies that include very similar patient populations.  As NMA increases the number and type of studies being compared and combined, there is more likelihood of studies getting combined, which are heterogeneous.1,3,9  In addition, the various overlapping meta-analyses with heterogeneous findings may potentially confound the readers and decision makers. Further, NMA from a practical point of view is more complex than the conventional pair-wise meta-analysis, and requires more time and resources. The various assumptions underlying conventional pairwise meta-analyses are well researched and understood; however, the assumptions related to NMA are seen to be more complex, leading to misinterpretations.

    The methodological work to address the limitations of NMAs is an on-going work, and in light of this fact, researchers and end-users should be cautious when interpreting results from NMAs, as inappropriate combination of studies may result in overestimation of treatment effects and therefore misleading results, with some uncertainty in improving patient outcomes! Nevertheless, NMAs are seen as useful tools that are increasingly becoming attractive because they provide a comprehensive framework for decision-making.

    Become an Certified HEOR Professional – Enrol yourself here!

    References

    1. Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013 Jul 16; 159(2):130-7.
    2. Mills EJ, Thorlund K, Ioannidis JP. Demystifying trial networks and network meta-analysis. BMJ. 2013 May 14; 346: f2914.
    3. National Institute for Health and Care Excellence. Guide to the Methods of Technology Appraisal 2013 [Internet]. London: National Institute for Health and Care Excellence (NICE); 2013 Apr. Process and Methods Guides No. 9. NICE Process and Methods Guides. [Viewed on 02/08/2018]
    4. Li T, Puhan MA, Vedula SS, Singh S, Dickersin K; Ad Hoc Network Meta-analysis Methods Meeting Working Group. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med. 2011 Jun 27; 9:79.
    5. Rawlins MD. In pursuit of quality: the National Institute for Clinical Excellence. Lancet. 1999; 353:1079–82.
    6. Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005; 331(7521):897–900.
    7. Tu YK, Faggion CM Jr. A primer on network meta-analysis for dental research. ISRN Dent. 2012; 2012:276520.
    8. Sutton A, Ades AE, Cooper N, Abrams K. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics. 2008; 26(9):753–767.
    9. Donegan S, Williamson P, D’Alessandro U, Tudur Smith C. Assessing key assumptions of network meta-analysis: a review of methods. Res Synth Methods. 2013 Dec; 4(4):291-323.

    Written By – Dr. Sandeep Moola (Research Fellow, The University of Adelaide, Australia)

    MarksMan Healthcare

    , , ,

  • How Rapid Reviews Are Assisting in Healthcare Decision Making?

    How Rapid Reviews Are Assisting in Healthcare Decision Making?

    Traditionally, systematic reviews are considered as the gold standard to inform clinical practice and policy decisions. However, systematic reviews are resource and time intensive. The time factor has been identified as a barrier to implementing results from evidence synthesis, as a result of an incongruence between the time required to produce a full systematic review and the time within which policy and other decision makers must take decisions.1,2 Hence, there is an increasing demand and a rising interest from healthcare decision makers and knowledge users for a summary of high-quality evidence within a short time period to support their practice and policy decision-making.1,3

    Rapid reviews are viewed as valid forms of evidence synthesis products that provide sufficient information and advice to base clinical and policy decisions.3 A rapid review enables the provision of a concise summary of the evidence to answer specific policy or research-related questions. This recent interest in RRs could be due to the relatively large resource, time, and budget demands of well conducted systematic reviews.

    The term ‘rapid review’ (RR) has broad and varied definitions, and it is important to know how it differs from a systematic review. Broadly, a rapid review is a type of evidence synthesis product, which includes components of a systematic review, albeit in a simplified form that enables it’s completion in a timely fashion.3,4

    A rapid review differs from a systematic review in terms of the scope of the review question, comprehensiveness of the search, rigour and/or quality control, and the type of synthesis.3,4,5 Overall, a RR is similar to a systematic review in terms of how the evidence is identified, appraised, selected and synthesised. However, to enable the review to be completed within a short timeframe, certain steps in the systematic review process are altered or skipped or modified or omitted!

    In order to achieve best possible outcomes to facilitate the use of RRs in decision-making and overcome the barriers of lack of timely and relevant research, a range of methods have been developed, which involve modifications to the systematic review methods.1,5,6,7 According to Khangura et al (2012), limiting the scope of RRs or having a more targeted research question is probably the most efficient shortcut because of its impact on the number of articles, including full-texts to be retrieved, screened, assessed and synthesised (including data extraction).5 However, the authors do state that it is important to engage and collaborate closely with the knowledge end-users.1,5,8 Other modifications to the traditional systematic review method include: a reduced list of sources or databases to be searched, including limiting these to specialised sources (e.g. of systematic reviews), or by date, or by language; exclusion of grey literature; relying on existing systematic reviews; full-text review limitation; the use of only one reviewer for study selection and data extraction; providing minimal conclusions or recommendations; and limiting external peer review.1,5,8

    Currently, there is no formal established methodological guidance on conducting and reporting RRs, and there is some recent work that identified a variety of approaches to RRs.4 A project work related to the extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guidelines (for systematic reviews) for the conduct and reporting of RRs is currently underway and registered with the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network.9

    In a scoping review of RRs that compared the results of RRs to full systematic reviews (SRs) in four studies, it was found that the results reported in both the types of reviews were similar, with no incongruence.4 A rapid review in collaboration with clinical experts and/or knowledge end-users is thus a useful tool for assisting clinicians and policy decision-makers to identify evidence-based strategies for implementation into practice and to identify future research priorities.

    Become an Certified HEOR Professional – Enrol yourself here!

    References:

    1. Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an emerging approach to evidence synthesis in health technology assessment. Int J Technol Assess Health Care. 2014; 30(1):20-7.
    2. Featherstone RM, Dryden DM, Foisy M, Guise JM, Mitchell MD, Paynter RA, et al. Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Systems Review. 2015; 4(50).
    3. Munn Z, Lockwood C, Moola S. The Development and Use of Evidence Summaries for Point of Care Information Systems: A Streamlined Rapid Review Approach. Worldviews Evid Based Nurs. 2015 Jun; 12(3):131-8.
    4. Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Medicine. 2015; 13(224).
    5. Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Systems Review. 2012; 1(10).
    6. Polisena J, Garrity C, Kamel C, Stevens A, Abou-Setta AM. Rapid review programs to support health care and policy decision making: a descriptive analysis of processes and methods. Syst Rev. 2015; 4:26.
    7. Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci. 2010; 5:56.
    8. Haby MM, Chapman E, Clark R, Barreto J, Reveiz L, Lavis JN. What are the best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review. Health Res Policy Syst. 2016; 14: 83.
    9. The EQUATOR network 2015, PRISMA-RR 2017: an extension to PRISMA for rapid reviews, Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network, Centre for Statistics in Medicine, NDORMS, University of Oxford, London. 

    Written By – Dr. Sandeep Moola (Research Fellow, The University of Adelaide, Australia)

    MarksMan Healthcare

    , ,