Simulation optimization for Bayesian multi-arm multi-stage clinical trial with binary endpoints.
Ontology highlight
ABSTRACT:
SUBMITTER: Yu Z
PROVIDER: S-EPMC7751063 | biostudies-literature | 2019
REPOSITORIES: biostudies-literature
Publications
Simulation optimization for Bayesian multi-arm multi-stage clinical trial with binary endpoints.
Yu Zhenning Z Ramakrishnan Viswanathan V Meinzer Caitlyn C
Journal of biopharmaceutical statistics 20190214 2
Multi-arm multi-stage designs, in which multiple active treatments are compared to a control and accumulated information from interim data are used to add or remove arms from the trial, may reduce development costs and shorten the drug development timeline. As such, this adaptive update is a natural complement to Bayesian methodology in which the prior clinical belief is sequentially updated using the observed probability of success. Simulation is often required for planning clinical trials to a ...[more]
Similar Datasets
Project description:BackgroundRandomised controlled trials are becoming increasingly costly and time-consuming. In 2011, Royston and colleagues proposed a particular class of multi-arm multi-stage (MAMS) designs intended to speed up the evaluation of new treatments in phase II and III clinical trials. Their design, which controls the type I error rate and power for each pairwise comparison, discontinues randomisation to poorly performing arms at interim analyses if they fail to show a pre-specified level of benefit over the control arm. Arms in which randomisation is continued to the final stage of the trial are compared against the control on a definitive time-to-event outcome measure. To increase efficiency, interim comparisons can be made on an intermediate time-to-event outcome which is on the causal pathway to the definitive outcome.MethodsWe adapt Royston's MAMS design to binary outcomes observed at the end of a fixed follow-up period and analysed using an absolute difference in proportions. We apply the design to tuberculosis (TB), an area where many new drugs are in development, and demonstrate how it can greatly accelerate the evaluation of new TB regimens. We use simulations to support the extensions to the methodology and to investigate the amount of bias in the estimated treatment effects of arms in which randomisation is ceased at the first interim analysis and arms which continue to the final stage of the trial.ResultsThe proposed seamless phase II/III TB trial designs are shown to greatly reduce sample size requirements and trial duration compared to conducting separate phase II and III trials. The bias in the estimated treatment effects for the definitive outcome is shown to be small, especially when treatment selection is based on an intermediate outcome or when a reanalysis is performed at the planned end of the trial after all recruited patients have completed follow-up.ConclusionsThe proposed designs are practical and could be used in a variety of disease areas. They hold considerable promise for speeding up the evaluation of new treatments particularly in TB where many new regimens will soon be available for testing in phase II and phase III trials.
| S-EPMC3840569 | biostudies-literature
Project description:One family of designs that can noticeably improve efficiency in later stages of drug development are multi-arm multi-stage (MAMS) designs. They allow several arms to be studied concurrently and gain efficiency by dropping poorly performing treatment arms during the trial as well as by allowing to stop early for benefit. Conventional MAMS designs were developed for the setting, in which treatment arms are independent and hence can be inefficient when an order in the effects of the arms can be assumed (eg, when considering different treatment durations or different doses). In this work, we extend the MAMS framework to incorporate the order of treatment effects when no parametric dose-response or duration-response model is assumed. The design can identify all promising treatments with high probability. We show that the design provides strong control of the family-wise error rate and illustrate the design in a study of symptomatic asthma. Via simulations we show that the inclusion of the ordering information leads to better decision-making compared to a fixed sample and a MAMS design. Specifically, in the considered settings, reductions in sample size of around 15% were achieved in comparison to a conventional MAMS design.
| S-EPMC7612618 | biostudies-literature
Project description:Multi-arm multi-stage trial designs can bring notable gains in efficiency to the drug development process. However, for normally distributed endpoints, the determination of a design typically depends on the assumption that the patient variance in response is known. In practice, this will not usually be the case. To allow for unknown variance, previous research explored the performance of t-test statistics, coupled with a quantile substitution procedure for modifying the stopping boundaries, at controlling the familywise error-rate to the nominal level. Here, we discuss an alternative method based on Monte Carlo simulation that allows the group size and stopping boundaries of a multi-arm multi-stage t-test to be optimised, according to some nominated optimality criteria. We consider several examples, provide R code for general implementation, and show that our designs confer a familywise error-rate and power close to the desired level. Consequently, this methodology will provide utility in future multi-arm multi-stage trials.
| S-EPMC5886309 | biostudies-literature
Project description:BackgroundIn recent years, the popularity of multi-arm multi-stage, seamless adaptive, and platform trials has increased. However, many design-related questions and questions regarding which operating characteristics should be evaluated to determine the potential performance of a specific trial design remain and are often further complicated by the complexity of such trial designs.MethodsA systematic search was conducted to review existing software for the design of platform trials, whereby multi-arm multi-stage trials were also included. The results of this search are reported both on the literature level and the software level, highlighting the software judged to be particularly useful.ResultsIn recent years, many highly specialized software packages targeting single design elements on platform studies have been released. Only a few of the developed software packages provide extensive design flexibility, at the cost of limited access due to being commercial or not being usable as out-of-the-box solutions.ConclusionsWe believe that both an open-source modular software similar to OCTOPUS and a collaborative effort will be necessary to create software that takes advantage of and investigates the impact of all the flexibility that platform trials potentially provide.
| S-EPMC7931508 | biostudies-literature
Project description:Randomizing patients among treatments with equal probabilities in clinical trials is the established method to obtain unbiased comparisons. In recent years, motivated by ethical considerations, many authors have proposed outcome adaptive randomization, wherein the randomization probabilities are unbalanced, based on interim data, to favor treatment arms having more favorable outcomes. While there has been substantial controversy regarding the merits and flaws of adaptive versus equal randomization, there has not yet been a systematic simulation study in the multi-arm setting. A simulation study was conducted to evaluate four different Bayesian adaptive randomization methods and compare them to equal randomization in five-arm clinical trials. All adaptive randomization methods included an initial burn-in with equal randomization and some combination of other modifications to avoid extreme randomization probabilities. Trials either with or without a control arm were evaluated, using designs that may terminate arms early for futility and select one or more experimental treatments at the end. The designs were evaluated under a range of scenarios and sample sizes. For trials with a control arm and maximum same size 250 or 500, several commonly used adaptive randomization methods have very low probabilities of correctly selecting a truly superior treatment. Of those studied, the only adaptive randomization method with desirable properties has a burn-in with equal randomization and thereafter randomization probabilities restricted to the interval 0.10-0.90. Compared to equal randomization, this method has a favorable sample size imbalance but lower probability of correctly selecting a superior treatment. In multi-arm trials, compared to equal randomization, several commonly used adaptive randomization methods give much lower probabilities of selecting superior treatments. Aside from randomization method, conducting a multi-arm trial without a control arm may lead to very low probabilities of selecting any superior treatments if differences between the treatment success probabilities are small.
| S-EPMC5634533 | biostudies-literature
Project description:BackgroundAdaptive designs offer added flexibility in the execution of clinical trials, including the possibilities of allocating more patients to the treatments that turned out more successful, and early stopping due to either declared success or futility. Commonly applied adaptive designs, such as group sequential methods, are based on the frequentist paradigm and on ideas from statistical significance testing. Interim checks during the trial will have the effect of inflating the Type 1 error rate, or, if this rate is controlled and kept fixed, lowering the power.ResultsThe purpose of the paper is to demonstrate the usefulness of the Bayesian approach in the design and in the actual running of randomized clinical trials during phase II and III. This approach is based on comparing the performance of the different treatment arms in terms of the respective joint posterior probabilities evaluated sequentially from the accruing outcome data, and then taking a control action if such posterior probabilities fall below a pre-specified critical threshold value. Two types of actions are considered: treatment allocation, putting on hold at least temporarily further accrual of patients to a treatment arm, and treatment selection, removing an arm from the trial permanently. The main development in the paper is in terms of binary outcomes, but extensions for handling time-to-event data, including data from vaccine trials, are also discussed. The performance of the proposed methodology is tested in extensive simulation experiments, with numerical results and graphical illustrations documented in a Supplement to the main text. As a companion to this paper, an implementation of the methods is provided in the form of a freely available R package 'barts'.ConclusionThe proposed methods for trial design provide an attractive alternative to their frequentist counterparts.
| S-EPMC8858379 | biostudies-literature
Project description:BackgroundFor medical conditions with numerous interventions worthy of investigation, there are many advantages of a multi-arm multi-stage (MAMS) platform trial approach. However, there is currently limited knowledge on uptake of the MAMS design, especially in the late-phase setting. We sought to examine uptake and characteristics of late-phase MAMS platform trials, to enable better planning for teams considering future use of this approach.DesignWe examined uptake of registered, late-phase MAMS platforms in the EU clinical trials register, Australian New Zealand Clinical Trials Registry, International Standard Randomised Controlled Trial Number registry, Pan African Clinical Trials Registry, WHO International Clinical Trial Registry Platform and databases: PubMed, Medline, Cochrane Library, Global Health Library and EMBASE. Searching was performed and review data frozen on 1 April 2021. MAMS platforms were defined as requiring two or more comparison arms, with two or more trial stages, with an interim analysis allowing for stopping of recruitment to arms and typically the ability to add new intervention arms.Results62 late-phase clinical trials using an MAMS approach were included. Overall, the number of late-phase trials using the MAMS design has been increasing since 2001 and been accelerated by COVID-19. The majority of current MAMS platforms were either targeting infectious diseases (52%) or cancers (29%) and all identified trials were for treatment interventions. 89% (55/62) of MAMS platforms were evaluating medications, with 45% (28/62) of the MAMS platforms having at least one or more repurposed medication as a comparison arm.ConclusionsHistorically, late-phase trials have adhered to long-established standard (two-arm) designs. However, the number of late-phase MAMS platform trials is increasing, across a range of different disease areas. This study highlights the potential scope of MAMS platform trials and may assist research teams considering use of this approach in the late-phase randomised clinical trial setting.Prospero registration numberCRD42019153910.
| S-EPMC8915371 | biostudies-literature
Project description:An increase in the efficiency of clinical trial conduct has been successfully demonstrated in the oncology field, by the use of multi-arm, multi-stage trials allowing the evaluation of multiple therapeutic candidates simultaneously, and seamless recruitment to phase 3 for those candidates passing an interim signal of efficacy. Replicating this complex innovative trial design in diseases such as Parkinson's disease is appealing, but in addition to the challenges associated with any trial assessing a single potentially disease modifying intervention in Parkinson's disease, a multi-arm platform trial must also specifically consider the heterogeneous nature of the disease, alongside the desire to potentially test multiple treatments with different mechanisms of action. In a multi-arm trial, there is a need to appropriately stratify treatment arms to ensure each are comparable with a shared placebo/standard of care arm; however, in Parkinson's disease there may be a preference to enrich an arm with a subgroup of patients that may be most likely to respond to a specific treatment approach. The solution to this conundrum lies in having clearly defined criteria for inclusion in each treatment arm as well as an analysis plan that takes account of predefined subgroups of interest, alongside evaluating the impact of each treatment on the broader population of Parkinson's disease patients. Beyond this, there must be robust processes of treatment selection, and consensus derived measures to confirm target engagement and interim assessments of efficacy, as well as consideration of the infrastructure needed to support recruitment, and the long-term funding and sustainability of the platform. This has to incorporate the diverse priorities of clinicians, triallists, regulatory authorities and above all the views of people with Parkinson's disease.
| S-EPMC10316775 | biostudies-literature
Project description:BackgroundMulti-arm multi-stage trials are an efficient, adaptive approach for testing many treatments simultaneously within one protocol. In settings where numbers of patients available to be entered into trials and resources might be limited, such as primary postpartum haemorrhage, it may be necessary to select a pre-specified subset of arms at interim stages even if they are all showing some promise against the control arm. This will put a limit on the maximum number of patients required and reduce the associated costs. Motivated by the World Health Organization Refractory HaEmorrhage Devices trial in postpartum haemorrhage, we explored the properties of such a selection design in a randomised phase III setting and compared it with other alternatives. The objectives are: (1) to investigate how the timing of treatment selection affects the operating characteristics; (2) to explore the use of an information-rich (continuous) intermediate outcome to select the best-performing arm, out of four treatment arms, compared with using the primary (binary) outcome for selection at the interim stage; and (3) to identify factors that can affect the efficiency of the design.MethodsWe conducted simulations based on the refractory haemorrhage devices multi-arm multi-stage selection trial to investigate the impact of the timing of treatment selection and applying an adaptive allocation ratio on the probability of correct selection, overall power and familywise type I error rate. Simulations were also conducted to explore how other design parameters will affect both the maximum sample size and trial timelines.ResultsThe results indicate that the overall power of the trial is bounded by the probability of 'correct' selection at the selection stage. The results showed that good operating characteristics are achieved if the treatment selection is conducted at around 17% of information time. Our results also showed that although randomising more patients to research arms before selection will increase the probability of selecting correctly, this will not increase the overall efficiency of the (selection) design compared with the fixed allocation ratio of 1:1 to all arms throughout.ConclusionsMulti-arm multi-stage selection designs are efficient and flexible with desirable operating characteristics. We give guidance on many aspects of these designs including selecting the intermediate outcome measure, the timing of treatment selection, and choosing the operating characteristics.
| S-EPMC9940121 | biostudies-literature
Project description:BackgroundMulti-Arm Multi-Stage designs aim at comparing several new treatments to a common reference, in order to select or drop any treatment arm to move forward when such evidence already exists based on interim analyses. We redesigned a Bayesian adaptive design initially proposed for dose-finding, focusing our interest in the comparison of multiple experimental drugs to a control on a binary criterion measure.MethodsWe redesigned a phase II clinical trial that randomly allocates patients across three (one control and two experimental) treatment arms to assess dropping decision rules. We were interested in dropping any arm due to futility, either based on historical control rate (first rule) or comparison across arms (second rule), and in stopping experimental arm due to its ability to reach a sufficient response rate (third rule), using the difference of response probabilities in Bayes binomial trials between the treated and control as a measure of treatment benefit. Simulations were then conducted to investigate the decision operating characteristics under a variety of plausible scenarios, as a function of the decision thresholds.ResultsOur findings suggest that one experimental treatment was less efficient than the control and could have been dropped from the trial based on a sample of approximately 20 instead of 40 patients. In the simulation study, stopping decisions were reached sooner for the first rule than for the second rule, with close mean estimates of response rates and small bias. According to the decision threshold, the mean sample size to detect the required 0.15 absolute benefit ranged from 63 to 70 (rule 3) with false negative rates of less than 2 % (rule 1) up to 6 % (rule 2). In contrast, detecting a 0.15 inferiority in response rates required a sample size ranging on average from 23 to 35 (rules 1 and 2, respectively) with a false positive rate ranging from 3.6 to 0.6 % (rule 3).ConclusionAdaptive trial design is a good way to improve clinical trials. It allows removing ineffective drugs and reducing the trial sample size, while maintaining unbiased estimates. Decision thresholds can be set according to predefined fixed error decision rates.Trial registrationClinicalTrials.gov Identifier: NCT01342692 .
| S-EPMC5399439 | biostudies-literature