Dataset Information

Design and Evaluation of an External Control Arm Using Prior Clinical Trials and Real-World Data.

ABSTRACT: PURPOSE:We discuss designs and interpretable metrics of bias and statistical efficiency of "externally controlled" trials (ECT) and compare ECT performance to randomized and single-arm designs. EXPERIMENTAL DESIGN:We specify an ECT design that leverages information from real-world data (RWD) and prior clinical trials to reduce bias associated with interstudy variations of the enrolled populations. We then used a collection of clinical studies in glioblastoma (GBM) and RWD from patients treated with the current standard of care to evaluate ECTs. Validation is based on a "leave one out" scheme, with iterative selection of a single-arm from one of the studies, for which we estimate treatment effects using the remaining studies as external control. This produces interpretable and robust estimates on ECT bias and type I errors. RESULTS:We developed a model-free approach to evaluate ECTs based on collections of clinical trials and RWD. For GBM, we verified that inflated false positive error rates of standard single-arm trials can be considerably reduced (up to 30%) by using external control data. CONCLUSIONS:The use of ECT designs in GBM, with adjustments for the clinical profiles of the enrolled patients, should be preferred to single-arm studies with fixed efficacy thresholds extracted from published results on the current standard of care.

SUBMITTER: Ventz S

PROVIDER: S-EPMC6697596 | biostudies-literature | 2019 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Design and Evaluation of an External Control Arm Using Prior Clinical Trials and Real-World Data.

Ventz Steffen S Lai Albert A Cloughesy Timothy F TF Wen Patrick Y PY Trippa Lorenzo L Alexander Brian M BM

Clinical cancer research : an official journal of the American Association for Cancer Research 20190607 16

<h4>Purpose</h4>We discuss designs and interpretable metrics of bias and statistical efficiency of "externally controlled" trials (ECT) and compare ECT performance to randomized and single-arm designs.<h4>Experimental design</h4>We specify an ECT design that leverages information from real-world data (RWD) and prior clinical trials to reduce bias associated with interstudy variations of the enrolled populations. We then used a collection of clinical studies in glioblastoma (GBM) and RWD from pat ...[more]

PMID: 31175098

Similar Datasets

Project description:PurposeUsing real-world data (RWD)-based trial simulation approach, we aim to simulate colorectal cancer (CRC) trials and examine both effectiveness and safety end points in different simulation scenarios.MethodsWe identified five phase III trials comparing new treatment regimens with an US Food and Drug Administration-approved first-line treatment in patients with metastatic CRC (ie, fluorouracil, leucovorin, and irinotecan) as the standard-of-care (SOC) control arm. Using Electronic Health Record-derived data from the OneFlorida network, we defined the study populations and outcome measures using the protocols from the original trials. Our design scenarios were (1) simulation of the SOC fluorouracil, leucovorin, and irinotecan arm and (2) comparative effectiveness research (CER) simulation of the control and experimental arms. For each scenario, we adjusted for random assignment, sampling, and dropout. We used overall survival (OS) and severe adverse events (SAEs) to measure effectiveness and safety.ResultsWe conducted CER simulations for two trials, and SOC simulations for three trials. The effect sizes of our simulated trials were stable across all simulation runs. Compared with the original trials, we observed longer OS and higher mean number of SAEs in both CER and SOC simulation. In the two CER simulations, hazard ratios associated with death from simulations were similar to that reported in the original trials. Consistent with the original trials, we found higher risk ratios of SAEs in the experiment arm, suggesting potentially higher toxicities from the new treatment regimen. We also observed similar SAE rates across all simulations compared with the original trials.ConclusionIn this study, we simulated five CRC trials, and tested two simulation scenarios with several different configurations demonstrated that our simulations can robustly generate effectiveness and safety outcomes comparable with the original trials using real-world data.

Project description:One-in-four ophthalmology trials are single-armed, which poses challenges to their interpretation. We demonstrate how real-world cohorts used as external/synthetic control arms can contextualize such trials. We herein emulated a target trial on the intention-to-treat efficacy of off-label bevacizumab (q6w) pro re nata relative to fixed-interval aflibercept (q8w) for improving week 54 visual acuity of eyes affected by neovascular age-related macular degeneration. The bevacizumab arm (n = 65) was taken from the ABC randomized controlled trial. A total of 4,471 aflibercept-treated eyes aligning with the ABC trial eligibility were identified from electronic health records and synthetic control arms were created by emulating randomization conditional on age, sex, and baseline visual read via exact matching and propensity score methods. We undertook an inferiority analysis on mean difference at 54 weeks; outcomes regression on achieving a change in visual acuity of greater than or equal to 15, greater than or equal to 10, and less than or equal to -15 Early Treatment Diabetic Retinopathy (ETDRS) letters at week 54; and a time-to-event analysis on achieving a change in visual acuity of greater than or equal to 15, greater than or equal to 10, and less than or equal to -15 ETDRS letters by week 54. The findings suggest off-label bevacizumab to be neither inferior nor superior to licensed aflibercept. Our study highlights how real-world cohorts representing the counterfactual intervention could aid the interpretation of single-armed trials when analyzed in accord to the target trial framework. Study Highlights WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC? One-in-four randomized controlled trials in ophthalmology are single-armed, which poses challenges for interpreting their efficacy relative to standard of care. Recent conceptual advances in the methods of causal inference and in the emulation of target trials suggests that the standard-of-care arms representing the counterfactual intervention can be approximated with observational data. WHAT QUESTION DID THIS STUDY ADDRESS? How real-world cohorts representing the counterfactual intervention can aid the interpretation of single-armed ophthalmological trials. WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE? Our study highlights how real-world cohorts representing the counterfactual intervention could aid the interpretation of single-armed ophthalmological trials when undertaken in accord with the target trial framework. HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE? External counterfactual arms could reduce the time and cost to reach potential regulatory approval.

Project description:Developing data-driven solutions that address real-world problems requires understanding of these problems' causes and how their interaction affects the outcome-often with only observational data. Causal Bayesian Networks (BN) have been proposed as a powerful method for discovering and representing the causal relationships from observational data as a Directed Acyclic Graph (DAG). BNs could be especially useful for research in global health in Lower and Middle Income Countries, where there is an increasing abundance of observational data that could be harnessed for policy making, program evaluation, and intervention design. However, BNs have not been widely adopted by global health professionals, and in real-world applications, confidence in the results of BNs generally remains inadequate. This is partially due to the inability to validate against some ground truth, as the true DAG is not available. This is especially problematic if a learned DAG conflicts with pre-existing domain doctrine. Here we conceptualize and demonstrate an idea of a "Causal Datasheet" that could approximate and document BN performance expectations for a given dataset, aiming to provide confidence and sample size requirements to practitioners. To generate results for such a Causal Datasheet, a tool was developed which can generate synthetic Bayesian networks and their associated synthetic datasets to mimic real-world datasets. The results given by well-known structure learning algorithms and a novel implementation of the OrderMCMC method using the Quotient Normalized Maximum Likelihood score were recorded. These results were used to populate the Causal Datasheet, and recommendations could be made dependent on whether expected performance met user-defined thresholds. We present our experience in the creation of Causal Datasheets to aid analysis decisions at different stages of the research process. First, one was deployed to help determine the appropriate sample size of a planned study of sexual and reproductive health in Madhya Pradesh, India. Second, a datasheet was created to estimate the performance of an existing maternal health survey we conducted in Uttar Pradesh, India. Third, we validated generated performance estimates and investigated current limitations on the well-known ALARM dataset. Our experience demonstrates the utility of the Causal Datasheet, which can help global health practitioners gain more confidence when applying BNs.

Project description:PurposeWe compared overall survival (OS) in patients with human epidermal growth factor receptor 2 (HER2)-amplified, treatment-refractory metastatic colorectal cancer (mCRC) receiving pertuzumab plus trastuzumab (PER-HER) in the phase IIa MyPathway multibasket study (ClinicalTrials.gov identifier: NCT02091141) with OS in those receiving routine clinical care in an electronic health record-derived external control arm.MethodsA noninterventional study was conducted using patient-level data from MyPathway participants receiving PER-HER and real-world patients with HER2-amplified treatment-refractory mCRC receiving routine clinical care. This study used a deidentified US-based clinico-genomic database (CGDB). For patients in the CGDB who met study eligibility criteria at multiple index dates (treatment initiation dates in the treatment-refractory setting), all eligible index dates were used for the analysis. Standardized mortality ratio weighting on the basis of propensity score derived a pseudopopulation (postweighting population) balancing key prognostic variables between arms. Multivariate Cox proportional hazards models were used for estimation of the hazard ratio (HR) in the primary OS analysis. A series of sensitivity analyses were conducted to investigate the robustness and consistency of the primary analysis.ResultsThe PER-HER arm comprised 57 patients enrolled in the MyPathway study by August 1, 2017 (data cutoff); the external control arm comprised 18 patients (27 index dates) with HER2-amplified mCRC who met the major MyPathway eligibility criteria in CGDB collected between 2011 and 2019. The estimated HR for OS from the multivariate Cox proportional hazards model in the postweighting population was 0.729 (95% CI, 0.184 to 3.900). The results of sensitivity analyses were consistent with the primary analysis in terms of the point estimate of HR.ConclusionDespite a small sample size, these findings suggest that PER-HER could have a potential OS benefit for this population.

Dataset Information

Design and Evaluation of an External Control Arm Using Prior Clinical Trials and Real-World Data.

Publications

Design and Evaluation of an External Control Arm Using Prior Clinical Trials and Real-World Data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets