Dataset Information

Emulating a Novel Clinical Trial Using Existing Observational Data. Predicting Results of the PreVent Study.

ABSTRACT: Rationale: "Target trial emulation" has been proposed as an observational method to answer comparative effectiveness questions, but it has rarely been attempted concurrently with a randomized clinical trial (RCT).Objectives: We tested the hypothesis that blinded analysts applying target trial emulation to existing observational data could predict the results of an RCT.Methods: PreVent (Preventing Hypoxemia with Manual Ventilation during Endotracheal Intubation) was a multicenter RCT examining the effects of positive-pressure ventilation during tracheal intubation on oxygen saturation and severe hypoxemia. Analysts unaware of PreVent's results used patient-level data from three previous trials evaluating airway management interventions to emulate PreVent's eligibility criteria, randomization procedure, and statistical analysis. After PreVent's release, results of this blinded observational analysis were compared with those of the RCT. Difference-in-differences estimates for comparison of treatment effects between the observational analysis and the PreVent trial are reported on the absolute scale.Results: Using observational data, we were able to emulate PreVent's randomization procedure to produce balanced groups for comparison. The lowest oxygen saturation during intubation was higher in the positive-pressure ventilation group than the no positive-pressure ventilation group in the observational analysis (n = 360; mean difference = 1.8%; 95% confidence interval [CI] = -1.0 to 4.6) and in the PreVent trial (n = 401; mean difference = 3.9%; 95% CI = 1.4 to 6.4), though the observational analysis could not exclude no difference. Difference-in-differences estimates comparing treatment effects showed reasonable agreement for lowest oxygen saturation between the observational analysis and the PreVent trial (mean difference = -2.1%; 95% CI = -5.9 to 1.7). Positive-pressure ventilation resulted in lower rates of severe hypoxemia in both the observational analysis (risk ratio = 0.60; 95% CI = 0.38 to 0.93) and in the PreVent trial (risk ratio = 0.48; 95% CI = 0.30 to 0.77). The absolute reduction in the incidence of severe hypoxemia with positive-pressure ventilation was similar in the observational analysis (9.4%) and the PreVent trial (12.0%), though the difference between these estimates had wide CIs (mean difference = 2.5%; 95% CI = -8.0 to 13.6%).Conclusions: Applying target trial emulation methods to existing observational data for the evaluation of a novel intervention produced results similar to those of a randomized trial. These findings support the use of target trial emulation for comparative effectiveness research.

SUBMITTER: Admon AJ

PROVIDER: S-EPMC6774748 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMany dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records.MethodsWe identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters' Dementia Risk Score, Mehta's RxDx-Dementia Risk Index, and Nori's ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models.ResultsWe reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67-0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69-0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62-0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard.ConclusionWe observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.

Project description:PURPOSE: Validated biomarkers predictive of response/resistance to anthracyclines in breast cancer are currently lacking. The neoadjuvant TOP trial, in which patients with estrogen receptor (ER)-negative tumors were treated with anthracycline (epirubicin) monotherapy, was specifically designed to evaluate the predictive value of topoisomerase IIα (TOP2A) and to develop a gene expression signature to identify those patients who do not benefit from anthracyclines. METHODS: The TOP trial included 149 patients, of which 141 were evaluable for response prediction analyses. The primary endpoint was pathological complete response (pCR). TOP2A and gene expression profiles were evaluated using pre-epirubicin biopsies. Gene expression data from ER-negative samples of the EORTC 10994/BIG 00-01 and MDACC 2003-0321 neoadjuvant trials were used for validation purposes. RESULTS: A pCR was obtained in 14% of the evaluable TOP patients. TOP2A amplification, but not protein overexpression, was significantly associated with pCR (p=0.001 and 0.22). We developed an “anthracycline-based score (A-Score)” that combines three signatures: a TOP2A gene signature and two previously published signatures related to tumor invasion and immune response. The A-Score was characterized by a high negative predictive value (NPV=0.98 [95% CI: 0.90-1.00]) overall, and in the HER2-negative and HER2-positive subpopulations. Its performance was independently confirmed in the anthracycline-based (FAC/FEC) arms of the two validation trials (BIG 00-01: 0.80 [0.61-0.92] and MDACC 2003-0321: 1.00 [0.80-1.00]). CONCLUSION: Given its high NPV, the A-Score could become, if further validated, a useful clinical tool to identify those patients who do not benefit from anthracyclines and could therefore be spared the non-negligible side effects. Predicting the efficacy of anthracyclines (epirubicin) in breast cancer (BC) patients (TOP trial) 120 microarray experiments from primary ER-negative breast tumors of anthracycline-treated patients. No replicate, no reference sample.

Dataset Information

Emulating a Novel Clinical Trial Using Existing Observational Data. Predicting Results of the PreVent Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets