Dataset Information

Assessment of a Clinical Trial-Derived Survival Model in Patients With Metastatic Castration-Resistant Prostate Cancer.

ABSTRACT:

Importance

Randomized clinical trials (RCTs) are considered the criterion standard for clinical evidence. Despite their many benefits, RCTs have limitations, such as costliness, that may reduce the generalizability of their findings among diverse populations and routine care settings.

Objective

To assess the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic castration-resistant prostate cancer (CRPC) when the model is applied to real-world data from electronic health records (EHRs).

Design, setting, and participants

The RCT-trained model and patient data from the RCTs were obtained from the Dialogue for Reverse Engineering Assessments and Methods (DREAM) challenge for prostate cancer, which occurred from March 16 to July 27, 2015. This challenge included 4 phase 3 clinical trials of patients with metastatic CRPC. Real-world data were obtained from the EHRs of a tertiary care academic medical center that includes a comprehensive cancer center. In this study, the DREAM challenge RCT-trained model was applied to real-world data from January 1, 2008, to December 31, 2019; the model was then retrained using EHR data with optimized feature selection. Patients with metastatic CRPC were divided into RCT and EHR cohorts based on data source. Data were analyzed from March 23, 2018, to October 22, 2020.

Exposures

Patients who received treatment for metastatic CRPC.

Main outcomes and measures

The primary outcome was the performance of an RCT-derived prognostic model that predicts survival among patients with metastatic CRPC when the model is applied to real-world data. Model performance was compared using 10-fold cross-validation according to time-dependent integrated area under the curve (iAUC) statistics.

Results

Among 2113 participants with metastatic CRPC, 1600 participants were included in the RCT cohort, and 513 participants were included in the EHR cohort. The RCT cohort comprised a larger proportion of White participants (1390 patients [86.9%] vs 337 patients [65.7%]) and a smaller proportion of Hispanic participants (14 patients [0.9%] vs 42 patients [8.2%]), Asian participants (41 patients [2.6%] vs 88 patients [17.2%]), and participants older than 75 years (388 patients [24.3%] vs 191 patients [37.2%]) compared with the EHR cohort. Participants in the RCT cohort also had fewer comorbidities (mean [SD], 1.6 [1.8] comorbidities vs 2.5 [2.6] comorbidities, respectively) compared with those in the EHR cohort. Of the 101 variables used in the RCT-derived model, 10 were not available in the EHR data set, 3 of which were among the top 10 features in the DREAM challenge RCT model. The best-performing EHR-trained model included only 25 of the 101 variables included in the RCT-trained model. The performance of the RCT-trained and EHR-trained models was adequate in the EHR cohort (mean [SD] iAUC, 0.722 [0.118] and 0.762 [0.106], respectively); model optimization was associated with improved performance of the best-performing EHR model (mean [SD] iAUC, 0.792 [0.097]). The EHR-trained model classified 256 patients as having a high risk of mortality and 256 patients as having a low risk of mortality (hazard ratio, 2.7; 95% CI, 2.0-3.7; log-rank P < .001).

Conclusions and relevance

In this study, although the RCT-trained models did not perform well when applied to real-world EHR data, retraining the models using real-world EHR data and optimizing variable selection was beneficial for model performance. As clinical evidence evolves to include more real-world data, both industry and academia will likely search for ways to balance model optimization with generalizability. This study provides a pragmatic approach to applying RCT-trained models to real-world data.

SUBMITTER: Coquet J

PROVIDER: S-EPMC7823224 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ImportanceStandard adverse event (AE) reporting in oncology clinical trials has historically relied on clinician grading, which prior research has shown can lead to underestimation of rates of symptomatic AEs. Industry sponsors are beginning to implement in trials the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE), which was developed to allow patients to self-report symptomatic AEs and improve the quality of symptomatic AE detection.ObjectivesTo evaluate the feasibility of implementing PRO-CTCAE in a prespecified correlative analysis of the phase 3 COMET-2 trial and enumerate statistically significant between-group differences in symptomatic AEs using PRO-CTCAE and the CTCAE.Design, setting, and participantsThis correlative study of 119 men in the randomized, double-blind, placebo-controlled phase 3 COMET-2 trial with metastatic castration-resistant prostate cancer who had undergone at least 2 prior lines of systemic treatment was conducted from March 2012 to July 2014. Participants completed PRO-CTCAE items using an automated telephone system from home prior to treatment and every 3 weeks during treatment. Statistical analysis was performed from May 2018 to June 2019.Main outcomes and measuresThe proportion of patients who completed expected PRO-CTCAE self-reports was computed as a measure of feasibility.ResultsAmong the 119 men in the study (median age, 65 years [range, 44-80 years]), 534 of 587 (91.0%) expected PRO-CTCAE self-reports were completed, with consistently high rates of completion throughout participation. Rates of self-report adherence were similar between groups (cabozantinib s-maleate, 286 of 317 [90.2%]; and mitoxantrone hydrochloride-prednisone, 248 of 270 [91.9%]). Of 12 measured, patient-reported PRO-CTCAE symptomatic AEs, 4 reached statistical significance when comparing the proportion of patients with at least 1 postbaseline score greater than 0 between groups (differences ranged from 20.1% to 34.1% with higher proportions in the cabozantinib group; all P < .05), and use of a method for accounting for preexisting symptoms at baseline yielded 7 AEs with statistically significant differences between groups (differences ranged from 20.5% to 41.2% with higher proportions in the cabozantinib group; all P < .05). In the same analysis using investigator-reported CTCAE data, no statistically significant differences were found between groups for any symptomatic AEs.Conclusions and relevancePRO-CTCAE data collection was feasible and improved the accuracy of symptomatic AE detection in a phase 3 cancer trial. This analysis adds to mounting evidence of the feasibility and value of patient-reported AEs in oncology, which should be considered for inclusion in cancer trials that incorporate AE evaluation.Trial registrationClinicalTrials.gov identifier: NCT01522443.

Project description:Importance:Drug development for metastatic castration-resistant prostate cancer has been limited by a lack of clinically relevant trial end points short of overall survival (OS). Radiographic progression-free survival (rPFS) as defined by the Prostate Cancer Clinical Trials Working Group 2 (PCWG2) is a candidate end point that represents a clinically meaningful benefit to patients. Objective:To demonstrate the robustness of the PCWG2 definition and to examine the relationship between rPFS and OS. Design, Setting, and Participants:PREVAIL was a phase 3, randomized, double-blind, placebo-controlled multinational study that enrolled 1717 chemotherapy-naive men with metastatic castration-resistant prostate cancer from September 2010 through September 2012. The data were analyzed in November 2016. Interventions:Patients were randomized 1:1 to enzalutamide 160 mg or placebo until confirmed radiographic disease progression or a skeletal-related event and initiation of either cytotoxic chemotherapy or an investigational agent for prostate cancer treatment. Main Outcomes and Measures:Sensitivity analyses (SAs) of investigator-assessed rPFS were performed using the final rPFS data cutoff (May 6, 2012; 439 events; SA1) and the interim OS data cutoff (September 16, 2013; 540 events; SA2). Additional SAs using investigator-assessed rPFS from the final rPFS data cutoff assessed the impact of skeletal-related events (SA3), clinical progression (SA4), a confirmatory scan for soft-tissue disease progression (SA5), and all deaths regardless of time after study drug discontinuation (SA6). Correlations between investigator-assessed rPFS (SA2) and OS were calculated using Spearman ? and Kendall ? via Clayton copula. Results:In the 1717 men (mean age, 72.0 [range, 43.0-93.0] years in enzalutamide arm and 71.0 [range, 42.0-93.0] years in placebo arm), enzalutamide significantly reduced risk of radiographic progression or death in all SAs, with hazard ratios of 0.22 (SA1; 95% CI, 0.18-0.27), 0.31 (SA2; 95% CI, 0.27-0.35), 0.21 (SA3; 95% CI, 0.18-0.26), 0.21 (SA4; 95% CI, 0.17-0.26), 0.23 (SA5; 95% CI, 0.19-0.30), and 0.23 (SA6; 95% CI, 0.19-0.30) (P < .001 for all). Correlations of rPFS and OS in enzalutamide-treated patients were 0.89 (95% CI, 0.86-0.92) by Spearman ? and 0.72 (95% CI, 0.68-0.77) by Kendall ?. Conclusions and Relevance:Sensitivity analyses in PREVAIL demonstrated the robustness of the PCWG2 rPFS definition using additional measures of progression. There was concordance between central and investigator review and a positive correlation between rPFS and OS among enzalutamide-treated patients. Trial Registration:clinicaltrials.gov Identifier: NCT01212991.

Project description:BackgroundImprovements to prognostic models in metastatic castration-resistant prostate cancer have the potential to augment clinical trial design and guide treatment strategies. In partnership with Project Data Sphere, a not-for-profit initiative allowing data from cancer clinical trials to be shared broadly with researchers, we designed an open-data, crowdsourced, DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenge to not only identify a better prognostic model for prediction of survival in patients with metastatic castration-resistant prostate cancer but also engage a community of international data scientists to study this disease.MethodsData from the comparator arms of four phase 3 clinical trials in first-line metastatic castration-resistant prostate cancer were obtained from Project Data Sphere, comprising 476 patients treated with docetaxel and prednisone from the ASCENT2 trial, 526 patients treated with docetaxel, prednisone, and placebo in the MAINSAIL trial, 598 patients treated with docetaxel, prednisone or prednisolone, and placebo in the VENICE trial, and 470 patients treated with docetaxel and placebo in the ENTHUSE 33 trial. Datasets consisting of more than 150 clinical variables were curated centrally, including demographics, laboratory values, medical history, lesion sites, and previous treatments. Data from ASCENT2, MAINSAIL, and VENICE were released publicly to be used as training data to predict the outcome of interest-namely, overall survival. Clinical data were also released for ENTHUSE 33, but data for outcome variables (overall survival and event status) were hidden from the challenge participants so that ENTHUSE 33 could be used for independent validation. Methods were evaluated using the integrated time-dependent area under the curve (iAUC). The reference model, based on eight clinical variables and a penalised Cox proportional-hazards model, was used to compare method performance. Further validation was done using data from a fifth trial-ENTHUSE M1-in which 266 patients with metastatic castration-resistant prostate cancer were treated with placebo alone.Findings50 independent methods were developed to predict overall survival and were evaluated through the DREAM challenge. The top performer was based on an ensemble of penalised Cox regression models (ePCR), which uniquely identified predictive interaction effects with immune biomarkers and markers of hepatic and renal function. Overall, ePCR outperformed all other methods (iAUC 0·791; Bayes factor >5) and surpassed the reference model (iAUC 0·743; Bayes factor >20). Both the ePCR model and reference models stratified patients in the ENTHUSE 33 trial into high-risk and low-risk groups with significantly different overall survival (ePCR: hazard ratio 3·32, 95% CI 2·39-4·62, p<0·0001; reference model: 2·56, 1·85-3·53, p<0·0001). The new model was validated further on the ENTHUSE M1 cohort with similarly high performance (iAUC 0·768). Meta-analysis across all methods confirmed previously identified predictive clinical variables and revealed aspartate aminotransferase as an important, albeit previously under-reported, prognostic biomarker.InterpretationNovel prognostic factors were delineated, and the assessment of 50 methods developed by independent international teams establishes a benchmark for development of methods in the future. The results of this effort show that data-sharing, when combined with a crowdsourced challenge, is a robust and powerful framework to develop new prognostic models in advanced prostate cancer.FundingSanofi US Services, Project Data Sphere.