Dataset Information

Creation of an ustekinumab external control arm for Crohn's disease using electronic health records data: A pilot study.

ABSTRACT:

Background

Randomized trials are the gold-standard for clinical evidence generation, but they can sometimes be limited by infeasibility and unclear generalizability to real-world practice. External control arm (ECA) studies may help address this evidence gaps by constructing retrospective cohorts that closely emulate prospective ones. Experience in constructing these outside the context of rare diseases or cancer is limited. We piloted an approach for developing an ECA in Crohn's disease using electronic health records (EHR) data.

Methods

We queried EHR databases and manually screened records at the University of California, San Francisco to identify patients meeting the eligibility criteria of TRIDENT, a recently completed interventional trial involving an ustekinumab reference arm. We defined timepoints to balance missing data and bias. We compared imputation models by their impacts on cohort membership and outcomes. We assessed the accuracy of algorithmic data curation against manual review. Lastly, we assessed disease activity following treatment with ustekinumab.

Results

Screening identified 183 patients. 30% of the cohort had missing baseline data. Nonetheless, cohort membership and outcomes were robust to the method of imputation. Algorithms for ascertaining non-symptom-based elements of disease activity using structured data were accurate against manual review. The cohort consisted of 56 patients, exceeding planned enrollment in TRIDENT. 34% of the cohort was in steroid-free remission at week 24.

Conclusion

We piloted an approach for creating an ECA in Crohn's disease from EHR data by using a combination of informatics and manual methods. However, our study reveals significant missing data when standard-of-care clinical data are repurposed. More work will be needed to improve the alignment of trial design with typical patterns of clinical practice, and thereby enable a future of more robust ECAs in chronic diseases like Crohn's disease.

SUBMITTER: Rudrapatna VA

PROVIDER: S-EPMC9980824 | biostudies-literature | 2023

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Creation of an ustekinumab external control arm for Crohn's disease using electronic health records data: A pilot study.

Rudrapatna Vivek A VA Cheng Yao-Wen YW Feuille Colin C Mosenia Arman A Shih Jonathan J Shi Yongmei Y Roberson Olivia O Rubin Benjamin B Butte Atul J AJ Mahadevan Uma U Skomrock Nicholas N Erondu Ngozi N Chehoud Christel C Rahim Saquib S Apfel David D Curran Mark M Khan Najat S NS O'Brien Christopher C Terry Natalie N Martini Benjamin D BD

PloS one 20230302 3

<h4>Background</h4>Randomized trials are the gold-standard for clinical evidence generation, but they can sometimes be limited by infeasibility and unclear generalizability to real-world practice. External control arm (ECA) studies may help address this evidence gaps by constructing retrospective cohorts that closely emulate prospective ones. Experience in constructing these outside the context of rare diseases or cancer is limited. We piloted an approach for developing an ECA in Crohn's disease ...[more]

PMID: 36862717

Similar Datasets

Project description:PurposeExternal control (EC) arms derived from electronic health records (EHRs) can provide appropriate comparison groups when randomized control arms are not feasible, but have not been explored for metastatic colorectal cancer (mCRC) trials. We constructed EC arms from two patient-level EHR-derived databases and evaluated them against the control arm from a phase III, randomized controlled mCRC trial.MethodsIMblaze370 evaluated atezolizumab with or without cobimetinib versus regorafenib in patients with mCRC. EC arms were constructed from the Flatiron Health (FH) EHR-derived de-identified database and the combined FH/Foundation Medicine Clinico-Genomic Database (CGDB). IMblaze370 eligibility criteria were applied to the EC cohorts. Propensity scores and standardized mortality ratio weighting were used to balance baseline characteristics between the IMblaze370 and EC arms; balance was assessed using standardized mean differences. Kaplan-Meier method estimated median overall survival (OS). Cox proportional hazards models estimated hazard ratios with bootstrapped 95% CIs to compare differences in OS between study arms.ResultsThe FH EC included 184 patients; the CGDB EC included 108 patients. Most characteristics were well-balanced (standardized mean difference < 0.1) between each EC arm and the IMblaze370 population. Median OS was similar between the IMblaze370 control arm (8.5 months [95% CI, 6.41 to 10.71]) and both EC arms: FH (8.5 months [6.93 to 9.92]) and CGDB (8.8 months [7.85 to 9.92]). OS comparisons between the IMblaze370 experimental arm and the FH EC (hazard ratio, 0.85 [0.64 to 1.14]) and CGDB EC (0.86 [0.65 to 1.18]) yielded similar results as the comparison with the IMblaze370 control arm (1.01 [0.75 to 1.37]).ConclusionEC arms constructed from the FH database and the CGDB closely replicated the control arm from IMblaze370. EHR-derived EC arms can provide meaningful comparators in mCRC trials when recruiting a randomized control arm is not feasible.

Project description:An external control arm based on health registry data can serve as an alternative comparator in single-arm drug development studies that lack a benchmark for comparison to the experimental treatment. However, accessing such observational healthcare data involves a lengthy and intricate application process, delaying drug approval studies and access to novel treatments. Clinical trials typically comprise only a few hundred patients usually with high-cardinality features, which makes individual data instances more exposed to re-identification attacks. We examine whether synthetic data can serve as a proxy for the empirical control arm data by providing the same research outcomes while reducing the risk of information disclosure. We propose a reversible data generalization procedure to address these particular data characteristics that can be used in conjunction with any generator algorithm. It reduces the input data cardinality pre-synthesis and reverses it post-synthesis to regain the original data structure. Finally, we test a selection of state-of-the-art generators against a suite of utility and privacy metrics. The external control arm benchmark was generated using data from Norwegian health registries. In this retrospective study, we compare various synthetic data generation algorithms in numerical experiments, focusing on the utility of the synthetic data to support the conclusions drawn from the empirical data, and analysing the risk of sensitive information disclosure. Our results indicate that data generalization is advantageous to enhance both data utility and privacy in smaller datasets with high cardinality. Moreover, the generator algorithms demonstrate the ability to generate synthetic data of high utility without compromising the confidentiality of the empirical data. Our finding suggests that synthetic external control arms could serve as a viable alternative to observational data in drug development studies, while reducing the risk of revealing sensitive patient information.

Project description:BackgroundSubject-level real-world data (RWD) collected during daily healthcare practices are increasingly used in medical research to assess questions that cannot be addressed in the context of a randomized controlled trial (RCT). A novel application of RWD arises from the need to create external control arms (ECAs) for single-arm RCTs. In the analysis of ECAs against RCT data, there is an evident need to manage and analyze RCT data and RWD in the same technical environment. In the Nordic countries, legal requirements may require that the original subject-level data be anonymized, i.e., modified so that the risk to identify any individual is minimal. The aim of this study was to conduct initial exploration on how well pseudonymized and anonymized RWD perform in the creation of an ECA for an RCT.MethodsThis was a hybrid observational cohort study using clinical data from the control arm of the completed randomized phase II clinical trial (PACIFIC-AF) and RWD cohort from Finnish healthcare data sources. The initial pseudonymized RWD were anonymized within the (k, ε)-anonymity framework (a model for protecting individuals against identification). Propensity score matching and weighting methods were applied to the anonymized and pseudonymized RWD, to balance potential confounders against the RCT data. Descriptive statistics for the potential confounders and overall survival analyses were conducted prior to and after matching and weighting, using both the pseudonymized and anonymized RWD sets.ResultsAnonymization affected the baseline characteristics of potential confounders only marginally. The greatest difference was in the prevalence of chronic obstructive pulmonary disease (4.6% vs. 5.4% in the pseudonymized compared to the anonymized data, respectively). Moreover, the overall survival changed in anonymization by only 8% (95% CI 4-22%). Both the pseudonymized and anonymized RWD were able to produce matched ECAs for the RCT data. Anonymization after matching impacted overall survival analysis by 22% (95% CI -21-87%).ConclusionsAnonymization may be a viable technique for cases where flexible data transfer and sharing are required. As anonymization necessarily affects some aspects of the original data, further research and careful consideration of anonymization strategies are needed.

Dataset Information

Creation of an ustekinumab external control arm for Crohn's disease using electronic health records data: A pilot study.

Background

Methods

Results

Conclusion

Publications

Creation of an ustekinumab external control arm for Crohn's disease using electronic health records data: A pilot study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets