Dataset Information

Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations.

ABSTRACT: Randomized controlled trials (RCTs) are regarded as the most reputable source of evidence. In some studies, factors beyond the intervention itself may contribute to the measured effect, an occurrence known as heterogeneity of treatment effect (HTE). If the RCT population differs from the real-world population on factors that induce HTE, the trials effect will not replicate. The RCTs eligibility criteria should identify the sub-population in which its evidence will replicate. However, the extent to which the eligibility criteria identify the appropriate population is unknown, which raises concerns for generalizability. We compared reported data from RCTs with real-world data from the electronic health records of a large, academic medical center that was curated according to RCT eligibility criteria. Our results show fundamental differences between the RCT population and our observational cohorts, which suggests that eligibility criteria may be insufficient for identifying the applicable real-world population in which RCT evidence will replicate.

SUBMITTER: Averitt AJ

PROVIDER: S-EPMC7214444 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations.

Averitt Amelia J AJ Weng Chunhua C Ryan Patrick P Perotte Adler A

NPJ digital medicine 20200511

Randomized controlled trials (RCTs) are regarded as the most reputable source of evidence. In some studies, factors beyond the intervention itself may contribute to the measured effect, an occurrence known as heterogeneity of treatment effect (HTE). If the RCT population differs from the real-world population on factors that induce HTE, the trials effect will not replicate. The RCTs eligibility criteria should identify the sub-population in which its evidence will replicate. However, the extent ...[more]

PMID: 32411828

Similar Datasets

Project description:PurposeTo develop and evaluate machine learning models based on MRI to predict clinically significant prostate cancer (csPCa) and International Society of Urological Pathology (ISUP) grade group as well as explore the potential value of radiomics models for improving the performance of radiologists for Prostate Imaging Reporting and Data System (PI-RADS) assessment.Material and methodsA total of 1616 patients from 4 tertiary care medical centers were retrospectively enrolled. PI-RADS assessments were performed by junior, senior, and expert-level radiologists. The radiomics models for predicting csPCa were built using 4 machine-learning algorithms. The PI-RADS were adjusted by the radiomics model. The relationship between the Rad-score and ISUP was evaluated by Spearman analysis.ResultsThe radiomics models made using the random forest algorithm yielded areas under the receiver operating characteristic curves (AUCs) of 0.874, 0.876, and 0.893 in an internal testing cohort and external testing cohorts, respectively. The AUC of the adjusted_PI-RADS was improved, and the specificity was improved at a slight sacrifice of sensitivity. The participant-level correlation showed that the Rad-score was positively correlated with ISUP in all testing cohorts (r > 0.600 and p < 0.0001).ConclusionsThis radiomics model resulted as a powerful, non-invasive auxiliary tool for accurately predicting prostate cancer aggressiveness. The radiomics model could reduce unnecessary biopsies and help improve the diagnostic performance of radiologists' PI-RADS. Yet, prospective studies are still needed to validate the radiomics models further.Critical relevance statementThe radiomics model with MRI may help to accurately screen out clinically significant prostate cancer, thereby assisting physicians in making individualized treatment plans.Key points• The diagnostic performance of the radiomics model using the Random Forest algorithm is comparable to the Prostate Imaging Reporting and Data System (PI-RADS) obtained by radiologists. • The performance of the adjusted Prostate Imaging Reporting and Data System (PI-RADS) was improved, which implied that the radiomics model could be a potential radiological assessment tool. • The radiomics model lowered the percentage of equivocal cases. Moreover, the Rad-scores can be used to characterize prostate cancer aggressiveness.

Project description:BackgroundEven though clinical trials are indispensable for medical research, they are frequently impaired by delayed or incomplete patient recruitment, resulting in cost overruns or aborted studies. Study protocols based on real-world data with precisely expressed eligibility criteria and realistic cohort estimations are crucial for successful study execution. The increasing availability of routine clinical data in electronic health records (EHRs) provides the opportunity to also support patient recruitment during the prescreening phase. While solutions for electronic recruitment support have been published, to our knowledge, no method for the prioritization of eligibility criteria in this context has been explored.MethodsIn the context of the Electronic Health Records for Clinical Research (EHR4CR) project, we examined the eligibility criteria of the KATHERINE trial. Criteria were extracted from the study protocol, deduplicated, and decomposed. A paper chart review and data warehouse query were executed to retrieve clinical data for the resulting set of simplified criteria separately from both sources. Criteria were scored according to disease specificity, data availability, and discriminatory power based on their content and the clinical dataset.ResultsThe study protocol contained 35 eligibility criteria, which after simplification yielded 70 atomic criteria. For a cohort of 106 patients with breast cancer and neoadjuvant treatment, 47.9% of data elements were captured through paper chart review, with the data warehouse query yielding 26.9% of data elements. Score application resulted in a prioritized subset of 17 criteria, which yielded a sensitivity of 1.00 and specificity 0.57 on EHR data (paper charts, 1.00 and 0.80) compared with actual recruitment in the trial.ConclusionIt is possible to prioritize clinical trial eligibility criteria based on real-world data to optimize prescreening of patients on a selected subset of relevant and available criteria and reduce implementation efforts for recruitment support. The performance could be further improved by increasing EHR data coverage.

Dataset Information

Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations.

Publications

Translating evidence into practice: eligibility criteria fail to eliminate clinically significant differences between real-world and study populations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets