Dataset Information

An Ensemble Learning Strategy for Eligibility Criteria Text Classification for Clinical Trial Recruitment: Algorithm Development and Validation.

ABSTRACT: BACKGROUND:Eligibility criteria are the main strategy for screening appropriate participants for clinical trials. Automatic analysis of clinical trial eligibility criteria by digital screening, leveraging natural language processing techniques, can improve recruitment efficiency and reduce the costs involved in promoting clinical research. OBJECTIVE:We aimed to create a natural language processing model to automatically classify clinical trial eligibility criteria. METHODS:We proposed a classifier for short text eligibility criteria based on ensemble learning, where a set of pretrained models was integrated. The pretrained models included state-of-the-art deep learning methods for training and classification, including Bidirectional Encoder Representations from Transformers (BERT), XLNet, and A Robustly Optimized BERT Pretraining Approach (RoBERTa). The classification results by the integrated models were combined as new features for training a Light Gradient Boosting Machine (LightGBM) model for eligibility criteria classification. RESULTS:Our proposed method obtained an accuracy of 0.846, a precision of 0.803, and a recall of 0.817 on a standard data set from a shared task of an international conference. The macro F1 value was 0.807, outperforming the state-of-the-art baseline methods on the shared task. CONCLUSIONS:We designed a model for screening short text classification criteria for clinical trials based on multimodel ensemble learning. Through experiments, we concluded that performance was improved significantly with a model ensemble compared to a single model. The introduction of focal loss could reduce the impact of class imbalance to achieve better performance.

SUBMITTER: Zeng K

PROVIDER: S-EPMC7367522 | biostudies-literature | 2020 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An Ensemble Learning Strategy for Eligibility Criteria Text Classification for Clinical Trial Recruitment: Algorithm Development and Validation.

Zeng Kun K Pan Zhiwei Z Xu Yibin Y Qu Yingying Y

JMIR medical informatics 20200701 7

<h4>Background</h4>Eligibility criteria are the main strategy for screening appropriate participants for clinical trials. Automatic analysis of clinical trial eligibility criteria by digital screening, leveraging natural language processing techniques, can improve recruitment efficiency and reduce the costs involved in promoting clinical research.<h4>Objective</h4>We aimed to create a natural language processing model to automatically classify clinical trial eligibility criteria.<h4>Methods</h4> ...[more]

PMID: 32609092

Similar Datasets

Project description:ObjectiveEfficiently identifying eligible patients is a crucial first step for a successful clinical trial. The objective of this study was to test whether an approach using electronic health record (EHR) data and an ensemble machine learning algorithm incorporating billing codes and data from clinical notes processed by natural language processing (NLP) can improve the efficiency of eligibility screening.MethodsWe studied patients screened for a clinical trial of rheumatoid arthritis (RA) with one or more International Classification of Diseases (ICD) code for RA and age greater than 35 years, from a tertiary care center and a community hospital. The following three groups of EHR features were considered for the algorithm: 1) structured features, 2) the counts of NLP concepts from notes, 3) health care utilization. All features were linked to dates. We applied random forest and logistic regression with least absolute shrinkage and selection operator penalty against the following two standard approaches: 1) one or more RA ICD code and no ICD codes related to exclusion criteria (ScreenRAICD1 +EX ) and 2) two or more RA ICD codes (ScreenRAICD2 ). To test the portability, we trained the algorithm at one institution and tested it at the other.ResultsIn total, 3359 patients at Brigham and Women's Hospital (BWH) and 642 patients at Faulkner Hospital (FH) were studied, with 461 (13.7%) eligible patients at BWH and 84 (13.4%) at FH. The application of the algorithm reduced ineligible patients from chart review by 40.5% at the tertiary care center and by 57.0% at the community hospital. In contrast, ScreenRAICD2 reduced patients for chart review by 2.7% to 11.3%; ScreenRAICD1+EX reduced patients for chart review by 63% to 65% but excluded 22% to 27% of eligible patients.ConclusionThe ensemble machine learning algorithm incorporating billing codes and NLP data increased the efficiency of eligibility screening by reducing the number of patients requiring chart review while not excluding eligible patients. Moreover, this approach can be trained at one institution and applied at another for multicenter clinical trials.

Project description:BackgroundEstimating surgical case duration accurately is an important operating room efficiency metric. Current predictive techniques in spine surgery include less sophisticated approaches such as classical multivariable statistical models. Machine learning approaches have been used to predict outcomes such as length of stay and time returning to normal work, but have not been focused on case duration.ObjectiveThe primary objective of this 4-year, single-academic-center, retrospective study was to use an ensemble learning approach that may improve the accuracy of scheduled case duration for spine surgery. The primary outcome measure was case duration.MethodsWe compared machine learning models using surgical and patient features to our institutional method, which used historic averages and surgeon adjustments as needed. We implemented multivariable linear regression, random forest, bagging, and XGBoost (Extreme Gradient Boosting) and calculated the average R2, root-mean-square error (RMSE), explained variance, and mean absolute error (MAE) using k-fold cross-validation. We then used the SHAP (Shapley Additive Explanations) explainer model to determine feature importance.ResultsA total of 3189 patients who underwent spine surgery were included. The institution's current method of predicting case times has a very poor coefficient of determination with actual times (R2=0.213). On k-fold cross-validation, the linear regression model had an explained variance score of 0.345, an R2 of 0.34, an RMSE of 162.84 minutes, and an MAE of 127.22 minutes. Among all models, the XGBoost regressor performed the best with an explained variance score of 0.778, an R2 of 0.770, an RMSE of 92.95 minutes, and an MAE of 44.31 minutes. Based on SHAP analysis of the XGBoost regression, body mass index, spinal fusions, surgical procedure, and number of spine levels involved were the features with the most impact on the model.ConclusionsUsing ensemble learning-based predictive models, specifically XGBoost regression, can improve the accuracy of the estimation of spine surgery times.

Project description:A novel coronavirus (COVID-19) has globally attracted attention as a severe respiratory condition. The epidemic has been first tracked in Wuhan, China, and has progressively been expanded in the entire world. The growing expansion of COVID-19 around the globe has made X-ray images crucial for accelerated diagnostics. Therefore, an effective computerized system must be established as a matter of urgency, to facilitate health care professionals in recognizing X-ray images from COVID-19 patients. In this work, we design a novel artificial intelligent-based automated X-ray image analysis framework based on an ensemble of deep optimized convolutional neural networks (CNNs) in order to distinguish coronavirus patients from non-patients. By developing a modified version of gaining-sharing knowledge (GSK) optimization algorithm using the Opposition-based learning (OBL) and Cauchy mutation operators, the architectures of the deployed deep CNNs are optimized automatically without performing the general trial and error procedures. After obtaining the optimized CNNs, it is also very critical to identify how to decrease the number of ensemble deep CNN classifiers to ensure the classification effectiveness. To this end, a selective ensemble approach is proposed for COVID-19 X-ray based image classification using a deep Q network that combines reinforcement learning (RL) with the optimized CNNs. This approach increases the model performance in particular and therefore decreases the ensemble size of classifiers. The experimental results show that the proposed deep RL optimized ensemble approach has an excellent performance over two popular X-ray image based COVID-19 datasets. Our proposed advanced algorithm can accurately identify the COVID-19 patients from the normal individuals with a significant accuracy of 0.991441, precision of 0.993568, recall (sensitivity) of 0.981445, F-measure of 0.989666 and AUC of 0.990337 for Kaggle dataset as well as an excellent accuracy of 0.987742, precision of 0.984334, recall (sensitivity) of 0.989123, F-measure of 0.984939 and AUC of 0.988466 for Mendely dataset.

Project description:Accurate diagnostic discrimination of benign renal oncocytoma (OC) and malignant renal cell carcinomas (RCC) is not only useful for planning appropriate treatment strategies of patients with renal masses but also for estimating prognosis. Classification of renal neoplasms solely by histopathology can often be challenging for a variety of reasons. The aim of this study was to develop and validate a genomic algorithm for molecular classification of renal cortical neoplasms that could be implemented in a routine clinical diagnostic setting. Using TCGA (The Cancer Genome Atlas) copy number profiles of over 600 RCC specimens, prior FISH studies and published literature, a classification algorithm was developed consisting of 15 genomic markers: loss of VHL, 3p21, 8p, and chromosomes 1, 2, 6, 10 and 17, and gain of 5qter, 16p, 17q, 20q, and chromosomes 3, 7, and 12. Criteria for scoring specimens for the presence of each genomic marker were established. As validation, 191 surgically resected formalin-fixed paraffin-embedded renal neoplasms were blindly submitted to targeted array-CGH and were classified according to the algorithm. Upon histologic re-review leading to exclusion of three specimens and using histology as the gold standard, the algorithm correctly classified 58 of 62 (93%) clear cell renal cell carcinoma, 51 of 56 (91%) papillary RCC, and 33 of 34 (97%) chromophobe RCC. Of the 36 OC specimens, 17 were classified as OC, two as a malignant subtype, 14 as benign, and three exhibited alterations not associated with a specific subtype. In ten of the latter two groups, CCND1-rearrangement was detected by fluorescence in situ hybridization, affording a classification as OC. Together, 33 of 36 (92%) OC were classified as OC or benign. For the entire validation cohort, an overall diagnostic sensitivity of 93% and above 97% specificity was achieved, suggesting that the implementation of genome-based molecular classification in a clinical diagnostic setting could impact the overall management and outcome of patients with renal tumors.

Dataset Information

An Ensemble Learning Strategy for Eligibility Criteria Text Classification for Clinical Trial Recruitment: Algorithm Development and Validation.

Publications

An Ensemble Learning Strategy for Eligibility Criteria Text Classification for Clinical Trial Recruitment: Algorithm Development and Validation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets