Dataset Information

Automated detection of altered mental status in emergency department clinical notes: a deep learning approach.

ABSTRACT: BACKGROUND:Machine learning has been used extensively in clinical text classification tasks. Deep learning approaches using word embeddings have been recently gaining momentum in biomedical applications. In an effort to automate the identification of altered mental status (AMS) in emergency department provider notes for the purpose of decision support, we compare the performance of classic bag-of-words-based machine learning classifiers and novel deep learning approaches. METHODS:We used a case-control study design to extract an adequate number of clinical notes with AMS and non-AMS based on ICD codes. The notes were parsed to extract the history of present illness, which was used as the clinical text for the classifiers. The notes were manually labeled by clinicians. As a baseline for comparison, we tested several traditional bag-of-words based classifiers. We then tested several deep learning models using a convolutional neural network architecture with three different types of word embeddings, a pre-trained word2vec model and two models without pre-training but with different word embedding dimensions. RESULTS:We evaluated the models on 1130 labeled notes from the emergency department. The deep learning models had the best overall performance with an area under the ROC curve of 98.5% and an accuracy of 94.5%. Pre-training word embeddings on the unlabeled corpus reduced training iterations and had performance that was statistically no different than the other deep learning models. CONCLUSION:This supervised deep learning approach performs exceedingly well for the detection of AMS symptoms in clinical text in our environment. Further work is needed for the generalizability of these findings, including evaluation of these models in other types of clinical notes and other environments. The results seem promising for the ultimate use of these types of classifiers in combination with other information derived from the electronic health records as input for clinical decision support.

SUBMITTER: Obeid JS

PROVIDER: S-EPMC6701023 | biostudies-literature | 2019 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automated detection of altered mental status in emergency department clinical notes: a deep learning approach.

Obeid Jihad S JS Weeda Erin R ER Matuskowitz Andrew J AJ Gagnon Kevin K Crawford Tami T Carr Christine M CM Frey Lewis J LJ

BMC medical informatics and decision making 20190819 1

<h4>Background</h4>Machine learning has been used extensively in clinical text classification tasks. Deep learning approaches using word embeddings have been recently gaining momentum in biomedical applications. In an effort to automate the identification of altered mental status (AMS) in emergency department provider notes for the purpose of decision support, we compare the performance of classic bag-of-words-based machine learning classifiers and novel deep learning approaches.<h4>Methods</h4> ...[more]

PMID: 31426779

Similar Datasets

Project description:BackgroundThe proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data.MethodsThis study used EHR data for children and youth aged 4-17 seeking services at McMaster Children's Hospital's Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores.ResultsThe GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value.ConclusionsThis study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth.

Project description:BackgroundWhen a paucity of clinical information is communicated from ordering physicians to radiologists at the time of radiology order entry, suboptimal imaging interpretations and patient care may result.ObjectivesCompare documentation of relevant clinical information in electronic health record (EHR) provider note to computed tomography (CT) order requisition, prior to ordering of head CT for emergency department (ED) patients presenting with headache.MethodsIn this institutional review board-approved retrospective observational study performed between April 1, 2013 and September 30, 2014 at an adult quaternary academic hospital, we reviewed data from 666 consecutive ED encounters for patients with headaches who received head CT. The primary outcome was the number of concept unique identifiers (CUIs) relating to headache extracted via ontology-based natural language processing from the history of present illness (HPI) section in ED notes compared with the number of concepts obtained from the imaging order requisition.ResultsOur analysis was conducted on cases where the HPI note section was completed prior to image order entry, which occurred in 23.1% (154/666) of encounters. For these 154 encounters, the number of CUIs specific to headache per note extracted from the HPI (median = 3, interquartile range [IQR]: 2-4) was significantly greater than the number of CUIs per encounter obtained from the imaging order requisition (median = 1, IQR: 1-2; Wilcoxon signed rank p < 0.0001). Extracted concepts from notes were distinct from order requisition indications in 92.9% (143/154) of cases.ConclusionEHR provider notes are a valuable source of relevant clinical information at the time of imaging test ordering. Automated extraction of clinical information from notes to prepopulate imaging order requisitions may improve communication between ordering physicians and radiologists, enhance efficiency of ordering process by reducing redundant data entry, and may help improve clinical relevance of clinical decision support at the time of order entry, potentially reducing provider burnout from extraneous alerts.

Project description:ObjectivesAltered mental status (AMS) is a common presentation in the emergency department (ED). A previous study revealed 78% electroencephalogram (EEG) abnormalities, including nonconvulsive seizure (NCS; 5%), in ED patients with AMS. The objective of this study was to assess the impact of EEG on clinical management and outcomes of ED patients with AMS.MethodsThis was a randomized controlled trial at two urban teaching hospitals. Adult patients (≥18 years old) with AMS were included. Excluded patients had immediately correctable AMS (e.g., hypoglycemia) or were admitted before enrollment. Patients were randomized to routine care (control) or routine care plus EEG (intervention). Research assistants used a scalp electrode set with a miniature, wireless EEG device (microEEG) to record standard 30-minute EEGs at presentation, and results were reported to the ED attending physician by an off-site epileptologist within 30 minutes. Primary outcomes included changes in ED management (differential diagnosis, diagnostic work-up, and treatment plan from enrollment to disposition) as determined by surveying the treating physicians. Secondary outcomes were length of ED and hospital stay, intensive care unit (ICU) requirement, and in-hospital mortality.ResultsA total of 149 patients were enrolled (76 control and 73 intervention). Patients in the two groups were comparable at baseline. EEG in the intervention group revealed abnormal findings in 93% (95% confidence interval [CI] = 85% to 97%), including NCS in 5% (95% CI = 2% to 13%). Using microEEG was associated with change in diagnostic work-up in 49% (95% CI = 38% to 60%) of cases and therapeutic plan in 42% (95% CI = 31% to 53%) of cases immediately after the release of EEG results. Changes in probabilities of differential diagnoses and the secondary outcomes were not statistically significant between the groups.ConclusionsAn EEG can be obtained in the ED with minimal resources and can affect clinical management of AMS patients.

Project description:BackgroundSuicide is an important public health concern in the United States and around the world. There has been significant work examining machine learning approaches to identify and predict intentional self-harm and suicide using existing data sets. With recent advances in computing, deep learning applications in health care are gaining momentum.ObjectiveThis study aimed to leverage the information in clinical notes using deep neural networks (DNNs) to (1) improve the identification of patients treated for intentional self-harm and (2) predict future self-harm events.MethodsWe extracted clinical text notes from electronic health records (EHRs) of 835 patients with International Classification of Diseases (ICD) codes for intentional self-harm and 1670 matched controls who never had any intentional self-harm ICD codes. The data were divided into training and holdout test sets. We tested a number of algorithms on clinical notes associated with the intentional self-harm codes using the training set, including several traditional bag-of-words-based models and 2 DNN models: a convolutional neural network (CNN) and a long short-term memory model. We also evaluated the predictive performance of the DNNs on a subset of patients who had clinical notes 1 to 6 months before the first intentional self-harm event. Finally, we evaluated the impact of a pretrained model using Word2vec (W2V) on performance.ResultsThe area under the receiver operating characteristic curve (AUC) for the CNN on the phenotyping task, that is, the detection of intentional self-harm in clinical notes concurrent with the events was 0.999, with an F1 score of 0.985. In the predictive task, the CNN achieved the highest performance with an AUC of 0.882 and an F1 score of 0.769. Although pretraining with W2V shortened the DNN training time, it did not improve performance.ConclusionsThe strong performance on the first task, namely, phenotyping based on clinical notes, suggests that such models could be used effectively for surveillance of intentional self-harm in clinical text in an EHR. The modest performance on the predictive task notwithstanding, the results using DNN models on clinical text alone are competitive with other reports in the literature using risk factors from structured EHR data.

Project description:ObjectiveTo demonstrate the incremental benefit of using free text data in addition to vital sign and demographic data to identify patients with suspected infection in the emergency department.MethodsThis was a retrospective, observational cohort study performed at a tertiary academic teaching hospital. All consecutive ED patient visits between 12/17/08 and 2/17/13 were included. No patients were excluded. The primary outcome measure was infection diagnosed in the emergency department defined as a patient having an infection related ED ICD-9-CM discharge diagnosis. Patients were randomly allocated to train (64%), validate (20%), and test (16%) data sets. After preprocessing the free text using bigram and negation detection, we built four models to predict infection, incrementally adding vital signs, chief complaint, and free text nursing assessment. We used two different methods to represent free text: a bag of words model and a topic model. We then used a support vector machine to build the prediction model. We calculated the area under the receiver operating characteristic curve to compare the discriminatory power of each model.ResultsA total of 230,936 patient visits were included in the study. Approximately 14% of patients had the primary outcome of diagnosed infection. The area under the ROC curve (AUC) for the vitals model, which used only vital signs and demographic data, was 0.67 for the training data set, 0.67 for the validation data set, and 0.67 (95% CI 0.65-0.69) for the test data set. The AUC for the chief complaint model which also included demographic and vital sign data was 0.84 for the training data set, 0.83 for the validation data set, and 0.83 (95% CI 0.81-0.84) for the test data set. The best performing methods made use of all of the free text. In particular, the AUC for the bag-of-words model was 0.89 for training data set, 0.86 for the validation data set, and 0.86 (95% CI 0.85-0.87) for the test data set. The AUC for the topic model was 0.86 for the training data set, 0.86 for the validation data set, and 0.85 (95% CI 0.84-0.86) for the test data set.ConclusionCompared to previous work that only used structured data such as vital signs and demographic information, utilizing free text drastically improves the discriminatory ability (increase in AUC from 0.67 to 0.86) of identifying infection.

Project description:Study objectiveTriage quickly identifies critically ill patients, facilitating timely interventions. Many emergency departments (EDs) use emergency severity index (ESI) or abnormal vital sign triggers to guide triage. However, both use fixed thresholds, and false activations are costly. Prior approaches using machinelearning have relied on information that is often unavailable during the triage process. We examined whether deep-learning approaches could identify critically ill patients only using data immediately available at triage.MethodsWe conducted a retrospective, cross-sectional study at an urban tertiary care center, from January 1, 2012-January 1, 2020. De-identified triage information included structured (age, sex, initial vital signs) and textual (chief complaint) data, with critical illness (mortality or ICU admission within 24 hours) as the outcome. Four progressively complex deep-learning models were trained and applied to triage information from all patients. We compared the accuracy of the models against ESI as the standard diagnostic test, using area under the receiver-operator curve (AUC).ResultsA total of 445,925 patients were included, with 60,901 (13.7%) critically ill. Vital sign thresholds identified critically ill patients with AUC 0.521 (95% confidence interval [CI] = 0.519-0.522), and ESI <3 demonstrated AUC 0.672 (95% CI = 0.671-0.674), logistic regression classified patients with AUC 0.803 (95% CI = 0.802-0.804), 2-layer neural network with structured data with AUC 0.811 (95% CI = 0.807-0.815), gradient tree boosting with AUC 0.820 (95% CI = 0.818-0.821), and the neural network model with textual data with AUC 0.851 (95% CI = 0.849-0.852). All successive increases in AUC were statistically significant.ConclusionDeep-learning techniques represent a promising method of augmenting triage, even with limited information. Further research is needed to determine if improved predictions yield clinical and operational benefits.

Project description:The neurological ailment known as Parkinson's disease (PD) affects people throughout the globe. The neurodegenerative PD-related disorder primarily affects people in middle to late life. Motor symptoms such as tremors, muscle rigidity, and sluggish, clumsy movement are common in patients with this disorder. Genetic and environmental variables play significant roles in the development of PD. Despite much investigation, the root cause of this neurodegenerative disease is still unidentified. Clinical diagnostics rely heavily on promptly detecting such irregularities to slow or stop the progression of illnesses successfully. Because of its direct correlation with brain activity, electroencephalography (EEG) is an essential PD diagnostic technique. Electroencephalography, or EEG, data are biomarkers of brain activity changes. However, these signals are non-linear, non-stationary, and complicated, making analysis difficult. One must often resort to a lengthy human labor process to accomplish results using traditional machine-learning approaches. The breakdown, feature extraction, and classification processes are typical examples of these stages. To overcome these obstacles, we present a novel deep-learning model for the automated identification of Parkinson's disease (PD). The Gabor transform, a standard method in EEG signal processing, was used to turn the raw data from the EEG recordings into spectrograms. In this research, we propose densely linked bidirectional long short-term memory (DLBLSTM), which first represents each layer as the sum of its hidden state plus the hidden states of all layers above it, then recursively transmits that representation to all layers below it. This study's suggested deep learning model was trained using these spectrograms as input data. Using a robust sixfold cross-validation method, the proposed model showed excellent accuracy with a classification accuracy of 99.6%. The results indicate that the suggested algorithm can automatically identify PD.

Dataset Information

Automated detection of altered mental status in emergency department clinical notes: a deep learning approach.

Publications

Automated detection of altered mental status in emergency department clinical notes: a deep learning approach.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets