Dataset Information

Developing and maintaining clinical decision support using clinical knowledge and machine learning: the case of order sets.

ABSTRACT: Development and maintenance of order sets is a knowledge-intensive task for off-the-shelf machine-learning algorithms alone. We hypothesize that integrating clinical knowledge with machine learning can facilitate effective development and maintenance of order sets while promoting best practices in ordering. To this end, we simulated the revision of an "AM Lab Order Set" under 6 revision approaches. Revisions included changes in the order set content or default settings through 1) population statistics, 2) individualized prediction using machine learning, and 3) clinical knowledge. Revision criteria were determined using electronic health record (EHR) data from 2014 to 2015. Each revision's clinical appropriateness, workload from using the order set, and generalizability across time were evaluated using EHR data from 2016 and 2017. Our results suggest a potential order set revision approach that jointly leverages clinical knowledge and machine learning to improve usability while updating contents based on latest clinical knowledge and best practices.

SUBMITTER: Zhang Y

PROVIDER: S-EPMC7646906 | biostudies-literature | 2018 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Developing and maintaining clinical decision support using clinical knowledge and machine learning: the case of order sets.

Zhang Yiye Y Trepp Richard R Wang Weiguang W Luna Jorge J Vawdrey David K DK Tiase Victoria V

Journal of the American Medical Informatics Association : JAMIA 20181101 11

Development and maintenance of order sets is a knowledge-intensive task for off-the-shelf machine-learning algorithms alone. We hypothesize that integrating clinical knowledge with machine learning can facilitate effective development and maintenance of order sets while promoting best practices in ordering. To this end, we simulated the revision of an "AM Lab Order Set" under 6 revision approaches. Revisions included changes in the order set content or default settings through 1) population stat ...[more]

PMID: 30101305

Similar Datasets

Project description:IntroductionChronic musculoskeletal pain is a prevalent condition impacting around 20% of people globally; resulting in patients living with pain, fatigue, restricted social and employment capacity, and reduced quality of life. Interdisciplinary multimodal pain treatment programs have been shown to provide positive outcomes by supporting patients modify their behavior and improve pain management through focusing attention on specific patient valued goals rather than fighting pain.MethodsGiven the complex nature of chronic pain there is no single clinical measure to assess outcomes from multimodal pain programs. Using Centre for Integral Rehabilitation data from 2019-2021 (n = 2,364), we developed a multidimensional machine learning framework of 13 outcome measures across 5 clinically relevant domains including activity/disability, pain, fatigue, coping and quality of life. Machine learning models for each endpoint were separately trained using the most important 30 of 55 demographic and baseline variables based on minimum redundancy maximum relevance feature selection. Five-fold cross validation identified best performing algorithms which were rerun on deidentified source data to verify prognostic accuracy.ResultsIndividual algorithm performance ranged from 0.49 to 0.65 AUC reflecting characteristic outcome variation across patients, and unbalanced training data with high positive proportions of up to 86% for some measures. As expected, no single outcome provided a reliable indicator, however the complete set of algorithms established a stratified prognostic patient profile. Patient level validation achieved consistent prognostic assessment of outcomes for 75.3% of the study group (n = 1,953). Clinician review of a sample of predicted negative patients (n = 81) independently confirmed algorithm accuracy and suggests the prognostic profile is potentially valuable for patient selection and goal setting.DiscussionThese results indicate that although no single algorithm was individually conclusive, the complete stratified profile consistently identified patient outcomes. Our predictive profile provides promising positive contribution for clinicians and patients to assist with personalized assessment and goal setting, program engagement and improved patient outcomes.

Project description:ObjectiveTo demonstrate the incremental benefit of using free text data in addition to vital sign and demographic data to identify patients with suspected infection in the emergency department.MethodsThis was a retrospective, observational cohort study performed at a tertiary academic teaching hospital. All consecutive ED patient visits between 12/17/08 and 2/17/13 were included. No patients were excluded. The primary outcome measure was infection diagnosed in the emergency department defined as a patient having an infection related ED ICD-9-CM discharge diagnosis. Patients were randomly allocated to train (64%), validate (20%), and test (16%) data sets. After preprocessing the free text using bigram and negation detection, we built four models to predict infection, incrementally adding vital signs, chief complaint, and free text nursing assessment. We used two different methods to represent free text: a bag of words model and a topic model. We then used a support vector machine to build the prediction model. We calculated the area under the receiver operating characteristic curve to compare the discriminatory power of each model.ResultsA total of 230,936 patient visits were included in the study. Approximately 14% of patients had the primary outcome of diagnosed infection. The area under the ROC curve (AUC) for the vitals model, which used only vital signs and demographic data, was 0.67 for the training data set, 0.67 for the validation data set, and 0.67 (95% CI 0.65-0.69) for the test data set. The AUC for the chief complaint model which also included demographic and vital sign data was 0.84 for the training data set, 0.83 for the validation data set, and 0.83 (95% CI 0.81-0.84) for the test data set. The best performing methods made use of all of the free text. In particular, the AUC for the bag-of-words model was 0.89 for training data set, 0.86 for the validation data set, and 0.86 (95% CI 0.85-0.87) for the test data set. The AUC for the topic model was 0.86 for the training data set, 0.86 for the validation data set, and 0.85 (95% CI 0.84-0.86) for the test data set.ConclusionCompared to previous work that only used structured data such as vital signs and demographic information, utilizing free text drastically improves the discriminatory ability (increase in AUC from 0.67 to 0.86) of identifying infection.

Dataset Information

Developing and maintaining clinical decision support using clinical knowledge and machine learning: the case of order sets.

Publications

Developing and maintaining clinical decision support using clinical knowledge and machine learning: the case of order sets.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets