Dataset Information

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review.

ABSTRACT: BACKGROUND:Decision support systems based on reinforcement learning (RL) have been implemented to facilitate the delivery of personalized care. This paper aimed to provide a comprehensive review of RL applications in the critical care setting. OBJECTIVE:This review aimed to survey the literature on RL applications for clinical decision support in critical care and to provide insight into the challenges of applying various RL models. METHODS:We performed an extensive search of the following databases: PubMed, Google Scholar, Institute of Electrical and Electronics Engineers (IEEE), ScienceDirect, Web of Science, Medical Literature Analysis and Retrieval System Online (MEDLINE), and Excerpta Medica Database (EMBASE). Studies published over the past 10 years (2010-2019) that have applied RL for critical care were included. RESULTS:We included 21 papers and found that RL has been used to optimize the choice of medications, drug dosing, and timing of interventions and to target personalized laboratory values. We further compared and contrasted the design of the RL models and the evaluation metrics for each application. CONCLUSIONS:RL has great potential for enhancing decision making in critical care. Challenges regarding RL system design, evaluation metrics, and model choice exist. More importantly, further work is required to validate RL in authentic clinical environments.

SUBMITTER: Liu S

PROVIDER: S-EPMC7400046 | biostudies-literature | 2020 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review.

Liu Siqi S See Kay Choong KC Ngiam Kee Yuan KY Celi Leo Anthony LA Sun Xingzhi X Feng Mengling M

Journal of medical Internet research 20200720 7

<h4>Background</h4>Decision support systems based on reinforcement learning (RL) have been implemented to facilitate the delivery of personalized care. This paper aimed to provide a comprehensive review of RL applications in the critical care setting.<h4>Objective</h4>This review aimed to survey the literature on RL applications for clinical decision support in critical care and to provide insight into the challenges of applying various RL models.<h4>Methods</h4>We performed an extensive search ...[more]

PMID: 32706670

Similar Datasets

Project description:Subtle differences in a patient's genetics and physiology may alter radiotherapy (RT) treatment responses, motivating the need for a more personalized treatment plan. Accordingly, we have developed a novel quantum deep reinforcement learning (qDRL) framework for clinical decision support that can estimate an individual patient's dose response mid-treatment and recommend an optimal dose adjustment. Our framework considers patients' specific information including biological, physical, genetic, clinical, and dosimetric factors. Recognizing that physicians must make decisions amidst uncertainty in RT treatment outcomes, we employed indeterministic quantum states to represent human decision making in a real-life scenario. We paired quantum decision states with a model-based deep q-learning algorithm to optimize the clinical decision-making process in RT. We trained our proposed qDRL framework on an institutional dataset of 67 stage III non-small cell lung cancer (NSCLC) patients treated on prospective adaptive protocols and independently validated our framework in an external multi-institutional dataset of 174 NSCLC patients. For a comprehensive evaluation, we compared three frameworks: DRL, qDRL trained in a Qiskit quantum computing simulator, and qDRL trained in an IBM quantum computer. Two metrics were considered to evaluate our framework: (1) similarity score, defined as the root mean square error between retrospective clinical decisions and the AI recommendations, and (2) self-evaluation scheme that compares retrospective clinical decisions and AI recommendations based on the improvement in the observed clinical outcomes. Our analysis shows that our framework, which takes into consideration individual patient dose response in its decision-making, can potentially improve clinical RT decision-making by at least about 10% compared to unaided clinical practice. Further validation of our novel quantitative approach in a prospective study will provide a necessary framework for improving the standard of care in personalized RT.

Project description:The emergency department (ED) is a fast-paced environment responsible for large volumes of patients with varied disease acuity. Operational pressures on EDs are increasing, which creates the imperative to efficiently identify patients at imminent risk of acute deterioration. The aim of this study is to systematically compare the performance of machine learning algorithms based on logistic regression, gradient boosted decision trees, and support vector machines for predicting imminent clinical deterioration for patients based on cross-sectional patient data extracted from electronic patient records (EPR) at the point of entry to the hospital. We apply state-of-the-art machine learning methods to predict early patient deterioration, based on their first recorded vital signs, observations, laboratory results, and other predictors documented in the EPR. Clinical deterioration in this study is measured by in-hospital mortality and/or admission to critical care. We build on prior work by incorporating interpretable machine learning and fairness-aware modelling, and use a dataset comprising 118, 886 unplanned admissions to Salford Royal Hospital, UK, to systematically compare model variations for predicting mortality and critical care utilisation within 24 hours of admission. We compare model performance to the National Early Warning Score 2 (NEWS2) and yield up to a 0.366 increase in average precision, up to a [Formula: see text] reduction in daily alert rate, and a median 0.599 reduction in differential bias amplification across the protected demographics of age and sex. We use Shapely Additive exPlanations to justify the models' outputs, verify that the captured data associations align with domain knowledge, and pair predictions with the causal context of each patient's most influential characteristics. Introducing our modelling to clinical practice has the potential to reduce alert fatigue and identify high-risk patients with a lower NEWS2 that might be missed currently, but further work is needed to trial the models in clinical practice. We encourage future research to follow a systematised approach to data-driven risk modelling to obtain clinically applicable support tools.

Project description:BackgroundSurgical patients incur preventable harm from cognitive and judgment errors made under time constraints and uncertainty regarding patients' diagnoses and predicted response to treatment. Decision analysis and techniques of reinforcement learning theoretically can mitigate these challenges but are poorly understood and rarely used clinically. This review seeks to promote an understanding of decision analysis and reinforcement learning by describing their use in the context of surgical decision-making.MethodsCochrane, EMBASE, and PubMed databases were searched from their inception to June 2019. Included were 41 articles about cognitive and diagnostic errors, decision-making, decision analysis, and machine-learning. The articles were assimilated into relevant categories according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines.ResultsRequirements for time-consuming manual data entry and crude representations of individual patients and clinical context compromise many traditional decision-support tools. Decision analysis methods for calculating probability thresholds can inform population-based recommendations that jointly consider risks, benefits, costs, and patient values but lack precision for individual patient-centered decisions. Reinforcement learning, a machine-learning method that mimics human learning, can use a large set of patient-specific input data to identify actions yielding the greatest probability of achieving a goal. This methodology follows a sequence of events with uncertain conditions, offering potential advantages for personalized, patient-centered decision-making. Clinical application would require secure integration of multiple data sources and attention to ethical considerations regarding liability for errors and individual patient preferences.ConclusionTraditional decision-support tools are ill-equipped to accommodate time constraints and uncertainty regarding diagnoses and the predicted response to treatment, both of which often impair surgical decision-making. Decision analysis and reinforcement learning have the potential to play complementary roles in delivering high-value surgical care through sound judgment and optimal decision-making.

Project description:Background: Clinical decision support (CDS) systems have emerged as tools providing intelligent decision making to address challenges of critical care. CDS systems can be based on existing guidelines or best practices; and can also utilize machine learning to provide a diagnosis, recommendation, or therapy course. Methods: This research aimed to identify evidence-based study designs and outcome measures to determine the clinical effectiveness of clinical decision support systems in the detection and prediction of hemodynamic instability, respiratory distress, and infection within critical care settings. PubMed, ClinicalTrials.gov and Cochrane Database of Systematic Reviews were systematically searched to identify primary research published in English between 2013 and 2018. Studies conducted in the USA, Canada, UK, Germany and France with more than 10 participants per arm were included. Results: In studies on hemodynamic instability, the prediction and management of septic shock were the most researched topics followed by the early prediction of heart failure. For respiratory distress, the most popular topics were pneumonia detection and prediction followed by pulmonary embolisms. Given the importance of imaging and clinical notes, this area combined Machine Learning with image analysis and natural language processing. In studies on infection, the most researched areas were the detection, prediction, and management of sepsis, surgical site infections, as well as acute kidney injury. Overall, a variety of Machine Learning algorithms were utilized frequently, particularly support vector machines, boosting techniques, random forest classifiers and neural networks. Sensitivity, specificity, and ROC AUC were the most frequently reported performance measures. Conclusion: This review showed an increasing use of Machine Learning for CDS in all three areas. Large datasets are required for training these algorithms; making it imperative to appropriately address, challenges such as class imbalance, correct labelling of data and missing data. Recommendations are formulated for the development and successful adoption of CDS systems.

Project description:BackgroundElectronic medical records are widely used in family practices across Canada and can improve health outcomes. However, recent reports indicate that physicians using electronic medical records work longer and have less direct patient contact which may contribute to burnout. Therefore, new and innovative digital tools are essential to reduce physician workloads and improve patient-physician interaction to address physician burnout. The objective of this study was to assess the efficiency and accuracy of clinical decision-making when using a new preventive care point-of-care clinical decision support system (CDSS). An estimate of the potential annual time savings was also determined. This study also assessed physician reported perceived usefulness and ease of use of the CDSS.MethodsQuantitative and qualitative data were collected during this study. Each participant evaluated two simulated patient charts and identified which preventive care metrics were due. The participants recorded their decisions and the time required to assess each chart. Participants then completed a Technology Acceptance Model survey regarding the perceived usefulness and ease of use of the CDSS, which included qualitative feedback. The amount of time saved was determined and participants' clinical decision-making accuracy was scored against current Canadian preventive care guidelines. The number of preventive care specific visits completed per year was determined using clinic billing data.ResultsThe preventive care CDSS saved an average of 195.7 s of chart review time (249.5 s vs 445.2 s; P < 0.001). A total of 1520 preventive visits were performed at Primrose and Bruyère Family Medicine Centres. Extrapolated across the organization, implementation of the new tool could save 82.6 h per year. Decision-making accuracy was not affected by the new tool (78.4% vs 80.9%, P > 0.05). Participants rated the perceived ease of use and usefulness to be very high.ConclusionsNew digital tools may reduce providers' workload without impacting clinical decision-making accuracy. Participants indicated that the preventive care CDSS was useful and easy to use. Further software development and clinical studies are required to further improve and characterize the effect this new CDSS has when implemented in clinical practice.

Project description:ObjectiveTo identify features of clinical decision support systems critical for improving clinical practice.DesignSystematic review of randomised controlled trials.Data sourcesLiterature searches via Medline, CINAHL, and the Cochrane Controlled Trials Register up to 2003; and searches of reference lists of included studies and relevant reviews.Study selectionStudies had to evaluate the ability of decision support systems to improve clinical practice.Data extractionStudies were assessed for statistically and clinically significant improvement in clinical practice and for the presence of 15 decision support system features whose importance had been repeatedly suggested in the literature.ResultsSeventy studies were included. Decision support systems significantly improved clinical practice in 68% of trials. Univariate analyses revealed that, for five of the system features, interventions possessing the feature were significantly more likely to improve clinical practice than interventions lacking the feature. Multiple logistic regression analysis identified four features as independent predictors of improved clinical practice: automatic provision of decision support as part of clinician workflow (P < 0.00001), provision of recommendations rather than just assessments (P = 0.0187), provision of decision support at the time and location of decision making (P = 0.0263), and computer based decision support (P = 0.0294). Of 32 systems possessing all four features, 30 (94%) significantly improved clinical practice. Furthermore, direct experimental justification was found for providing periodic performance feedback, sharing recommendations with patients, and requesting documentation of reasons for not following recommendations.ConclusionsSeveral features were closely correlated with decision support systems' ability to improve patient care significantly. Clinicians and other stakeholders should implement clinical decision support systems that incorporate these features whenever feasible and appropriate.

Project description:BackgroundComputerized clinical decision support systems (CDSSs) are a promising knowledge translation tool, but often fail to meaningfully influence the outcomes they target. Low CDSS provider uptake is a potential contributor to this problem but has not been systematically studied. The objective of this systematic review and meta-regression was to determine reported CDSS uptake and identify which CDSS features may influence uptake.MethodsMedline, Embase, CINAHL, and the Cochrane Database of Controlled Trials were searched from January 2000 to August 2020. Randomized, non-randomized, and quasi-experimental trials reporting CDSS uptake in any patient population or setting were included. The main outcome extracted was CDSS uptake, reported as a raw proportion, and representing the number of times the CDSS was used or accessed over the total number of times it could have been interacted with. We also extracted context, content, system, and implementation features that might influence uptake, for each CDSS. Overall weighted uptake was calculated using random-effects meta-analysis and determinants of uptake were investigated using multivariable meta-regression.ResultsAmong 7995 citations screened, 55 studies involving 373,608 patients and 3607 providers met full inclusion criteria. Meta-analysis revealed that overall CDSS uptake was 34.2% (95% CI 23.2 to 47.1%). Uptake was only reported in 12.4% of studies that otherwise met inclusion criteria. Multivariable meta-regression revealed the following factors significantly associated with uptake: (1) formally evaluating the availability and quality of the patient data needed to inform CDSS advice; and (2) identifying and addressing other barriers to the behaviour change targeted by the CDSS.Conclusions and relevanceSystem uptake was seldom reported in CDSS trials. When reported, uptake was low. This represents a major and potentially modifiable barrier to overall CDSS effectiveness. We found that features relating to CDSS context and implementation strategy best predicted uptake. Future studies should measure the impact of addressing these features as part of the CDSS implementation strategy. Uptake reporting must also become standard in future studies reporting CDSS intervention effects.RegistrationPre-registered on PROSPERO, CRD42018092337.

Dataset Information

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review.

Publications

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets