Dataset Information

Use of Machine Learning to Develop and Evaluate Models Using Preoperative and Intraoperative Data to Identify Risks of Postoperative Complications.

ABSTRACT:

Importance

Postoperative complications can significantly impact perioperative care management and planning.

Objectives

To assess machine learning (ML) models for predicting postoperative complications using independent and combined preoperative and intraoperative data and their clinically meaningful model-agnostic interpretations.

Design, setting, and participants

This retrospective cohort study assessed 111 888 operations performed on adults at a single academic medical center from June 1, 2012, to August 31, 2016, with a mean duration of follow-up based on the length of postoperative hospital stay less than 7 days. Data analysis was performed from February 1 to September 31, 2020.

Main outcomes and measures

Outcomes included 5 postoperative complications: acute kidney injury (AKI), delirium, deep vein thrombosis (DVT), pulmonary embolism (PE), and pneumonia. Patient and clinical characteristics available preoperatively, intraoperatively, and a combination of both were used as inputs for 5 candidate ML models: logistic regression, support vector machine, random forest, gradient boosting tree (GBT), and deep neural network (DNN). Model performance was compared using the area under the receiver operating characteristic curve (AUROC). Model interpretations were generated using Shapley Additive Explanations by transforming model features into clinical variables and representing them as patient-specific visualizations.

Results

A total of 111 888 patients (mean [SD] age, 54.4 [16.8] years; 56 915 [50.9%] female; 82 533 [73.8%] White) were included in this study. The best-performing model for each complication combined the preoperative and intraoperative data with the following AUROCs: pneumonia (GBT), 0.905 (95% CI, 0.903-0.907); AKI (GBT), 0.848 (95% CI, 0.846-0.851); DVT (GBT), 0.881 (95% CI, 0.878-0.884); PE (DNN), 0.831 (95% CI, 0.824-0.839); and delirium (GBT), 0.762 (95% CI, 0.759-0.765). Performance of models that used only preoperative data or only intraoperative data was marginally lower than that of models that used combined data. When adding variables with missing data as input, AUROCs increased from 0.588 to 0.905 for pneumonia, 0.579 to 0.848 for AKI, 0.574 to 0.881 for DVT, 0.5 to 0.831 for PE, and 0.6 to 0.762 for delirium. The Shapley Additive Explanations analysis generated model-agnostic interpretation that illustrated significant clinical contributors associated with risks of postoperative complications.

Conclusions and relevance

The ML models for predicting postoperative complications with model-agnostic interpretation offer opportunities for integrating risk predictions for clinical decision support. Such real-time clinical decision support can mitigate patient risks and help in anticipatory management for perioperative contingency planning.

SUBMITTER: Xue B

PROVIDER: S-EPMC8010590 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ImportanceA variety of perioperative risk factors are associated with postoperative mortality risk. However, the relative contribution of routinely collected intraoperative clinical parameters to short-term and long-term mortality remains understudied.ObjectiveTo examine the performance of multiple machine learning models with data from different perioperative periods to predict 30-day, 1-year, and 5-year mortality and investigate factors that contribute to these predictions.Design, setting, and participantsIn this prognostic study using prospectively collected data, risk prediction models were developed for short-term and long-term mortality after cardiac surgery. Included participants were adult patients undergoing a first-time valve operation, coronary artery bypass grafting, or a combination of both between 1997 and 2017 in a single center, the University Medical Centre Groningen in the Netherlands. Mortality data were obtained in November 2017. Data analysis took place between February 2020 and August 2021.ExposureCardiac surgery.Main outcomes and measuresPostoperative mortality rates at 30 days, 1 year, and 5 years were the primary outcomes. The area under the receiver operating characteristic curve (AUROC) was used to assess discrimination. The contribution of all preoperative, intraoperative hemodynamic and temperature, and postoperative factors to mortality was investigated using Shapley additive explanations (SHAP) values.ResultsData from 9415 patients who underwent cardiac surgery (median [IQR] age, 68 [60-74] years; 2554 [27.1%] women) were included. Overall mortality rates at 30 days, 1 year, and 5 years were 268 patients (2.8%), 420 patients (4.5%), and 612 patients (6.5%), respectively. Models including preoperative, intraoperative, and postoperative data achieved AUROC values of 0.82 (95% CI, 0.78-0.86), 0.81 (95% CI, 0.77-0.85), and 0.80 (95% CI, 0.75-0.84) for 30-day, 1-year, and 5-year mortality, respectively. Models including only postoperative data performed similarly (30 days: 0.78 [95% CI, 0.73-0.82]; 1 year: 0.79 [95% CI, 0.74-0.83]; 5 years: 0.77 [95% CI, 0.73-0.82]). However, models based on all perioperative data provided less clinically usable predictions, with lower detection rates; for example, postoperative models identified a high-risk group with a 2.8-fold increase in risk for 5-year mortality (4.1 [95% CI, 3.3-5.1]) vs an increase of 11.3 (95% CI, 6.8-18.7) for the high-risk group identified by the full perioperative model. Postoperative markers associated with metabolic dysfunction and decreased kidney function were the main factors contributing to mortality risk.Conclusions and relevanceThis study found that the addition of continuous intraoperative hemodynamic and temperature data to postoperative data was not associated with improved machine learning-based identification of patients at increased risk of short-term and long-term mortality after cardiac operations.

Project description:BackgroundAccurate, pragmatic risk stratification for postoperative delirium (POD) is necessary to target preventative resources toward high-risk patients. Machine learning (ML) offers a novel approach to leveraging electronic health record (EHR) data for POD prediction. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression.MethodsThis was a retrospective analysis of preoperative EHR data from 24,885 adults undergoing a procedure requiring anesthesia care, recovering in the main post-anesthesia care unit, and staying in the hospital at least overnight between December 2016 and December 2019 at either of two hospitals in a tertiary care health system. One hundred fifteen preoperative risk features including demographics, comorbidities, nursing assessments, surgery type, and other preoperative EHR data were used to predict postoperative delirium (POD), defined as any instance of Nursing Delirium Screening Scale ≥2 or positive Confusion Assessment Method for the Intensive Care Unit within the first 7 postoperative days. Two ML models (Neural Network and XGBoost), two traditional logistic regression models ("clinician-guided" and "ML hybrid"), and a previously described delirium risk stratification tool (AWOL-S) were evaluated using the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, positive likelihood ratio, and positive predictive value. Model calibration was assessed with a calibration curve. Patients with no POD assessments charted or at least 20% of input variables missing were excluded.ResultsPOD incidence was 5.3%. The AUC-ROC for Neural Net was 0.841 [95% CI 0. 816-0.863] and for XGBoost was 0.851 [95% CI 0.827-0.874], which was significantly better than the clinician-guided (AUC-ROC 0.763 [0.734-0.793], p < 0.001) and ML hybrid (AUC-ROC 0.824 [0.800-0.849], p < 0.001) regression models and AWOL-S (AUC-ROC 0.762 [95% CI 0.713-0.812], p < 0.001). Neural Net, XGBoost, and ML hybrid models demonstrated excellent calibration, while calibration of the clinician-guided and AWOL-S models was moderate; they tended to overestimate delirium risk in those already at highest risk.ConclusionUsing pragmatically collected EHR data, two ML models predicted POD in a broad perioperative population with high discrimination. Optimal application of the models would provide automated, real-time delirium risk stratification to improve perioperative management of surgical patients at risk for POD.

Project description:BackgroundCompared with historic ventilation strategies, modern lung-protective ventilation includes lower tidal volumes (VT), lower driving pressures, and application of positive end-expiratory pressure (PEEP). The contributions of each component to an overall intraoperative protective ventilation strategy aimed at reducing postoperative pulmonary complications have neither been adequately resolved, nor comprehensively evaluated within an adult cardiac surgical population. The authors hypothesized that a bundled intraoperative protective ventilation strategy was independently associated with decreased odds of pulmonary complications after cardiac surgery.MethodsIn this observational cohort study, the authors reviewed nonemergent cardiac surgical procedures using cardiopulmonary bypass at a tertiary care academic medical center from 2006 to 2017. The authors tested associations between bundled or component intraoperative protective ventilation strategies (VT below 8 ml/kg ideal body weight, modified driving pressure [peak inspiratory pressure - PEEP] below 16 cm H2O, and PEEP greater than or equal to 5 cm H2O) and postoperative outcomes, adjusting for previously identified risk factors. The primary outcome was a composite pulmonary complication; secondary outcomes included individual pulmonary complications, postoperative mortality, as well as durations of mechanical ventilation, intensive care unit stay, and hospital stay.ResultsAmong 4,694 cases reviewed, 513 (10.9%) experienced pulmonary complications. After adjustment, an intraoperative lung-protective ventilation bundle was associated with decreased pulmonary complications (adjusted odds ratio, 0.56; 95% CI, 0.42-0.75). Via a sensitivity analysis, modified driving pressure below 16 cm H2O was independently associated with decreased pulmonary complications (adjusted odds ratio, 0.51; 95% CI, 0.39-0.66), but VT below 8 ml/kg and PEEP greater than or equal to 5 cm H2O were not.ConclusionsThe authors identified an intraoperative lung-protective ventilation bundle as independently associated with pulmonary complications after cardiac surgery. The findings offer insight into components of protective ventilation associated with adverse outcomes and may serve as targets for future prospective interventional studies investigating the impact of specific protective ventilation strategies on postoperative outcomes after cardiac surgery.

Project description:BackgroundPostoperative delirium is a common complication that hinders recovery after surgery. Intraoperative electroencephalogram suppression has been linked to postoperative delirium, but it is unknown if this relationship is causal or if electroencephalogram suppression is merely a marker of underlying cognitive abnormalities. The hypothesis of this study was that intraoperative electroencephalogram suppression mediates a nonzero portion of the effect between preoperative abnormal cognition and postoperative delirium.MethodsThis is a prespecified secondary analysis of the Electroencephalography Guidance of Anesthesia to Alleviate Geriatric Syndromes (ENGAGES) randomized trial, which enrolled patients age 60 yr or older undergoing surgery with general anesthesia at a single academic medical center between January 2015 and May 2018. Patients were randomized to electroencephalogram-guided anesthesia or usual care. Preoperative abnormal cognition was defined as a composite of previous delirium, Short Blessed Test cognitive score greater than 4 points, or Eight Item Interview to Differentiate Aging and Dementia score greater than 1 point. Duration of intraoperative electroencephalogram suppression was defined as number of minutes with suppression ratio greater than 1%. Postoperative delirium was detected via Confusion Assessment Method or chart review on postoperative days 1 to 5.ResultsAmong 1,113 patients, 430 patients showed evidence of preoperative abnormal cognition. These patients had an increased incidence of postoperative delirium (151 of 430 [35%] vs.123 of 683 [18%], P < 0.001). Of this 17.2% total effect size (99.5% CI, 9.3 to 25.1%), an absolute 2.4% (99.5% CI, 0.6 to 4.8%) was an indirect effect mediated by electroencephalogram suppression, while an absolute 14.8% (99.5% CI, 7.2 to 22.5%) was a direct effect of preoperative abnormal cognition. Randomization to electroencephalogram-guided anesthesia did not change the mediated effect size (P = 0.078 for moderation).ConclusionsA small portion of the total effect of preoperative abnormal cognition on postoperative delirium was mediated by electroencephalogram suppression. Study precision was too low to determine if the intervention changed the mediated effect.

Project description:Introduction: The role of intraoperative ventilation strategies in subjects undergoing surgery is still contested. This meta-analysis study was performed to assess the relationship between the low tidal volumes strategy and conventional mechanical ventilation in subjects undergoing surgery. Methods: A systematic literature search up to December 2020 was performed in OVID, Embase, Cochrane Library, PubMed, and Google scholar, and 28 studies including 11,846 subjects undergoing surgery at baseline and reporting a total of 2,638 receiving the low tidal volumes strategy and 3,632 receiving conventional mechanical ventilation, were found recording relationships between low tidal volumes strategy and conventional mechanical ventilation in subjects undergoing surgery. Odds ratio (OR) or mean difference (MD) with 95% confidence intervals (CIs) were calculated between the low tidal volumes strategy vs. conventional mechanical ventilation using dichotomous and continuous methods with a random or fixed-effect model. Results: The low tidal volumes strategy during surgery was significantly related to a lower rate of postoperative pulmonary complications (OR, 0.60; 95% CI, 0.44-0.83, p < 0.001), aspiration pneumonitis (OR, 0.63; 95% CI, 0.46-0.86, p < 0.001), and pleural effusion (OR, 0.72; 95% CI, 0.56-0.92, p < 0.001) compared to conventional mechanical ventilation. However, the low tidal volumes strategy during surgery was not significantly correlated with length of hospital stay (MD, -0.48; 95% CI, -0.99-0.02, p = 0.06), short-term mortality (OR, 0.88; 95% CI, 0.70-1.10, p = 0.25), atelectasis (OR, 0.76; 95% CI, 0.57-1.01, p = 0.06), acute respiratory distress (OR, 1.06; 95% CI, 0.67-1.66, p = 0.81), pneumothorax (OR, 1.37; 95% CI, 0.88-2.15, p = 0.17), pulmonary edema (OR, 0.70; 95% CI, 0.38-1.26, p = 0.23), and pulmonary embolism (OR, 0.65; 95% CI, 0.26-1.60, p = 0.35) compared to conventional mechanical ventilation. Conclusions: The low tidal volumes strategy during surgery may have an independent relationship with lower postoperative pulmonary complications, aspiration pneumonitis, and pleural effusion compared to conventional mechanical ventilation. This relationship encouraged us to recommend the low tidal volumes strategy during surgery to avoid any possible complications.