Development and validation of machine learning algorithms for postoperative opioid prescriptions after TKA.
Ontology highlight
ABSTRACT:
SUBMITTER: Katakam A
PROVIDER: S-EPMC7152687 | biostudies-literature | 2020 Nov-Dec
REPOSITORIES: biostudies-literature
Publications
Development and validation of machine learning algorithms for postoperative opioid prescriptions after TKA.
Katakam Akhil A Karhade Aditya V AV Schwab Joseph H JH Chen Antonia F AF Bedair Hany S HS
Journal of orthopaedics 20200328
<h4>Objective</h4>The aims of this study were to develop machine learning algorithms for preoperative prediction of prolonged opioid prescriptions after TKA and to identify variables that can predict the probability of this adverse outcome.<h4>Methods</h4>Five algorithms were developed for prediction of prolonged postoperative opioid prescriptions.<h4>Results</h4>The stochastic gradient boosting (SGB) model had the best performance. Age, history of preoperative opioid use, marital status, diagno ...[more]
Similar Datasets
Project description:ImportanceCurrent approaches to identifying individuals at high risk for opioid overdose target many patients who are not truly at high risk.ObjectiveTo develop and validate a machine-learning algorithm to predict opioid overdose risk among Medicare beneficiaries with at least 1 opioid prescription.Design, setting, and participantsA prognostic study was conducted between September 1, 2017, and December 31, 2018. Participants (n = 560 057) included fee-for-service Medicare beneficiaries without cancer who filled 1 or more opioid prescriptions from January 1, 2011, to December 31, 2015. Beneficiaries were randomly and equally divided into training, testing, and validation samples.ExposuresPotential predictors (n = 268), including sociodemographics, health status, patterns of opioid use, and practitioner-level and regional-level factors, were measured in 3-month windows, starting 3 months before initiating opioids until loss of follow-up or the end of observation.Main outcomes and measuresOpioid overdose episodes from inpatient and emergency department claims were identified. Multivariate logistic regression (MLR), least absolute shrinkage and selection operator-type regression (LASSO), random forest (RF), gradient boosting machine (GBM), and deep neural network (DNN) were applied to predict overdose risk in the subsequent 3 months after initiation of treatment with prescription opioids. Prediction performance was assessed using the C statistic and other metrics (eg, sensitivity, specificity, and number needed to evaluate [NNE] to identify one overdose). The Youden index was used to identify the optimized threshold of predicted score that balanced sensitivity and specificity.ResultsBeneficiaries in the training (n = 186 686), testing (n = 186 685), and validation (n = 186 686) samples had similar characteristics (mean [SD] age of 68.0 [14.5] years, and approximately 63% were female, 82% were white, 35% had disabilities, 41% were dual eligible, and 0.60% had at least 1 overdose episode). In the validation sample, the DNN (C statistic = 0.91; 95% CI, 0.88-0.93) and GBM (C statistic = 0.90; 95% CI, 0.87-0.94) algorithms outperformed the LASSO (C statistic = 0.84; 95% CI, 0.80-0.89), RF (C statistic = 0.80; 95% CI, 0.75-0.84), and MLR (C statistic = 0.75; 95% CI, 0.69-0.80) methods for predicting opioid overdose. At the optimized sensitivity and specificity, DNN had a sensitivity of 92.3%, specificity of 75.7%, NNE of 542, positive predictive value of 0.18%, and negative predictive value of 99.9%. The DNN classified patients into low-risk (76.2% [142 180] of the cohort), medium-risk (18.6% [34 579] of the cohort), and high-risk (5.2% [9747] of the cohort) subgroups, with only 1 in 10 000 in the low-risk subgroup having an overdose episode. More than 90% of overdose episodes occurred in the high-risk and medium-risk subgroups, although positive predictive values were low, given the rare overdose outcome.Conclusions and relevanceMachine-learning algorithms appear to perform well for risk prediction and stratification of opioid overdose, especially in identifying low-risk subgroups that have minimal risk of overdose.
| S-EPMC6583312 | biostudies-literature
Project description:BackgroundPostoperative pneumonia, a prevalent form of hospital-acquired pneumonia, poses significant risks to patients' prognosis and even their lives. This study aimed to develop and validate a predictive model for postoperative pneumonia in surgical patients using nine machine learning methods.ObjectiveOur study aims to develop and validate a predictive model for POP in surgical patients using nine machine learning algorithms. By evaluating the performance differences among these machine learning models, this study aims to assist clinicians in early prediction and diagnosis of POP, providing optimal interventions and treatments.MethodsRetrospective data from electronic medical records was collected for 264 patients diagnosed with postoperative pneumonia and 264 healthy control surgical patients. Through correlation screening, chi-square tests, and feature importance ranking, 47 variables were narrowed down to 5 potential predictive factors based on the main cohort of 528 patients. Nine machine learning models, including k-nearest neighbors, support vector machine, random forest, decision tree, gradient boosting machine, adaptive boosting, naive bayes, general linear model, and linear discriminant analysis, were developed and validated to predict postoperative pneumonia. Model performance was evaluated using the area under the receiver operating curve, sensitivity, specificity, accuracy, precision, recall, and F1 score. A distribution plot of feature importance and feature interaction was obtained to interpret the machine learning models.ResultsAmong 17,190 surgical patients, 264 (1.54%) experienced postoperative pneumonia, which resulted in adverse outcomes such as prolonged hospital stay, increased ICU admission rates, and mortality. We successfully established nine machine learning models for predicting postoperative pneumonia in surgical patients, with the general linear model demonstrating the best overall performance. The AUC of the general linear model on the testing set was 0.877, with an accuracy of 0.82, specificity of 0.89, sensitivity of 0.74, precision of 0.88, and F1 score of 0.80. Our study revealed that the duration of bed rest, unplanned re-operation, end-tidal CO2, postoperative albumin, and chest X-ray film were significant predictors of postoperative pneumonia.ConclusionOur study firstly demonstrated that the general linear model based on 5 common variables might predict postoperative pneumonia in the general surgical population.
| S-EPMC11670315 | biostudies-literature
Project description:Associations between delirium and postoperative adverse events in cardiovascular surgery have been reported and the preoperative identification of high-risk patients of delirium is needed to implement focused interventions. We aimed to develop and validate machine learning models to predict post-cardiovascular surgery delirium. Patients aged ≥ 40 years who underwent cardiovascular surgery at a single hospital were prospectively enrolled. Preoperative and intraoperative factors were assessed. Each patient was evaluated for postoperative delirium 7 days after surgery. We developed machine learning models using the Bernoulli naive Bayes, Support vector machine, Random forest, Extra-trees, and XGBoost algorithms. Stratified fivefold cross-validation was performed for each developed model. Of the 87 patients, 24 (27.6%) developed postoperative delirium. Age, use of psychotropic drugs, cognitive function (Mini-Cog < 4), index of activities of daily living (Barthel Index < 100), history of stroke or cerebral hemorrhage, and eGFR (estimated glomerular filtration rate) < 60 were selected to develop delirium prediction models. The Extra-trees model had the best area under the receiver operating characteristic curve (0.76 [standard deviation 0.11]; sensitivity: 0.63; specificity: 0.78). XGBoost showed the highest sensitivity (AUROC, 0.75 [0.07]; sensitivity: 0.67; specificity: 0.79). Machine learning algorithms could predict post-cardiovascular delirium using preoperative data.Trial registration: UMIN-CTR (ID; UMIN000049390).
| S-EPMC10689441 | biostudies-literature
Project description:ObjectivePONV reduces patient satisfaction and increases hospital costs as patients remain in the hospital for longer durations. In this study, we build a preliminary artificial intelligence algorithm model to predict early PONV in patients.MethodsWe use R for statistical analysis and Python for the machine learning prediction model.ResultsAverage characteristic engineering results showed that haloperidol, sex, age, history of smoking, and history of PONV were the first 5 contributing factors in the occurrence of early PONV. Test group results for artificial intelligence prediction of early PONV: in terms of accuracy, the four best algorithms were CNNRNN (0.872), Decision Tree (0.868), SVC (0.866) and adab (0.865); in terms of precision, the three best algorithms were CNNRNN (1.000), adab (0.400) and adab (0.868); in terms of AUC, the top three algorithms were Logistic Regression (0.732), SVC (0.731) and adab (0.722). Finally, we built a website to predict early PONV online using the Streamlit app on the following website: ( https://zhouchengmao-streamlit-app-lsvc-ad-st-app-lsvc-adab-ponv-m9ynsb.streamlit.app/ ).ConclusionArtificial intelligence algorithms can predict early PONV, whereas logistic regression, SVC and adab were the top three artificial intelligence algorithms in overall performance. Haloperidol, sex, age, smoking history, and PONV history were the first 5 contributing factors associated with early PONV.
| S-EPMC10230679 | biostudies-literature
Project description:BackgroundPre-operative risk assessment can help clinicians prepare patients for surgery, reducing the risk of perioperative complications, length of hospital stay, readmission and mortality. Further, it can facilitate collaborative decision-making and operational planning.ObjectiveTo develop effective pre-operative risk assessment algorithms (referred to as Patient Optimizer or POP) using Machine Learning (ML) that predict the development of post-operative complications and provide pilot data to inform the design of a larger prospective study.MethodsAfter institutional ethics approval, we developed a base model that encapsulates the standard manual approach of combining patient-risk and procedure-risk. In an automated process, additional variables were included and tested with 10-fold cross-validation, and the best performing features were selected. The models were evaluated and confidence intervals calculated using bootstrapping. Clinical expertise was used to restrict the cardinality of categorical variables (e.g. pathology results) by including the most clinically relevant values. The models were created with logistic regression (LR) and extreme gradient-boosted trees using XGBoost (Chen and Guestrin, 2016). We evaluated performance using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Data was obtained from a metropolitan university teaching hospital from January 2015 to July 2020. Data collection was restricted to adult patients undergoing elective surgery.ResultsA total of 11,475 adult admissions were included. The performance of XGBoost and LR was very similar across endpoints and metrics. For predicting the risk of any post-operative complication, kidney failure and length-of-stay (LOS), POP with XGBoost achieved an AUROC (95%CI) of 0.755 (0.744, 0.767), 0.869 (0.846, 0.891) and 0.841 (0.833, 0.847) respectively and AUPRC of 0.651 (0.632, 0.669), 0.336 (0.282, 0.390) and 0.741 (0.729, 0.753) respectively. For 30-day readmission and in-patient mortality, POP with XGBoost achieved an AUROC (95%CI) of 0.610 (0.587, 0.635) and 0.866 (0.777, 0.943) respectively and AUPRC of 0.116 (0.104, 0.132) and 0.031 (0.015, 0.072) respectively.ConclusionThe POP algorithms effectively predicted any post-operative complication, kidney failure and LOS in the sample population. A larger study is justified to improve the algorithm to better predict complications and length of hospital stay. A larger dataset may also improve the prediction of additional specific complications, readmission and mortality.
| S-EPMC10929113 | biostudies-literature
Project description:ObjectivesTo develop and externally validate machine learning models using structured and unstructured electronic health record data to predict postoperative acute kidney injury (AKI) across inpatient settings.Materials and methodsData for adult postoperative admissions to the Loyola University Medical Center (2009-2017) were used for model development and admissions to the University of Wisconsin-Madison (2009-2020) were used for validation. Structured features included demographics, vital signs, laboratory results, and nurse-documented scores. Unstructured text from clinical notes were converted into concept unique identifiers (CUIs) using the clinical Text Analysis and Knowledge Extraction System. The primary outcome was the development of Kidney Disease Improvement Global Outcomes stage 2 AKI within 7 days after leaving the operating room. We derived unimodal extreme gradient boosting machines (XGBoost) and elastic net logistic regression (GLMNET) models using structured-only data and multimodal models combining structured data with CUI features. Model comparison was performed using the receiver operating characteristic curve (AUROC), with Delong's test for statistical differences.ResultsThe study cohort included 138 389 adult patient admissions (mean [SD] age 58 [16] years; 11 506 [8%] African-American; and 70 826 [51%] female) across the 2 sites. Of those, 2959 (2.1%) developed stage 2 AKI or higher. Across all data types, XGBoost outperformed GLMNET (mean AUROC 0.81 [95% confidence interval (CI), 0.80-0.82] vs 0.78 [95% CI, 0.77-0.79]). The multimodal XGBoost model incorporating CUIs parameterized as term frequency-inverse document frequency (TF-IDF) showed the highest discrimination performance (AUROC 0.82 [95% CI, 0.81-0.83]) over unimodal models (AUROC 0.79 [95% CI, 0.78-0.80]).DiscussionA multimodality approach with structured data and TF-IDF weighting of CUIs increased model performance over structured data-only models.ConclusionThese findings highlight the predictive power of CUIs when merged with structured data for clinical prediction models, which may improve the detection of postoperative AKI.
| S-EPMC10746378 | biostudies-literature
Project description:ObjectiveTo utilize electronic health records (EHRs) to study SLE, algorithms are needed to accurately identify these patients. We used machine learning to generate data-driven SLE EHR algorithms and assessed performance of existing rule-based algorithms.MethodsWe randomly selected subjects with ≥ 1 SLE ICD-9/10 codes from our EHR and identified gold standard definite and probable SLE cases by chart review, based on 1997 ACR or 2012 SLICC Classification Criteria. From a training set, we extracted coded and narrative concepts using natural language processing and generated algorithms using penalized logistic regression to classify definite or definite/probable SLE. We assessed predictive characteristics in internal and external cohort validations. We also tested performance characteristics of published rule-based algorithms with pre-specified permutations of ICD-9 codes, laboratory tests and medications in our EHR.ResultsAt a specificity of 97%, our machine learning coded algorithm for definite SLE had 90% positive predictive value (PPV) and 64% sensitivity and for definite/probable SLE, 92% PPV and 47% sensitivity. In the external validation, at 97% specificity, the definite/probable algorithm had 94% PPV and 60% sensitivity. Adding NLP concepts did not improve performance metrics. The PPVs of published rule-based algorithms ranged from 45-79% in our EHR.ConclusionOur machine learning SLE algorithms performed well in internal and external validation. Rule-based SLE algorithms did not transport as well to our EHR. Unique EHR characteristics, clinical practices and research goals regarding the desired sensitivity and specificity of the case definition must be considered when applying algorithms to identify SLE patients.
| S-EPMC6609504 | biostudies-literature
Project description:BackgroundThe clinical impact of postoperative opioid use requires accurate prediction strategies to identify at-risk patients. We utilize preoperative claims data to predict postoperative opioid refill and new persistent use in opioid-naïve patients.MethodsA retrospective study was conducted on 112,898 opioid-naïve adult postoperative patients from Optum's de-identified Clinformatics® Data Mart database. Potential predictors included sociodemographic data, comorbidities, and prescriptions within one year prior to surgery.ResultsCompared to linear models, non-linear models led to modest improvements in predicting refills - area under the receiver operating characteristics curve (AUROC) 0.68 vs. 0.67 (p < 0.05) - and performed identically in predicting new persistent use - AUROC = 0.66. Undergoing major surgery, opioid prescriptions within 30 days prior to surgery, and abdominal pain were useful in predicting refills; back/joint/head pain were the most important features in predicting new persistent use.ConclusionsPreoperative patient attributes from insurance claims could potentially be useful in guiding prescription practices for opioid-naïve patients.
| S-EPMC8373633 | biostudies-literature
Project description:IntroductionHypothyroidism can be easily misdiagnosed in dogs, and prediction models can support clinical decision-making, avoiding unnecessary testing and treatment. The aim of this study is to develop and internally validate diagnostic prediction models for hypothyroidism in dogs by applying machine-learning algorithms.MethodsA single-institutional cross-sectional study was designed searching the electronic database of a Veterinary Teaching Hospital for dogs tested for hypothyroidism. Hypothyroidism was diagnosed based on suggestive clinical signs and thyroid function tests. Dogs were excluded if medical records were incomplete or a definitive diagnosis was lacking. Predictors identified after data processing were dermatological signs, alopecia, lethargy, hematocrit, serum concentrations of cholesterol, creatinine, total thyroxine (tT4), and thyrotropin (cTSH). Four models were created by combining clinical signs and clinicopathological variables expressed as quantitative (models 1 and 2) and qualitative variables (models 3 and 4). Models 2 and 4 included tT4 and cTSH, models 1 and 3 did not. Six different algorithms were applied to each model. Internal validation was performed using a 10-fold cross-validation. Apparent performance was evaluated by calculating the area under the receiver operating characteristic curve (AUROC).ResultsEighty-two hypothyroid and 233 euthyroid client-owned dogs were included. The best performing algorithms were naive Bayes in model 1 (AUROC = 0.85; 95% confidence interval [CI] = 0.83-0.86) and in model 2 (AUROC = 0.98; 95% CI = 0.97-0.99), logistic regression in model 3 (AUROC = 0.88; 95% CI = 0.86-0.89), and random forest in model 4 (AUROC = 0.99; 95% CI = 0.98-0.99). Positive predictive value was 0.76, 0.84, 0.93, and 0.97 in model 1, 2, 3, and 4, respectively. Negative predictive value was 0.89, 0.89, 0.99, and 0.99 in model 1, 2, 3, and 4, respectively.DiscussionMachine learning-based prediction models were accurate in predicting and quantifying the likelihood of hypothyroidism in dogs based on internal validation performed in a single-institution, but external validation is required to support the clinical applicability of these models.
| S-EPMC10758480 | biostudies-literature
Project description:ObjectiveTo compare outcomes after surgery between patients who were not prescribed opioids and patients who were prescribed opioids.Summary of background dataPostoperative opioid prescriptions carry significant risks. Understanding outcomes among patients who receive no opioids after surgery may inform efforts to reduce these risks.MethodsWe performed a retrospective study of adult patients who underwent surgery between January 1, 2019 and October 31, 2019. The primary outcome was the composite incidence of an emergency department visit, readmission, or reoperation within 30 days of surgery. Secondary outcomes were postoperative pain, satisfaction, quality of life, and regret collected via postoperative survey. A multilevel, mixed-effects logistic regression was performed to evaluate differences between groups.ResultsIn a cohort of 22,345 patients, mean age (standard deviation) was 52.1 (16.5) years and 13,269 (59.4%) patients were female. About 3175 (14.2%) patients were not prescribed opioids, of whom 422 (13.3%) met the composite adverse event endpoint compared to 2255 (11.8%) of patients not prescribed opioids ( P = 0.015). Patients not prescribed opioids had a similar probability of adverse events {11.7% [95% confidence interval (CI) 10.2%-13.2%] vs 11.9% (95% CI 10.6%-13.3%]}. Among 12,872 survey respondents, patients who were not prescribed an opioid had a similar rate of high satisfaction [81.7% (95% CI 77.3%-86.1%) vs 81.7% (95% CI 77.7%- 85.7%)] and no regret [(93.0% (95% CI 90.8%-95.2%) vs 92.6% (95% CI 90.4%-94.7%)].ConclusionsPatients who were not prescribed opioids after surgery had similar clinical and patient-reported outcomes as patients who were prescribed opioids. This suggests that minimizing opioids as part of routine postoperative care is unlikely to adversely affect patients.
| S-EPMC8787466 | biostudies-literature