Dataset Information

Derivation, Validation, Sustained Performance, and Clinical Impact of an Electronic Medical Record-Based Perioperative Delirium Risk Stratification Tool.

ABSTRACT:

Background

Postoperative delirium is an important problem for surgical inpatients and was the target of a multidisciplinary quality improvement project at our institution. We developed and tested a semiautomated delirium risk stratification instrument, Age, WORLD backwards, Orientation, iLlness severity, Surgery-specific risk (AWOL-S), in 3 independent cohorts from our tertiary care hospital and describe its performance characteristics and impact on clinical care.

Methods

The risk stratification instrument was derived with elective surgical patients who were admitted at least overnight and received at least 1 postoperative delirium screen (Nursing Delirium Screening Scale [NuDESC] or Confusion Assessment Method for the Intensive Care Unit [CAM-ICU]) and preoperative cognitive screening tests (orientation to place and ability to spell WORLD backward). Using data pragmatically collected between December 7, 2016, and June 15, 2017, we derived a logistic regression model predicting probability of delirium in the first 7 postoperative hospital days. A priori predictors included age, cognitive screening, illness severity or American Society of Anesthesiologists physical status, and surgical delirium risk. We applied model odds ratios to 2 subsequent cohorts ("validation" and "sustained performance") and assessed performance using area under the receiver operator characteristic curves (AUC-ROC). A post hoc sensitivity analysis assessed performance in emergency and preadmitted patients. Finally, we retrospectively evaluated the use of benzodiazepines and anticholinergic medications in patients who screened at high risk for delirium.

Results

The logistic regression model used to derive odds ratios for the risk prediction tool included 2091 patients. Model AUC-ROC was 0.71 (0.67-0.75), compared with 0.65 (0.58-0.72) in the validation (n = 908) and 0.75 (0.71-0.78) in the sustained performance (n = 3168) cohorts. Sensitivity was approximately 75% in the derivation and sustained performance cohorts; specificity was approximately 59%. The AUC-ROC for emergency and preadmitted patients was 0.71 (0.67-0.75; n = 1301). After AWOL-S was implemented clinically, patients at high risk for delirium (n = 3630) had 21% (3%-36%) lower relative risk of receiving an anticholinergic medication perioperatively after controlling for secular trends.

Conclusions

The AWOL-S delirium risk stratification tool has moderate accuracy for delirium prediction in a cohort of elective surgical patients, and performance is largely unchanged in emergent/preadmitted surgical patients. Using AWOL-S risk stratification as a part of a multidisciplinary delirium reduction intervention was associated with significantly lower rates of perioperative anticholinergic but not benzodiazepine, medications in those at high risk for delirium. AWOL-S offers a feasible starting point for electronic medical record-based postoperative delirium risk stratification and may serve as a useful paradigm for other institutions.

SUBMITTER: Whitlock EL

PROVIDER: S-EPMC7669577 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectivesTo develop and validate a simple clinical prediction rule, based on variables easily measurable at admission, to identify patients at high risk of developing delirium during their hospital stay on an internal medicine ward.DesignProspective study of two cohorts of patients admitted between 1 May and 30 June 2008 (derivation cohort), and between 1 May and 30 June 2009 (validation cohort).SettingA tertiary hospital in Donostia-Gipuzkoa (Spain).ParticipantsIn total 397 patients participated in the study. The mean age and incidence of delirium were 75.9 years and 13%, respectively, in the derivation cohort, and 75.8 years and 25% in the validation cohort.Main outcome measuresThe predictive variables analysed and finally included in the rule were: being aged 85 years old or older, being dependent in five or more activities of daily living, and taking two or more psychotropic drugs (antipsychotics, benzodiazepines, antidepressants, anticonvulsant and/or antidementia drugs). The variable of interest was delirium as defined by the short Confusion Assessment Method, which assesses four characteristics: acute onset and fluctuating course, inattention, disorganised thinking and altered level of consciousness.ResultsWe developed a rule in which the individual risk of delirium is obtained by adding one point for each criterion met (age?85, high level of dependence, and being on psychotropic medication). The result is considered positive if the score is ?1. The rule accuracy was: sensitivity=93.4% (95% CI 85.5% to 97.2%), specificity=60.6% (95% CI 54.1% to 66.8%), positive predictive value=44.4% (95% CI 36.9% to 52.1%) and negative predictive value=96.5% (95% CI: 92% to 98.5%). The area under the receiver operator characteristic (ROC) curve was 0.85 for the validation cohort.ConclusionsThe presence or absence of any of the three predictive factors (age?85, high level of dependence and psychotropic medication) allowed us to classify patients on internal medicine wards according to the risk of developing delirium. The simplicity of the variables in our clinical prediction rule means that the data collection required is feasible in busy medicine units.

Project description:BackgroundDeploying accurate computable phenotypes in pragmatic trials requires a trade-off between precise and clinically sensical variable selection. In particular, evaluating the medical encounter to assess a pattern leading to clinically significant impairment or distress indicative of disease is a difficult modeling challenge for the emergency department.ObjectiveThis study aimed to derive and validate an electronic health record-based computable phenotype to identify emergency department patients with opioid use disorder using physician chart review as a reference standard.MethodsA two-algorithm computable phenotype was developed and evaluated using structured clinical data across 13 emergency departments in two large health care systems. Algorithm 1 combined clinician and billing codes. Algorithm 2 used chief complaint structured data suggestive of opioid use disorder. To evaluate the algorithms in both internal and external validation phases, two emergency medicine physicians, with a third acting as adjudicator, reviewed a pragmatic sample of 231 charts: 125 internal validation (75 positive and 50 negative), 106 external validation (56 positive and 50 negative).ResultsCohen kappa, measuring agreement between reviewers, for the internal and external validation cohorts was 0.95 and 0.93, respectively. In the internal validation phase, Algorithm 1 had a positive predictive value (PPV) of 0.96 (95% CI 0.863-0.995) and a negative predictive value (NPV) of 0.98 (95% CI 0.893-0.999), and Algorithm 2 had a PPV of 0.8 (95% CI 0.593-0.932) and an NPV of 1.0 (one-sided 97.5% CI 0.863-1). In the external validation phase, the phenotype had a PPV of 0.95 (95% CI 0.851-0.989) and an NPV of 0.92 (95% CI 0.807-0.978).ConclusionsThis phenotype detected emergency department patients with opioid use disorder with high predictive values and reliability. Its algorithms were transportable across health care systems and have potential value for both clinical and research purposes.