Dataset Information

Detecting Incident Delirium within Routinely Collected Inpatient Rehabilitation Data: Validation of a Chart-Based Method.

ABSTRACT:

Background

Delirium is a brain condition associated with poor outcomes in rehabilitation. It is therefore important to assess delirium incidence in rehabilitation.

Purpose

To develop and validate a chart-based method to identify incident delirium episodes within the electronic database of a Swiss rehabilitation clinic, and to identify a study population of validated incident delirium episodes for further research purposes.

Design

Retrospective validation study.

Settings

Routinely collected inpatient clinical data from ZURZACH Care.

Participants

All patients undergoing rehabilitation at ZURZACH Care, Rehaklinik Bad Zurzach between 2015 and 2018 were included.

Methods

Within the study population, we identified all rehabilitation stays for which ≥2 delirium-predictive key words (common terms used to describe delirious patients) were recorded in the medical charts. We excluded all prevalent delirium episodes and defined the remaining episodes to be potentially incident. At least two physicians independently confirmed or refuted each potential incident delirium episode by reviewing the patient charts. We calculated the positive predictive value (PPV) with 95% confidence interval (95% CI) for all potential incident delirium episodes and for specific subgroups.

Results

Within 10,515 rehabilitation stays we identified 554 potential incident delirium episodes. Overall, 125 potential incident delirium episodes were confirmed by expert review. The PPV of the chart-based method varied from 0.23 (95% CI 0.19-0.26) overall to 0.69 (95% CI 0.56-0.79) in specific subgroups.

Conclusions

Our chart-based method was able to capture incident delirium episodes with low to moderate accuracy. By conducting an additional expert review of the medical charts, we identified a study population of validated incident delirium episodes. Our chart-based method contributes towards an automated detection of potential incident delirium episodes that, supplemented with expert review, efficiently yields a validated population of incident delirium episodes for research purposes.

SUBMITTER: Ceppi MG

PROVIDER: S-EPMC8705493 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo adapt and validate a chart-based delirium detection tool for use in critically ill adults.DesignValidation study.SettingMedical-surgical intensive care unit (ICU) in an academic hospital.MeasurementsA chart-based delirium detection tool (CHART-DEL) was adapted for use in critically ill adults (CHART-DEL-ICU) and compared with prospective delirium assessments (i.e., clinical assessments (reference standard) by a research nurse trained by a neuropsychiatrist and routine delirium screening tools Confusion Assessment Method (CAM-ICU)) and (Intensive Care Delirium Screening Checklist (ICDSC)). The original CHART-DEL tool was adapted to include physician-reported ICDSC score (for probable delirium) and Richmond-Agitation Sedation Scale score (for altered level of consciousness and agitation). Two trained chart abstractors blinded to all delirium assessments manually abstracted delirium-related information from medical charts and electronic medical records and rated if delirium was present (four levels: uncertain, possible, probable, definite) or absent (no evidence).ResultsCharts were manually abstracted for delirium-related information for 213 patients who were included in a prospective cohort study that included prospective delirium assessments. The CHART-DEL-ICU tool had excellent interrater reliability (kappa = 0.90). Compared to the reference standard, the sensitivity was 66.0% (95% CI = 59.3-72.3%) and specificity was 82.1% (95% CI = 78.0-85.7%), with a cut-point that included definite, probable, possible, and uncertain delirium. The AUC of the CHART-DEL-ICU alone is 74.1% (95% CI = 70.4-77.8%) compared with the addition of the CAM-ICU and ICDSC (CAM-ICU/CHART-DEL-ICU: 80.9% (95% CI = 77.8-83.9%), P = .01; ICDSC/CHART-DEL-ICU: 79.2% (95% CI = 75.9-82.6%), P = .03).ConclusionA chart-based delirium detection tool has improved diagnostic accuracy when combined with routine delirium screening tools (CAM-ICU and ICDSC), compared to a chart-based method on its own. This presents a potential for retrospective detection of delirium from medical charts for research or to augment routine delirium screening methods to find missed cases of delirium.

Project description:Importance:Inpatient violence remains a significant problem despite existing risk assessment methods. The lack of robustness and the high degree of effort needed to use current methods might be mitigated by using routinely registered clinical notes. Objective:To develop and validate a multivariable prediction model for assessing inpatient violence risk based on machine learning techniques applied to clinical notes written in patients' electronic health records. Design, Setting, and Participants:This prognostic study used retrospective clinical notes registered in electronic health records during admission at 2 independent psychiatric health care institutions in the Netherlands. No exclusion criteria for individual patients were defined. At site 1, all adults admitted between January 2013 and August 2018 were included, and at site 2 all adults admitted to general psychiatric wards between June 2016 and August 2018 were included. Data were analyzed between September 2018 and February 2019. Main Outcomes and Measures:Predictive validity and generalizability of prognostic models measured using area under the curve (AUC). Results:Clinical notes recorded during a total of 3189 admissions of 2209 unique individuals at site 1 (mean [SD] age, 34.0 [16.6] years; 1536 [48.2%] male) and 3253 admissions of 1919 unique individuals at site 2 (mean [SD] age, 45.9 [16.6] years; 2097 [64.5%] male) were analyzed. Violent outcome was determined using the Staff Observation Aggression Scale-Revised. Nested cross-validation was used to train and evaluate models that assess violence risk during the first 4 weeks of admission based on clinical notes available after 24 hours. The predictive validity of models was measured at site 1 (AUC = 0.797; 95% CI, 0.771-0.822) and site 2 (AUC = 0.764; 95% CI, 0.732-0.797). The validation of pretrained models in the other site resulted in AUCs of 0.722 (95% CI, 0.690-0.753) at site 1 and 0.643 (95% CI, 0.610-0.675) at site 2; the difference in AUCs between the internally trained model and the model trained on other-site data was significant at site 1 (AUC difference = 0.075; 95% CI, 0.045-0.105; P < .001) and site 2 (AUC difference = 0.121; 95% CI, 0.085-0.156; P < .001). Conclusions and Relevance:Internally validated predictions resulted in AUC values with good predictive validity, suggesting that automatic violence risk assessment using routinely registered clinical notes is possible. The validation of trained models using data from other sites corroborates previous findings that violence risk assessment generalizes modestly to different populations.

Project description:Validated methods of identifying childhood maltreatment (CM) in primary and secondary care data are needed. We aimed to create the first externally validated algorithm for identifying maltreatment using routinely collected healthcare data. Comprehensive code lists were created for use within GP and hospital admissions datasets in the SAIL Databank at Swansea University working with safeguarding clinicians and academics. These code lists build on and refine those previously published to include an exhaustive set of codes. Sensitivity, specificity and positive predictive value of previously published lists and the new algorithm were estimated against a clinically assessed cohort of CM cases from a child protection service secondary care-based setting-'the gold standard'. We conducted sensitivity analyses to examine the utility of wider codes indicating Possible CM. Trends over time from 2004 to 2020 were calculated using Poisson regression modelling. Our algorithm outperformed previously published lists identifying 43-72% of cases in primary care with a specificity ≥ 85%. Sensitivity of algorithms for identifying maltreatment in hospital admissions data was lower identifying between 9 and 28% of cases with high specificity (> 96%). Manual searching of records for those cases identified by the external dataset but not recorded in primary care suggest that this code list is exhaustive. Exploration of missed cases shows that hospital admissions data is often focused on the injury being treated rather than recording the presence of maltreatment. The absence of child protection or social care codes in hospital admissions data poses a limitation for identifying maltreatment in admissions data. Linking across GP and hospital admissions maximises the number of cases of maltreatment that can be accurately identified. Incidence of maltreatment in primary care using these code lists has increased over time. The updated algorithm has improved our ability to detect CM in routinely collected healthcare data. It is important to recognize the limitations of identifying maltreatment in individual healthcare datasets. The inclusion of child protection codes in primary care data makes this an important setting for identifying CM, whereas hospital admissions data is often focused on injuries with CM codes often absent. Implications and utility of algorithms for future research are discussed.

Project description:BackgroundGuidelines recommend identifying in early pregnancy women at elevated risk of pre-eclampsia. The aim of this study was to develop and validate a pre-eclampsia risk prediction model for nulliparous women attending routine antenatal care "the Western Sydney (WS) model"; and to compare its performance with the National Institute of Health and Care Excellence (NICE) risk factor-list approach for classifying women as high-risk.MethodsThis retrospective cohort study included all nulliparous women who gave birth in three public hospitals in the Western-Sydney-Local-Health-District, Australia 2011-2014. Using births from 2011 to 2012, multivariable logistic regression incorporated established maternal risk factors to develop and internally validate the WS model. The WS model was then externally validated using births from 2013 to 2014, assessing its discrimination and calibration. We fitted the final WS model for all births from 2011 to 2014, and compared its accuracy in predicting pre-eclampsia with the NICE approach.ResultsAmong 12,395 births to nulliparous women in 2011-2014, there were 293 (2.4%) pre-eclampsia events. The WS model included: maternal age, body mass index, ethnicity, multiple pregnancy, family history of pre-eclampsia, autoimmune disease, chronic hypertension and chronic renal disease. In the validation sample (6201 births), the model c-statistic was 0.70 (95% confidence interval 0.65-0.75). The observed:expected ratio for pre-eclampsia was 0.91, with a Hosmer-Lemeshow goodness-of-fit test p-value of 0.20. In the entire study sample of 12,395 births, 374 (3.0%) women had a WS model-estimated pre-eclampsia risk ≥8%, the pre-specified risk-threshold for considering aspirin prophylaxis. Of these, 54 (14.4%) developed pre-eclampsia (sensitivity 18% (14-23), specificity 97% (97-98)). Using the NICE approach, 1173 (9.5%) women were classified as high-risk, of which 107 (9.1%) developed pre-eclampsia (sensitivity 37% (31-42), specificity 91% (91-92)). The final model showed similar accuracy to the NICE approach when using lower risk-threshold of ≥4% to classify women as high-risk for pre-eclampsia.ConclusionThe WS risk model that combines readily-available maternal characteristics achieved modest performance for prediction of pre-eclampsia in nulliparous women. The model did not outperform the NICE approach, but has the advantage of providing individualised absolute risk estimates, to assist with counselling, inform decisions for further testing, and consideration of aspirin prophylaxis.