Dataset Information

Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19.

ABSTRACT: A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model's performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature's SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.

SUBMITTER: Duckworth C

PROVIDER: S-EPMC8626460 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundEmergency departments (EDs) see a rising number of patients, but only a small fraction of ED patients need immediate intensive care. The characteristics of these patients are mostly unknown and there is reason to believe that there are large inter-hospital differences in thresholds for intensive care admissions from the ED. The purpose of this study was to give a nationwide overview of ED admissions directly to intensive care units.MethodsWe used the Swedish Intensive care Registry to identify all patients admitted to intensive care from the ED (January 1, 2013 until June 7, 2018). The primary outcome was discharge diagnosis after intensive care (primary ICU diagnosis code). ICU mortality and" ICU admission due to only observation" were analyzed as secondary outcomes.Results110,072 ICU admissions were included, representing 94,546 unique patients. Intoxication, trauma and neurological conditions were the most common causes for intensive care, but large variations according to age, sex and hospital type were seen. Intoxication was the most prevalent diagnosis in young adults (46.8% of admissions in 18-29 years old), whereas infectious diseases predominated in the elderly (17.0% in 65-79 years old). Overall, ICU mortality was 7.2%, but varied substantially with age, sex, type of hospital and medical condition. Cardiac conditions had the highest mortality rates, reaching 32.9%. The mortality was higher in academic centers compared to rural hospitals (9.3% vs 5.0%). It was more common to be admitted to ICU for only observation in rural hospitals than in academic centers (20.1% vs 7.8%). Being admitted to ICU only for observation was most common in patients with intoxication (30.6%).ConclusionsOverall, intoxication was the most common cause for ICU admission from the ED. However, causes of ED ICU admissions differ substantially according to age, sex and hospital type. Being admitted to the ICU only for observation was most common in intoxicated patients.Trial registrationNot applicable (no interventions).

Project description:BACKGROUND: Emergency department (ED) visit and hospital admissions (HA) data have been an indispensible resource for assessing acute morbidity impacts of air pollution. ED visits and HAs are types of health care visits with similarities, but also potentially important differences. Little previous information is available regarding the impact of health care visit type on observed acute air pollution-health associations from studies conducted for the same location, time period, outcome definitions and model specifications. METHODS: As part of a broader study of air pollution and health in St. Louis, individual-level ED and HA data were obtained for a 6.5 year period for acute care hospitals in the eight Missouri counties of the St. Louis metropolitan area. Patient demographic characteristics and diagnostic code distributions were compared for four visit types including ED visits, HAs, HAs that came through the ED, and non-elective HAs. Time-series analyses of the relationship between daily ambient ozone and PM?.? and selected cardiorespiratory outcomes were conducted for each visit type. RESULTS: Our results indicate that, compared with ED patients, HA patients tended to be older, had evidence of greater severity for some outcomes, and had a different mix of specific outcomes. Consideration of 'HA through ED' appeared to more effectively select acute visits than consideration of 'non-elective HA'. While outcomes with the strongest observed temporal associations with air pollutants tended to show strong associations for all visit types, we found some differences in observed associations for ED visits and HAs. For example, risk ratios for the respiratory disease-ozone association were 1.020 for ED visits and 1.004 for 'HA through ED'; risk ratios for the asthma/wheeze-ozone association were 1.069 for ED visits and 1.106 for 'HA through ED'. Several factors (e.g. age) were identified that may be responsible, in part, for the differences in observed associations. CONCLUSIONS: Demographic and diagnostic differences between visit types may lead to preference for one visit type over another for some questions and populations. The strengths of observed associations with air pollutants sometimes varied between different health care visit types, but the relative strengths of association generally were specific to the pollutant-outcome combination.

Dataset Information

Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets