Dataset Information

Development of a clinical decision support system for severity risk prediction and triage of COVID-19 patients at hospital admission: an international multicentre study.

ABSTRACT:

Background

The outbreak of coronavirus disease 2019 (COVID-19) has globally strained medical resources and caused significant mortality.

Objective

To develop and validate a machine-learning model based on clinical features for severity risk assessment and triage for COVID-19 patients at hospital admission.

Method

725 patients were used to train and validate the model. This included a retrospective cohort from Wuhan, China of 299 hospitalised COVID-19 patients from 23 December 2019 to 13 February 2020, and five cohorts with 426 patients from eight centres in China, Italy and Belgium from 20 February 2020 to 21 March 2020. The main outcome was the onset of severe or critical illness during hospitalisation. Model performances were quantified using the area under the receiver operating characteristic curve (AUC) and metrics derived from the confusion matrix.

Results

In the retrospective cohort, the median age was 50 years and 137 (45.8%) were male. In the five test cohorts, the median age was 62 years and 236 (55.4%) were male. The model was prospectively validated on five cohorts yielding AUCs ranging from 0.84 to 0.93, with accuracies ranging from 74.4% to 87.5%, sensitivities ranging from 75.0% to 96.9%, and specificities ranging from 55.0% to 88.0%, most of which performed better than the pneumonia severity index. The cut-off values of the low-, medium- and high-risk probabilities were 0.21 and 0.80. The online calculators can be found at www.covid19risk.ai.

Conclusion

The machine-learning model, nomogram and online calculator might be useful to access the onset of severe and critical illness among COVID-19 patients and triage at hospital admission.

SUBMITTER: Wu G

PROVIDER: S-EPMC7331655 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundComputerized clinical decision support systems (CDSSs) are increasingly adopted in health care to optimize resources and streamline patient flow. However, they often lack scientific validation against standard medical care.ObjectiveThe purpose of this study was to assess the performance, safety, and usability of a CDSS in a university hospital emergency department setting in Kuopio, Finland.MethodsPatients entering the emergency department were asked to voluntarily participate in this study. Patients aged 17 years or younger, patients with cognitive impairments, and patients who entered the unit in an ambulance or with the need for immediate care were excluded. Patients completed the CDSS web-based form and usability questionnaire when waiting for the triage nurse's evaluation. The CDSS data were anonymized and did not affect the patients' usual evaluation or treatment. Retrospectively, 2 medical doctors evaluated the urgency of each patient's condition by using the triage nurse's information, and urgent and nonurgent groups were created. The International Statistical Classification of Diseases, Tenth Revision diagnoses were collected from the electronic health records. Usability was assessed by using a positive version of the System Usability Scale questionnaire.ResultsIn total, our analyses included 248 patients. Regarding urgency, the mean sensitivities were 85% and 19%, respectively, for urgent and nonurgent cases when assessing the performance of CDSS evaluations in comparison to that of physicians. The mean sensitivities were 85% and 35%, respectively, when comparing the evaluations between the two physicians. Our CDSS did not miss any cases that were evaluated to be emergencies by physicians; thus, all emergency cases evaluated by physicians were evaluated as either urgent cases or emergency cases by the CDSS. In differential diagnosis, the CDSS had an exact match accuracy of 45.5% (97/213). The usability was good, with a mean System Usability Scale score of 78.2 (SD 16.8).ConclusionsIn a university hospital emergency department setting with a large real-world population, our CDSS was found to be equally as sensitive in urgent patient cases as physicians and was found to have an acceptable differential diagnosis accuracy, with good usability. These results suggest that this CDSS can be safely assessed further in a real-world setting. A CDSS could accelerate triage by providing patient-provided data in advance of patients' initial consultations and categorize patient cases as urgent and nonurgent cases upon patients' arrival to the emergency department.

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BACKGROUND:Several approaches to medication optimisation by identifying drug-related problems in older people have been described. Although some interventions have shown reductions in drug-related problems (DRPs), evidence supporting the effectiveness of medication reviews on clinical and economic outcomes is lacking. Application of the STOPP/START (version 2) explicit screening tool for inappropriate prescribing has decreased inappropriate prescribing and significantly reduced adverse drug reactions (ADRs) and associated healthcare costs in older patients with multi-morbidity and polypharmacy. Therefore, application of STOPP/START criteria during a medication review is likely to be beneficial. Incorporation of explicit screening tools into clinical decision support systems (CDSS) has gained traction as a means to improve both quality and efficiency in the rather time-consuming medication review process. Although CDSS can generate more potential inappropriate medication recommendations, some of these have been shown to be less clinically relevant, resulting in alert fatigue. Moreover, explicit tools such as STOPP/START do not cover all relevant DRPs on an individual patient level. The OPERAM study aims to assess the impact of a structured drug review on the quality of pharmacotherapy in older people with multi-morbidity and polypharmacy. The aim of this paper is to describe the structured, multi-component intervention of the OPERAM trial and compare it with the approach in the comparator arm. METHOD:This paper describes a multi-component intervention, integrating interventions that have demonstrated effectiveness in defining DRPs. The intervention involves a structured history-taking of medication (SHiM), a medication review according to the systemic tool to reduce inappropriate prescribing (STRIP) method, assisted by a clinical decision support system (STRIP Assistant, STRIPA) with integrated STOPP/START criteria (version 2), followed by shared decision-making with both patient and attending physician. The developed method integrates patient input, patient data, involvement from other healthcare professionals and CDSS-assistance into one structured intervention. DISCUSSION:The clinical and economical effectiveness of this experimental intervention will be evaluated in a cohort of hospitalised, older patients with multi-morbidity and polypharmacy in the multicentre, randomized controlled OPERAM trial (OPtimising thERapy to prevent Avoidable hospital admissions in the Multi-morbid elderly), which will be completed in the last quarter of 2019. TRIAL REGISTRATION:Universal Trial Number: U1111-1181-9400 Clinicaltrials.gov: NCT02986425, Registered 08 December 2016. FOPH (Swiss national portal): SNCTP000002183. Netherlands Trial Register: NTR6012 (07-10-2016).

Project description:ObjectivesCOVID-19 causes lung parenchymal and endothelial damage that lead to hypoxic acute respiratory failure (hARF). The influence of hARF severity on patients' outcomes is still poorly understood.DesignObservational, prospective, multicentre study.SettingThree academic hospitals in Milan (Italy) involving three respiratory high dependency units and three general wards.ParticipantsConsecutive adult hospitalised patients with a virologically confirmed diagnosis of COVID-19. Patients aged <18 years or unable to provide informed consent were excluded.InterventionsAnthropometrical, clinical characteristics and blood biomarkers were assessed within the first 24 hours from admission. hARF was graded as follows: severe (partial pressure of oxygen to fraction of inspired oxygen ratio (PaO2/FiO2) <100 mm Hg); moderate (PaO2/FiO2 101-200 mm Hg); mild (PaO2/FiO2 201-300 mm Hg) and normal (PaO2/FiO2 >300 mm Hg).Primary and secondary outcome measuresThe primary outcome was the assessment of clinical characteristics and in-hospital mortality based on the severity of respiratory failure. Secondary outcomes were intubation rate and application of continuous positive airway pressure during hospital stay.Results412 patients were enrolled (280 males, 68%). Median (IQR) age was 66 (55-76) years with a PaO2/FiO2 at admission of 262 (140-343) mm Hg. 50.2% had a cardiovascular disease. Prevalence of mild, moderate and severe hARF was 24.4%, 21.9% and 15.5%, respectively. In-hospital mortality proportionally increased with increasing impairment of gas exchange (p<0.001). The only independent risk factors for mortality were age ≥65 years (HR 3.41; 95% CI 2.00 to 5.78, p<0.0001), PaO2/FiO2 ratio ≤200 mm Hg (HR 3.57; 95% CI 2.20 to 5.77, p<0.0001) and respiratory failure at admission (HR 3.58; 95% CI 1.05 to 12.18, p=0.04).ConclusionsA moderate-to-severe impairment in PaO2/FiO2 was independently associated with a threefold increase in risk of in-hospital mortality. Severity of respiratory failure is useful to identify patients at higher risk of mortality.Trial registration numberNCT04307459.

Project description:For supporting clinical decision-making in audiology, Common Audiological Functional Parameters (CAFPAs) were suggested as an interpretable intermediate representation of audiological information taken from various diagnostic sources within a clinical decision-support system (CDSS). Ten different CAFPAs were proposed to represent specific functional aspects of the human auditory system, namely hearing threshold, supra-threshold deficits, binaural hearing, neural processing, cognitive abilities, and a socio-economic component. CAFPAs were established as a viable basis for deriving audiological findings and treatment recommendations, and it has been demonstrated that model-predicted CAFPAs, with machine learning models trained on expert-labeled patient cases, are sufficiently accurate to be included in a CDSS, but it requires further validation by experts. The present study aimed to validate model-predicted CAFPAs based on previously unlabeled cases from the same data set. Here, we ask to which extent domain experts agree with the model-predicted CAFPAs and whether potential disagreement can be understood in terms of patient characteristics. To these aims, an expert survey was designed and applied to two highly-experienced audiology specialists. They were asked to evaluate model-predicted CAFPAs and estimate audiological findings of the given audiological information about the patients that they were presented with simultaneously. The results revealed strong relative agreement between the two experts and importantly between experts and the prediction for all CAFPAs, except for the neural processing and binaural hearing-related ones. It turned out, however, that experts tend to score CAFPAs in a larger value range, but, on average, across patients with smaller scores as compared with the machine learning models. For the hearing threshold-associated CAFPA in frequencies smaller than 0.75 kHz and the cognitive CAFPA, not only the relative agreement but also the absolute agreement between machine and experts was very high. For those CAFPAs with an average difference between the model- and expert-estimated values, patient characteristics were predictive of the disagreement. The findings are discussed in terms of how they can help toward further improvement of model-predicted CAFPAs to be incorporated in a CDSS for audiology.