Dataset Information

A hierarchical expert-guided machine learning framework for clinical decision support systems: an application to traumatic brain injury prognostication.

ABSTRACT: Prognosis of the long-term functional outcome of traumatic brain injury is essential for personalized management of that injury. Nonetheless, accurate prediction remains unavailable. Although machine learning has shown promise in many fields, including medical diagnosis and prognosis, such models are rarely deployed in real-world settings due to a lack of transparency and trustworthiness. To address these drawbacks, we propose a machine learning-based framework that is explainable and aligns with clinical domain knowledge. To build such a framework, additional layers of statistical inference and human expert validation are added to the model, which ensures the predicted risk score's trustworthiness. Using 831 patients with moderate or severe traumatic brain injury to build a model using the proposed framework, an area under the receiver operating characteristic curve (AUC) and accuracy of 0.8085 and 0.7488 were achieved, respectively, in determining which patients will experience poor functional outcomes. The performance of the machine learning classifier is not adversely affected by the imposition of statistical and domain knowledge "checks and balances". Finally, through a case study, we demonstrate how the decision made by a model might be biased if it is not audited carefully.

SUBMITTER: Farzaneh N

PROVIDER: S-EPMC8105342 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundClinical decision support systems are designed to utilize medical data, knowledge, and analysis engines and to generate patient-specific assessments or recommendations to health professionals in order to assist decision making. Artificial intelligence-enabled clinical decision support systems aid the decision-making process through an intelligent component. Well-defined evaluation methods are essential to ensure the seamless integration and contribution of these systems to clinical practice.ObjectiveThe purpose of this study was to develop and validate a measurement instrument and test the interrelationships of evaluation variables for an artificial intelligence-enabled clinical decision support system evaluation framework.MethodsAn artificial intelligence-enabled clinical decision support system evaluation framework consisting of 6 variables was developed. A Delphi process was conducted to develop the measurement instrument items. Cognitive interviews and pretesting were performed to refine the questions. Web-based survey response data were analyzed to remove irrelevant questions from the measurement instrument, to test dimensional structure, and to assess reliability and validity. The interrelationships of relevant variables were tested and verified using path analysis, and a 28-item measurement instrument was developed. Measurement instrument survey responses were collected from 156 respondents.ResultsThe Cronbach α of the measurement instrument was 0.963, and its content validity was 0.943. Values of average variance extracted ranged from 0.582 to 0.756, and values of the heterotrait-monotrait ratio ranged from 0.376 to 0.896. The final model had a good fit (χ262=36.984; P=.08; comparative fit index 0.991; goodness-of-fit index 0.957; root mean square error of approximation 0.052; standardized root mean square residual 0.028). Variables in the final model accounted for 89% of the variance in the user acceptance dimension.ConclusionsUser acceptance is the central dimension of artificial intelligence-enabled clinical decision support system success. Acceptance was directly influenced by perceived ease of use, information quality, service quality, and perceived benefit. Acceptance was also indirectly influenced by system quality and information quality through perceived ease of use. User acceptance and perceived benefit were interrelated.

Project description:ImportanceAn increasing number of machine learning (ML)-based clinical decision support systems (CDSSs) are described in the medical literature, but this research focuses almost entirely on comparing CDSS directly with clinicians (human vs computer). Little is known about the outcomes of these systems when used as adjuncts to human decision-making (human vs human with computer).ObjectivesTo conduct a systematic review to investigate the association between the interactive use of ML-based diagnostic CDSSs and clinician performance and to examine the extent of the CDSSs' human factors evaluation.Evidence reviewA search of MEDLINE, Embase, PsycINFO, and grey literature was conducted for the period between January 1, 2010, and May 31, 2019. Peer-reviewed studies published in English comparing human clinician performance with and without interactive use of an ML-based diagnostic CDSSs were included. All metrics used to assess human performance were considered as outcomes. The risk of bias was assessed using Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) and Risk of Bias in Non-Randomised Studies-Intervention (ROBINS-I). Narrative summaries were produced for the main outcomes. Given the heterogeneity of medical conditions, outcomes of interest, and evaluation metrics, no meta-analysis was performed.FindingsA total of 8112 studies were initially retrieved and 5154 abstracts were screened; of these, 37 studies met the inclusion criteria. The median number of participating clinicians was 4 (interquartile range, 3-8). Of the 107 results that reported statistical significance, 54 (50%) were increased by the use of CDSSs, 4 (4%) were decreased, and 49 (46%) showed no change or an unclear change. In the subgroup of studies carried out in representative clinical settings, no association between the use of ML-based diagnostic CDSSs and improved clinician performance could be observed. Interobserver agreement was the commonly reported outcome whose change was the most strongly associated with CDSS use. Four studies (11%) reported on user feedback, and, in all but 1 case, clinicians decided to override at least some of the algorithms' recommendations. Twenty-eight studies (76%) were rated as having a high risk of bias in at least 1 of the 4 QUADAS-2 core domains, and 6 studies (16%) were considered to be at serious or critical risk of bias using ROBINS-I.Conclusions and relevanceThis systematic review found only sparse evidence that the use of ML-based CDSSs is associated with improved clinician diagnostic performance. Most studies had a low number of participants, were at high or unclear risk of bias, and showed little or no consideration for human factors. Caution should be exercised when estimating the current potential of ML to improve human diagnostic performance, and more comprehensive evaluation should be conducted before deploying ML-based CDSSs in clinical settings. The results highlight the importance of considering supported human decisions as end points rather than merely the stand-alone CDSSs outputs.

Project description:BackgroundCurrent obstructive sleep apnea treatment relies on manual PAP titration, but it has limitations. Complex interactions during titration and variations in SpO2 data accuracy pose challenges. Patients with co-occurring chronic hypercapnia may require precise oxygen titration. To address these issues, we propose a Clinical Decision Support System using Markov decision processes.MethodsThis study, compliant with data protection laws, focused on adults with OSA-induced hypoxemia utilizing supplemental oxygen and CPAP/BiPAP therapy. PAP titration, conducted over one night, involved vigilant monitoring of vital signs and physiological parameters. Adjustments to CPAP pressure, potential BiLevel transitions, and supplemental oxygen were precisely guided by patient metrics. Markov decision processes outlined three treatment actions for disorder management, incorporating expert medical insights.ResultsIn our study involving 14 OSA patients (average age: 63 years, 27% females, BMI 41 kg m-2), significant improvements were observed in key health parameters after manual titration. The initial AHI of 61.8 events per hour significantly decreased to an average of 18.0 events per hour after PAP and oxygen titration (p < 0.0001), indicating a substantial reduction in sleep-disordered breathing severity. Concurrently, SpO2 levels increased significantly from an average of 79.7% before titration to 89.1% after titration (p < 0.0003). Pearson correlation coefficients demonstrated aggravation of hypercapnia in 50% of patients (N = 5) with initial pCO2 < 55 mmHg during the increase in CPAP pressure. However, transitioning to BiPAP exhibited a reduction in pCO2 levels, showcasing its efficacy in addressing hypercapnia. Simultaneously, BiPAP therapy correlated with a substantial increase in SpO2, underscoring its positive impact on oxygenation in OSA patients. Markov Decision Process analysis demonstrated realistic patient behavior during stable night conditions, emphasizing minimal apnea and good toleration to high CPAP pressure.ConclusionsThe development of a framework for Markov decision processes of PAP and oxygen titration algorithms holds promise for providing algorithms for improving pCO2 and SpO2 values. While challenges remain, including the need for high-quality data, the potential benefits in terms of patient management and care optimization are substantial, and this approach represents an exciting frontier in the realm of telemedicine and respiratory healthcare.

Project description:Advanced Computerized Decision Support Systems (CDSSs) assist clinicians in their decision-making process, generating recommendations based on up-to-date scientific evidence. Although this technology has the potential to improve the quality of patient care, its mere provision does not guarantee uptake: even where CDSSs are available, clinicians often fail to adopt their recommendations. This study examines the barriers and facilitators to the uptake of an evidence-based CDSS as perceived by diverse health professionals in hospitals at different stages of CDSS adoption.Qualitative study conducted as part of a series of randomized controlled trials of CDSSs. The sample includes two hospitals using a CDSS and two hospitals that aim to adopt a CDSS in the future. We interviewed physicians, nurses, information technology staff, and members of the boards of directors (n = 30). We used a constant comparative approach to develop a framework for guiding implementation.We identified six clusters of experiences of, and attitudes towards CDSSs, which we label as "positions." The six positions represent a gradient of acquisition of control over CDSSs (from low to high) and are characterized by different types of barriers to CDSS uptake. The most severe barriers (prevalent in the first positions) include clinicians' perception that the CDSSs may reduce their professional autonomy or may be used against them in the event of medical-legal controversies. Moving towards the last positions, these barriers are substituted by technical and usability problems related to the technology interface. When all barriers are overcome, CDSSs are perceived as a working tool at the service of its users, integrating clinicians' reasoning and fostering organizational learning.Barriers and facilitators to the use of CDSSs are dynamic and may exist prior to their introduction in clinical contexts; providing a static list of obstacles and facilitators, irrespective of the specific implementation phase and context, may not be sufficient or useful to facilitate uptake. Factors such as clinicians' attitudes towards scientific evidences and guidelines, the quality of inter-disciplinary relationships, and an organizational ethos of transparency and accountability need to be considered when exploring the readiness of a hospital to adopt CDSSs.

Project description:BackgroundThe clinical mitigation of intracranial hypertension due to traumatic brain injury requires timely knowledge of intracranial pressure to avoid secondary injury or death. Noninvasive intracranial pressure (nICP) estimation that operates sufficiently fast at multihour timescales and requires only common patient measurements is a desirable tool for clinical decision support and improving traumatic brain injury patient outcomes. However, existing model-based nICP estimation methods may be too slow or require data that are not easily obtained.ObjectiveThis work considers short- and real-time nICP estimation at multihour timescales based on arterial blood pressure (ABP) to better inform the ongoing development of practical models with commonly available data.MethodsWe assess and analyze the effects of two distinct pathways of model development, either by increasing physiological integration using a simple pressure estimation model, or by increasing physiological fidelity using a more complex model. Comparison of the model approaches is performed using a set of quantitative model validation criteria over hour-scale times applied to model nICP estimates in relation to observed ICP.ResultsThe simple fully coupled estimation scheme based on windowed regression outperforms a more complex nICP model with prescribed intracranial inflow when pulsatile ABP inflow conditions are provided. We also show that the simple estimation data requirements can be reduced to 1-minute averaged ABP summary data under generic waveform representation.ConclusionsStronger performance of the simple bidirectional model indicates that feedback between the systemic vascular network and nICP estimation scheme is crucial for modeling over long intervals. However, simple model reduction to ABP-only dependence limits its utility in cases involving other brain injuries such as ischemic stroke and subarachnoid hemorrhage. Additional methodologies and considerations needed to overcome these limitations are illustrated and discussed.

Project description:For supporting clinical decision-making in audiology, Common Audiological Functional Parameters (CAFPAs) were suggested as an interpretable intermediate representation of audiological information taken from various diagnostic sources within a clinical decision-support system (CDSS). Ten different CAFPAs were proposed to represent specific functional aspects of the human auditory system, namely hearing threshold, supra-threshold deficits, binaural hearing, neural processing, cognitive abilities, and a socio-economic component. CAFPAs were established as a viable basis for deriving audiological findings and treatment recommendations, and it has been demonstrated that model-predicted CAFPAs, with machine learning models trained on expert-labeled patient cases, are sufficiently accurate to be included in a CDSS, but it requires further validation by experts. The present study aimed to validate model-predicted CAFPAs based on previously unlabeled cases from the same data set. Here, we ask to which extent domain experts agree with the model-predicted CAFPAs and whether potential disagreement can be understood in terms of patient characteristics. To these aims, an expert survey was designed and applied to two highly-experienced audiology specialists. They were asked to evaluate model-predicted CAFPAs and estimate audiological findings of the given audiological information about the patients that they were presented with simultaneously. The results revealed strong relative agreement between the two experts and importantly between experts and the prediction for all CAFPAs, except for the neural processing and binaural hearing-related ones. It turned out, however, that experts tend to score CAFPAs in a larger value range, but, on average, across patients with smaller scores as compared with the machine learning models. For the hearing threshold-associated CAFPA in frequencies smaller than 0.75 kHz and the cognitive CAFPA, not only the relative agreement but also the absolute agreement between machine and experts was very high. For those CAFPAs with an average difference between the model- and expert-estimated values, patient characteristics were predictive of the disagreement. The findings are discussed in terms of how they can help toward further improvement of model-predicted CAFPAs to be incorporated in a CDSS for audiology.

Dataset Information

A hierarchical expert-guided machine learning framework for clinical decision support systems: an application to traumatic brain injury prognostication.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets