A Situation Awareness-Based Framework for Design and Evaluation of Explainable AI
Ontology highlight
ABSTRACT:
SUBMITTER: Calvaresi D
PROVIDER: S-EPMC7338174 | biostudies-literature | 2020 Jun
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Safe cycling requires situation awareness (SA), which is the basis for recognizing and anticipating hazards. Children have poorer SA than adults, which may put them at risk. This study investigates whether cyclists' SA can be trained with a video-based learning game. The effect of executive working memory on SA was also studied. Thirty-six children (9-10 years) and 22 adults (21-48 years) played the game. The game had 30 video clips filmed from a cyclist's perspective. Each clip was suddenly masked and two or three locations were presented. The player's task was to choose locations with a potential hazard and feedback was given for their answers. Working memory capacity (WMC) was tested with a counting span task. Children's and adults' performance improved while playing the game, which suggests that playing the game trains SA. Adults performed better than children, and they also glanced at hazards more while the video was playing. Children expectedly had a lower WMC than adults, but WMC did not predict performance within the groups. This indicates that SA does not depend on WMC when passively viewing videos.
| S-EPMC5383826 | biostudies-literature
Project description:Recently, the eXplainable AI (XAI) research community has focused on developing methods making Machine Learning (ML) predictors more interpretable and explainable. Unfortunately, researchers are struggling to converge towards an unambiguous definition of notions such as interpretation, or, explanation—which are often (and mistakenly) used interchangeably. Furthermore, despite the sound metaphors that Multi-Agent System (MAS) could easily provide to address such a challenge, and agent-oriented perspective on the topic is still missing. Thus, this paper proposes an abstract and formal framework for XAI-based MAS, reconciling notions, and results from the literature.
| S-EPMC7338184 | biostudies-literature
Project description:Tree-based machine learning models such as random forests, decision trees, and gradient boosted trees are popular non-linear predictive models, yet comparatively little attention has been paid to explaining their predictions. Here, we improve the interpretability of tree-based models through three main contributions: 1) The first polynomial time algorithm to compute optimal explanations based on game theory. 2) A new type of explanation that directly measures local feature interaction effects. 3) A new set of tools for understanding global model structure based on combining many local explanations of each prediction. We apply these tools to three medical machine learning problems and show how combining many high-quality local explanations allows us to represent global structure while retaining local faithfulness to the original model. These tools enable us to i) identify high magnitude but low frequency non-linear mortality risk factors in the US population, ii) highlight distinct population sub-groups with shared risk characteristics, iii) identify non-linear interaction effects among risk factors for chronic kidney disease, and iv) monitor a machine learning model deployed in a hospital by identifying which features are degrading the model's performance over time. Given the popularity of tree-based machine learning models, these improvements to their interpretability have implications across a broad set of domains.
| S-EPMC7326367 | biostudies-literature
Project description:The authors developed an artificial intelligence (AI)-based algorithm for the design and optimization of a nuclear reactor core based on a flexible geometry and demonstrated a 3× improvement in the selected performance metric: temperature peaking factor. The rapid development of advanced, and specifically, additive manufacturing (3-D printing) and its introduction into advanced nuclear core design through the Transformational Challenge Reactor program have presented the opportunity to explore the arbitrary geometry design of nuclear-heated structures. The primary challenge is that the arbitrary geometry design space is vast and requires the computational evaluation of many candidate designs, and the multiphysics simulation of nuclear systems is very time-intensive. Therefore, the authors developed a machine learning-based multiphysics emulator and evaluated thousands of candidate geometries on Summit, Oak Ridge National Laboratory's leadership class supercomputer. The results presented in this work demonstrate temperature distribution smoothing in a nuclear reactor core through the manipulation of the geometry, which is traditionally achieved in light water reactors through variable assembly loading in the axial direction and fuel shuffling during refueling in the radial direction. The conclusions discuss the future implications for nuclear systems design with arbitrary geometry and the potential for AI-based autonomous design algorithms.
| S-EPMC8490470 | biostudies-literature
Project description:Summary Stable operation of an electric power system requires strict operational limits for the grid frequency. Fluctuations and external impacts can cause large frequency deviations and increased control efforts. Although these complex interdependencies can be modeled using machine learning algorithms, the black box character of many models limits insights and applicability. In this article, we introduce an explainable machine learning model that accurately predicts frequency stability indicators for three European synchronous areas. Using Shapley additive explanations, we identify key features and risk factors for frequency stability. We show how load and generation ramps determine frequency gradients, and we identify three classes of generation technologies with converse impacts. Control efforts vary strongly depending on the grid and time of day and are driven by ramps as well as electricity prices. Notably, renewable power generation is central only in the British grid, while forecasting errors play a major role in the Nordic grid. Graphical abstract Highlights • Power grid frequency stability is analyzed via explainable artificial intelligence• Effect of generation ramps differ between Continental Europe, Britain, Nordic grids• Control efforts are driven by electricity prices and load ramps• Renewable generation and forecasting errors dominate Britain and Nordic grids The bigger picture The transition to a sustainable energy system is challenging for the operation and stability of electric power systems as power generation becomes increasingly uncertain, grid loads increase, and their dynamical properties fundamentally change. At the same time, operational data are available at an unprecedented level of detail, enabling new methods of monitoring and control. To fully harness these data, advanced methods from machine learning must be used. In this paper, we present explainable artificial intelligence (XAI) as a tool to quantify, predict, and explain essential aspects of power system operation and stability in three major European synchronous areas. We focus on the power grid frequency, which measures the balance of generation and load and thus provides the central observable for control and balancing. Combining XAI with domain knowledge, we identify the main drivers and stability risks, while our model and open dataset may enable further XAI research on power systems. The transition to a sustainable energy system is one of the greatest challenges of our time. With operational data becoming available at an unprecedented level of detail, advanced methods from machine learning must be used to fully harness these data.In our article, we present explainable artificial intelligence (XAI) as a tool to quantify, predict, and explain essential aspects of power system operation and stability in three major European areas, giving indications of how to stably operate future power systems.
| S-EPMC8600233 | biostudies-literature
Project description:Alterations in the human microbiome have been observed in a variety of conditions such as asthma, gingivitis, dermatitis and cancer, and much remains to be learned about the links between the microbiome and human health. The fusion of artificial intelligence with rich microbiome datasets can offer an improved understanding of the microbiome's role in human health. To gain actionable insights it is essential to consider both the predictive power and the transparency of the models by providing explanations for the predictions. We combine the collection of leg skin microbiome samples from two healthy cohorts of women with the application of an explainable artificial intelligence (EAI) approach that provides accurate predictions of phenotypes with explanations. The explanations are expressed in terms of variations in the relative abundance of key microbes that drive the predictions. We predict skin hydration, subject's age, pre/post-menopausal status and smoking status from the leg skin microbiome. The changes in microbial composition linked to skin hydration can accelerate the development of personalized treatments for healthy skin, while those associated with age may offer insights into the skin aging process. The leg microbiome signatures associated with smoking and menopausal status are consistent with previous findings from oral/respiratory tract microbiomes and vaginal/gut microbiomes respectively. This suggests that easily accessible microbiome samples could be used to investigate health-related phenotypes, offering potential for non-invasive diagnosis and condition monitoring. Our EAI approach sets the stage for new work focused on understanding the complex relationships between microbial communities and phenotypes. Our approach can be applied to predict any condition from microbiome samples and has the potential to accelerate the development of microbiome-based personalized therapeutics and non-invasive diagnostics.
| S-EPMC7907326 | biostudies-literature
Project description:BACKGROUND:Studies show that implementing huddles in healthcare can improve a variety of outcomes. Yet little is known about the mechanisms through which huddles exert their effects. To help remedy this gap, our study objectives were to explore hospital administrator and frontline staff perspectives on the benefits and challenges of implementing a tiered huddle system; and propose a model based on our findings depicting the mediating pathways through which implementing a huddle system may reduce patient harm. METHODS:Using qualitative methods, we conducted semi-structured interviews and focus groups to obtain a deeper understanding of the huddle system and its outcomes as implemented in an academic tertiary care children's hospital with 539 inpatient beds. We recruited healthcare providers representing all levels using a snowball sampling technique (10 interviews), and emails, flyers, and paper invitations (six focus groups). We transcribed recordings and analysed the data using established techniques. RESULTS:Five themes emerged and provided the foundational constructs of our model. Specifically we propose that huddle implementation leads to improved efficiencies and quality of information sharing, increased levels of accountability, empowerment, and sense of community, which together create a culture of collaboration and collegiality that increases the staff's quality of collective awareness and enhanced capacity for eliminating patient harm. CONCLUSIONS:While each construct in the proposed model is itself a beneficial outcome of implementing huddles, conceptualising the pathways by which they may work allows us to design ways to evaluate other huddle implementation efforts designed to help reduce failures and eliminate patient harm.
| S-EPMC6288816 | biostudies-other
Project description:BackgroundAdverse drug reactions (ADRs) are statistically characterized within randomized clinical trials and postmarketing pharmacovigilance, but their molecular mechanism remains unknown in most cases. This is true even for hepatic or skin toxicities, which are classically monitored during drug design. Aside from clinical trials, many elements of knowledge about drug ingredients are available in open-access knowledge graphs, such as their properties, interactions, or involvements in pathways. In addition, drug classifications that label drugs as either causative or not for several ADRs, have been established.MethodsWe propose in this paper to mine knowledge graphs for identifying biomolecular features that may enable automatically reproducing expert classifications that distinguish drugs causative or not for a given type of ADR. In an Explainable AI perspective, we explore simple classification techniques such as Decision Trees and Classification Rules because they provide human-readable models, which explain the classification itself, but may also provide elements of explanation for molecular mechanisms behind ADRs. In summary, (1) we mine a knowledge graph for features; (2) we train classifiers at distinguishing, on the basis of extracted features, drugs associated or not with two commonly monitored ADRs: drug-induced liver injuries (DILI) and severe cutaneous adverse reactions (SCAR); (3) we isolate features that are both efficient in reproducing expert classifications and interpretable by experts (i.e., Gene Ontology terms, drug targets, or pathway names); and (4) we manually evaluate in a mini-study how they may be explanatory.ResultsExtracted features reproduce with a good fidelity classifications of drugs causative or not for DILI and SCAR (Accuracy = 0.74 and 0.81, respectively). Experts fully agreed that 73% and 38% of the most discriminative features are possibly explanatory for DILI and SCAR, respectively; and partially agreed (2/3) for 90% and 77% of them.ConclusionKnowledge graphs provide sufficiently diverse features to enable simple and explainable models to distinguish between drugs that are causative or not for ADRs. In addition to explaining classifications, most discriminative features appear to be good candidates for investigating ADR mechanisms further.
| S-EPMC8157660 | biostudies-literature
Project description:Research into Explainable Artificial Intelligence (XAI) has been increasing in recent years as a response to the need for increased transparency and trust in AI. This is particularly important as AI is used in sensitive domains with societal, ethical, and safety implications. Work in XAI has primarily focused on Machine Learning (ML) for classification, decision, or action, with detailed systematic reviews already undertaken. This review looks to explore current approaches and limitations for XAI in the area of Reinforcement Learning (RL). From 520 search results, 25 studies (including 5 snowball sampled) are reviewed, highlighting visualization, query-based explanations, policy summarization, human-in-the-loop collaboration, and verification as trends in this area. Limitations in the studies are presented, particularly a lack of user studies, and the prevalence of toy-examples and difficulties providing understandable explanations. Areas for future study are identified, including immersive visualization, and symbolic representation.
| S-EPMC8172805 | biostudies-literature
Project description:BackgroundA mechanistic understanding of the spread of SARS-CoV-2 and diligent tracking of ongoing mutagenesis are of key importance to plan robust strategies for confining its transmission. Large numbers of available sequences and their dates of transmission provide an unprecedented opportunity to analyze evolutionary adaptation in novel ways. Addition of high-resolution structural information can reveal the functional basis of these processes at the molecular level. Integrated systems biology-directed analyses of these data layers afford valuable insights to build a global understanding of the COVID-19 pandemic.ResultsHere we identify globally distributed haplotypes from 15,789 SARS-CoV-2 genomes and model their success based on their duration, dispersal, and frequency in the host population. Our models identify mutations that are likely compensatory adaptive changes that allowed for rapid expansion of the virus. Functional predictions from structural analyses indicate that, contrary to previous reports, the Asp614Gly mutation in the spike glycoprotein (S) likely reduced transmission and the subsequent Pro323Leu mutation in the RNA-dependent RNA polymerase led to the precipitous spread of the virus. Our model also suggests that two mutations in the nsp13 helicase allowed for the adaptation of the virus to the Pacific Northwest of the USA. Finally, our explainable artificial intelligence algorithm identified a mutational hotspot in the sequence of S that also displays a signature of positive selection and may have implications for tissue or cell-specific expression of the virus.ConclusionsThese results provide valuable insights for the development of drugs and surveillance strategies to combat the current and future pandemics.
| S-EPMC7756312 | biostudies-literature