Dataset Information

A Quantitative Evaluation of Global, Rule-Based Explanations of Post-Hoc, Model Agnostic Methods.

ABSTRACT: Understanding the inferences of data-driven, machine-learned models can be seen as a process that discloses the relationships between their input and output. These relationships consist and can be represented as a set of inference rules. However, the models usually do not explicit these rules to their end-users who, subsequently, perceive them as black-boxes and might not trust their predictions. Therefore, scholars have proposed several methods for extracting rules from data-driven machine-learned models to explain their logic. However, limited work exists on the evaluation and comparison of these methods. This study proposes a novel comparative approach to evaluate and compare the rulesets produced by five model-agnostic, post-hoc rule extractors by employing eight quantitative metrics. Eventually, the Friedman test was employed to check whether a method consistently performed better than the others, in terms of the selected metrics, and could be considered superior. Findings demonstrate that these metrics do not provide sufficient evidence to identify superior methods over the others. However, when used together, these metrics form a tool, applicable to every rule-extraction method and machine-learned models, that is, suitable to highlight the strengths and weaknesses of the rule-extractors in various applications in an objective and straightforward manner, without any human interventions. Thus, they are capable of successfully modelling distinctively aspects of explainability, providing to researchers and practitioners vital insights on what a model has learned during its training process and how it makes its predictions.

SUBMITTER: Vilone G

PROVIDER: S-EPMC8596373 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundComputational modeling is an important tool for the study of complex biochemical processes associated with cell signaling networks. However, it is challenging to simulate processes that involve hundreds of large molecules due to the high computational cost of such simulations. Rule-based modeling is a method that can be used to simulate these processes with reasonably low computational cost, but traditional rule-based modeling approaches do not include details of molecular geometry. The incorporation of geometry into biochemical models can more accurately capture details of these processes, and may lead to insights into how geometry affects the products that form. Furthermore, geometric rule-based modeling can be used to complement other computational methods that explicitly represent molecular geometry in order to quantify binding site accessibility and steric effects.ResultsWe propose a novel implementation of rule-based modeling that encodes details of molecular geometry into the rules and binding rates. We demonstrate how rules are constructed according to the molecular curvature. We then perform a study of antigen-antibody aggregation using our proposed method. We simulate the binding of antibody complexes to binding regions of the shrimp allergen Pen a 1 using a previously developed 3D rigid-body Monte Carlo simulation, and we analyze the aggregate sizes. Then, using our novel approach, we optimize a rule-based model according to the geometry of the Pen a 1 molecule and the data from the Monte Carlo simulation. We use the distances between the binding regions of Pen a 1 to optimize the rules and binding rates. We perform this procedure for multiple conformations of Pen a 1 and analyze the impact of conformation and resolution on the optimal rule-based model.ConclusionsWe find that the optimized rule-based models provide information about the average steric hindrance between binding regions and the probability that antibodies will bind to these regions. These optimized models quantify the variation in aggregate size that results from differences in molecular geometry and from model resolution.

Project description:BackgroundGlobal health education partnerships should be collaborative and reciprocal to ensure mutual benefit. Utilisation of digital technologies can overcome geographic boundaries and facilitate collaborative global health learning. Global Health Classroom (GHCR) is a collaborative global health learning model involving medical students from different countries learning about each other's health systems, cultures, and determinants of health via videoconference. Principles of reciprocity and interinstitutional partnership informed the development of the GHCR. This study explores learning outcomes and experiences in the GHCR between students from New Zealand and Samoa.MethodsThis study used a mixed methods approach employing post-GHCR questionnaires and semi-structured face-to-face interviews to explore self-reported learning and experiences among medical students in the GHCR. The GHCR collaboration studied was between the medical schools at the University of Otago, New Zealand and the National University of Samoa, Samoa.ResultsQuestionnaire response rate was 85% (74/87). Nineteen interviews were conducted among New Zealand and Samoan students. Students reported acquiring the intended learning outcomes relating to patient care, health systems, culture, and determinants of health with regards to their partner country. Interview data was indicative of attitudinal changes in relation to cultural humility and curiosity. Some reported a vision for progress regarding their own health system. Students in the GHCR reported that learning with their international peers in the virtual classroom made learning about global health more real and tangible. The benefits to students from both countries indicated reciprocity.ConclusionsThis study demonstrates GHCR to be a promising model for collaborative and reciprocal global health learning using a student-led format and employing digital technology to create a virtual classroom. The self-reported learning outcomes align favourably with those recommended in the literature. In view of our positive findings, we present GHCR as an adaptable model for equitable, collaborative global health learning between students in internationally partnered institutions.

Dataset Information

A Quantitative Evaluation of Global, Rule-Based Explanations of Post-Hoc, Model Agnostic Methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets