Dataset Information

Adaptive trust calibration for human-AI collaboration.

ABSTRACT: Safety and efficiency of human-AI collaboration often depend on how humans could appropriately calibrate their trust towards the AI agents. Over-trusting the autonomous system sometimes causes serious safety issues. Although many studies focused on the importance of system transparency in keeping proper trust calibration, the research in detecting and mitigating improper trust calibration remains very limited. To fill these research gaps, we propose a method of adaptive trust calibration that consists of a framework for detecting the inappropriate calibration status by monitoring the user's reliance behavior and cognitive cues called "trust calibration cues" to prompt the user to reinitiate trust calibration. We evaluated our framework and four types of trust calibration cues in an online experiment using a drone simulator. A total of 116 participants performed pothole inspection tasks by using the drone's automatic inspection, the reliability of which could fluctuate depending upon the weather conditions. The participants needed to decide whether to rely on automatic inspection or to do the inspection manually. The results showed that adaptively presenting simple cues could significantly promote trust calibration during over-trust.

SUBMITTER: Okamura K

PROVIDER: S-EPMC7034851 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Adaptive trust calibration for human-AI collaboration.

Okamura Kazuo K Yamada Seiji S

PloS one 20200221 2

Safety and efficiency of human-AI collaboration often depend on how humans could appropriately calibrate their trust towards the AI agents. Over-trusting the autonomous system sometimes causes serious safety issues. Although many studies focused on the importance of system transparency in keeping proper trust calibration, the research in detecting and mitigating improper trust calibration remains very limited. To fill these research gaps, we propose a method of adaptive trust calibration that co ...[more]

PMID: 32084201

Similar Datasets

Project description:BackgroundArtificial Intelligence has the potential to revolutionize healthcare, and it is increasingly being deployed to support and assist medical diagnosis. One potential application of AI is as the first point of contact for patients, replacing initial diagnoses prior to sending a patient to a specialist, allowing health care professionals to focus on more challenging and critical aspects of treatment. But for AI systems to succeed in this role, it will not be enough for them to merely provide accurate diagnoses and predictions. In addition, it will need to provide explanations (both to physicians and patients) about why the diagnoses are made. Without this, accurate and correct diagnoses and treatments might otherwise be ignored or rejected.MethodIt is important to evaluate the effectiveness of these explanations and understand the relative effectiveness of different kinds of explanations. In this paper, we examine this problem across two simulation experiments. For the first experiment, we tested a re-diagnosis scenario to understand the effect of local and global explanations. In a second simulation experiment, we implemented different forms of explanation in a similar diagnosis scenario.ResultsResults show that explanation helps improve satisfaction measures during the critical re-diagnosis period but had little effect before re-diagnosis (when initial treatment was taking place) or after (when an alternate diagnosis resolved the case successfully). Furthermore, initial "global" explanations about the process had no impact on immediate satisfaction but improved later judgments of understanding about the AI. Results of the second experiment show that visual and example-based explanations integrated with rationales had a significantly better impact on patient satisfaction and trust than no explanations, or with text-based rationales alone. As in Experiment 1, these explanations had their effect primarily on immediate measures of satisfaction during the re-diagnosis crisis, with little advantage prior to re-diagnosis or once the diagnosis was successfully resolved.ConclusionThese two studies help us to draw several conclusions about how patient-facing explanatory diagnostic systems may succeed or fail. Based on these studies and the review of the literature, we will provide some design recommendations for the explanations offered for AI systems in the healthcare domain.

Project description:BackgroundIn the years to come, artificial intelligence will become an indispensable tool in medical practice. The digital transformation will undoubtedly affect today's medical students. This study focuses on trust from the perspective of three groups of medical students - students from Croatia, students from Slovakia, and international students studying in Slovakia.MethodsA paper-pen survey was conducted using a non-probabilistic convenience sample. In the second half of 2022, 1715 students were surveyed at five faculties in Croatia and three in Slovakia.ResultsSpecifically, 38.2% of students indicated familiarity with the concept of AI, while 44.8% believed they would use AI in the future. Patient readiness for the implementation of technologies was mostly assessed as being low. More than half of the students, 59.1%, believe that the implementation of digital technology (AI) will negatively impact the patient-physician relationship and 51,3% of students believe that patients will trust physicians less. The least agreement with the statement was observed among international students, while a higher agreement was expressed by Slovak and Croatian students 40.9% of Croatian students believe that users do not trust the healthcare system, 56.9% of Slovak students agree with this view, while only 17.3% of international students share this opinion. The ability to explain to patients how AI works if they were asked was statistically significantly different for the different student groups, international students expressed the lowest agreement, while the Slovak and Croatian students showed a higher agreement.ConclusionThis study provides insight into medical students' attitudes from Croatia, Slovakia, and international students regarding the role of artificial intelligence (AI) in the future healthcare system, with a particular emphasis on the concept of trust. A notable difference was observed between the three groups of students, with international students differing from their Croatian and Slovak colleagues. This study also highlights the importance of integrating AI topics into the medical curriculum, taking into account national social & cultural specificities that could negatively impact AI implementation if not carefully addressed.

Dataset Information

Adaptive trust calibration for human-AI collaboration.

Publications

Adaptive trust calibration for human-AI collaboration.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets