Dataset Information

Usability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study.

ABSTRACT: BACKGROUND:The usability and effectiveness of conversational agents (chatbots) that deliver psychological therapies is under-researched. OBJECTIVE:This study aimed to compare the system usability, acceptability, and effectiveness in older adults of 2 Web-based conversational agents that differ in theoretical orientation and approach. METHODS:In a randomized study, 112 older adults were allocated to 1 of the following 2 fully automated interventions: Manage Your Life Online (MYLO; ie, a chatbot that mimics a therapist using a method of levels approach) and ELIZA (a chatbot that mimics a therapist using a humanistic counseling approach). The primary outcome was problem distress and resolution, with secondary outcome measures of system usability and clinical outcome. RESULTS:MYLO participants spent significantly longer interacting with the conversational agent. Posthoc tests indicated that MYLO participants had significantly lower problem distress at follow-up. There were no differences between MYLO and ELIZA in terms of problem resolution. MYLO was rated as significantly more helpful and likely to be used again. System usability of both the conversational agents was associated with helpfulness of the agents and the willingness of the participants to reuse. Adherence was high. A total of 12% (7/59) of the MYLO group did not carry out their conversation with the chatbot. CONCLUSIONS:Controlled studies of chatbots need to be conducted in clinical populations across different age groups. The potential integration of chatbots into psychological care in routine services is discussed.

SUBMITTER: Bennion MR

PROVIDER: S-EPMC7287711 | biostudies-literature | 2020 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Usability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study.

Bennion Matthew Russell MR Hardy Gillian E GE Moore Roger K RK Kellett Stephen S Millings Abigail A

Journal of medical Internet research 20200527 5

<h4>Background</h4>The usability and effectiveness of conversational agents (chatbots) that deliver psychological therapies is under-researched.<h4>Objective</h4>This study aimed to compare the system usability, acceptability, and effectiveness in older adults of 2 Web-based conversational agents that differ in theoretical orientation and approach.<h4>Methods</h4>In a randomized study, 112 older adults were allocated to 1 of the following 2 fully automated interventions: Manage Your Life Online ...[more]

PMID: 32384055

Similar Datasets

Project description:IntroductionConversational agents (CAs; computer programs that use artificial intelligence to simulate a conversation with users through natural language) have evolved considerably in recent years to support healthcare by providing autonomous, interactive, and accessible services, making them potentially useful for supporting smoking cessation. We performed a systematic review and meta-analysis to provide an overarching evaluation of their effectiveness and acceptability to inform future development and adoption.Aims and methodsPsycInfo, Web of Science, ACM Digital Library, IEEE Xplore, Medline, EMBASE, Communication and Mass Media Complete, and CINAHL Complete were searched for studies examining the use of CAs for smoking cessation. Data from eligible studies were extracted and used for random-effects meta-analyses.ResultsThe search yielded 1245 publications with 13 studies eligible for systematic review (total N = 8236) and six studies for random-effects meta-analyses. All studies reported positive effects on cessation-related outcomes. A meta-analysis with randomized controlled trials reporting on abstinence yielded a sample-weighted odds ratio of 1.66 (95% CI = 1.33% to 2.07%, p < .001), favoring CAs over comparison groups. A narrative synthesis of all included studies showed overall high acceptability, while some barriers were identified from user feedback. Overall, included studies were diverse in design with mixed quality, and evidence of publication bias was identified. A lack of theoretical foundations was noted, as well as a clear need for relational communication in future designs.ConclusionsThe effectiveness and acceptability of CAs for smoking cessation are promising. However, standardization of reporting and designing of the agents is warranted for a more comprehensive evaluation.ImplicationsThis is the first systematic review to provide insight into the use of CAs to support smoking cessation. Our findings demonstrated initial promise in the effectiveness and user acceptability of these agents. We also identified a lack of theoretical and methodological limitations to improve future study design and intervention delivery.

Project description:ObjectivesProblem-solving skills training is adaptable, inexpensive and simple to deliver. However, its application with prisoners who self-harm is unknown. The study assessed the feasibility and acceptability of a problem-solving training (PST) intervention for prison staff and prisoners who self-harm, to inform the design of a large-scale study.Design and settingA mixed-methods design used routinely collected data, individual outcome measures, an economic protocol and qualitative interviews at four prisons in Yorkshire and Humber, UK.Participants(i) Front-line prison staff, (ii) male and female prisoners with an episode of self-harm in the previous 2 weeks.InterventionThe intervention comprised a 1 hour staff training session and a 30 min prisoner session using adapted workbooks and case studies.OutcomesWe assessed the study processes-coverage of training; recruitment and retention rates and adequacy of intervention delivery-and available data (completeness of outcome data, integrity of routinely collected data and access to the National Health Service (NHS) resource information). Prisoner outcomes assessed incidence of self-harm, quality of life and depression at baseline and at follow-up. Qualitative findings are presented elsewhere.ResultsRecruitment was higher than anticipated for staff n=280, but lower for prisoners, n=48. Retention was good with 43/48 (89%) prisoners completing the intervention, at follow-up we collected individual outcome data for 34/48 (71%) of prisoners. Access to routinely collected data was inconsistent. Prisoners were frequent users of NHS healthcare. The additional cost of training and intervention delivery was deemed minimal in comparison to 'treatment as usual'. Outcome measures of self-harm, quality of life and depression were found to be acceptable.ConclusionsThe intervention proved feasible to adapt. Staff training was delivered but on the whole it was not deemed feasible for staff to deliver the intervention. A large-scale study is warranted, but modifications to the implementation of the intervention are required.

Project description:BackgroundConversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years.ObjectiveThis review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success.MethodsWe performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map.ResultsOverall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue).ConclusionsThis review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary.Trial registrationPROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.

Project description:BackgroundThe high demand for health care services and the growing capability of artificial intelligence have led to the development of conversational agents designed to support a variety of health-related activities, including behavior change, treatment support, health monitoring, training, triage, and screening support. Automation of these tasks could free clinicians to focus on more complex work and increase the accessibility to health care services for the public. An overarching assessment of the acceptability, usability, and effectiveness of these agents in health care is needed to collate the evidence so that future development can target areas for improvement and potential for sustainable adoption.ObjectiveThis systematic review aims to assess the effectiveness and usability of conversational agents in health care and identify the elements that users like and dislike to inform future research and development of these agents.MethodsPubMed, Medline (Ovid), EMBASE (Excerpta Medica dataBASE), CINAHL (Cumulative Index to Nursing and Allied Health Literature), Web of Science, and the Association for Computing Machinery Digital Library were systematically searched for articles published since 2008 that evaluated unconstrained natural language processing conversational agents used in health care. EndNote (version X9, Clarivate Analytics) reference management software was used for initial screening, and full-text screening was conducted by 1 reviewer. Data were extracted, and the risk of bias was assessed by one reviewer and validated by another.ResultsA total of 31 studies were selected and included a variety of conversational agents, including 14 chatbots (2 of which were voice chatbots), 6 embodied conversational agents (3 of which were interactive voice response calls, virtual patients, and speech recognition screening systems), 1 contextual question-answering agent, and 1 voice recognition triage system. Overall, the evidence reported was mostly positive or mixed. Usability and satisfaction performed well (27/30 and 26/31), and positive or mixed effectiveness was found in three-quarters of the studies (23/30). However, there were several limitations of the agents highlighted in specific qualitative feedback.ConclusionsThe studies generally reported positive or mixed evidence for the effectiveness, usability, and satisfactoriness of the conversational agents investigated, but qualitative user perceptions were more mixed. The quality of many of the studies was limited, and improved study design and reporting are necessary to more accurately evaluate the usefulness of the agents in health care and identify key areas for improvement. Further research should also analyze the cost-effectiveness, privacy, and security of the agents.International registered report identifier (irrid)RR2-10.2196/16934.

Project description:BackgroundeHealth tools have the potential to meet the mental health needs of individuals who experience barriers to accessing in-person treatment. However, most users have less than optimal engagement with eHealth tools. Coaching from peer specialists may increase their engagement with eHealth.ObjectiveThis pilot study aims to test the feasibility and acceptability of a novel, completely automated web-based system to recruit, screen, enroll, assess, randomize, and then deliver an intervention to a national sample of military veterans with unmet mental health needs; investigate whether phone-based peer support increases the use of web-based problem-solving training compared with self-directed use; and generate hypotheses about potential mechanisms of action for problem-solving and peer support for future full-scale research.MethodsVeterans (N=81) with unmet mental health needs were recruited via social media advertising and enrolled and randomized to the self-directed use of a web-based problem-solving training called Moving Forward (28/81, 35%), peer-supported Moving Forward (27/81, 33%), or waitlist control (26/81, 32%). The objective use of Moving Forward was measured with the number of log-ins. Participants completed pre- and poststudy measures of mental health symptoms and problem-solving confidence. Satisfaction was also assessed post treatment.ResultsAutomated recruitment, enrollment, and initial assessment methods were feasible and resulted in a diverse sample of veterans with unmet mental health needs from 38 states. Automated follow-up methods resulted in 46% (37/81) of participants completing follow-up assessments. Peer support was delivered with high fidelity and was associated with favorable participant satisfaction. Participants randomized to receive peer support had significantly more Moving Forward log-ins than those of self-directed Moving Forward participants, and those who received peer support had a greater decrease in depression. Problem-solving confidence was associated with greater Moving Forward use and improvements in mental health symptoms among participants both with and without peer support.ConclusionsEnrolling and assessing individuals in eHealth studies without human contact is feasible; however, different methods or designs are necessary to achieve acceptable participant engagement and follow-up rates. Peer support shows potential for increasing engagement in web-based interventions and reducing symptoms. Future research should investigate when and for whom peer support for eHealth is helpful. Problem-solving confidence should be further investigated as a mechanism of action for web-based problem-solving training.Trial registrationClinicalTrials.gov NCT03555435; http://clinicaltrials.gov/ct2/show/NCT03555435.

Project description:In this paper we argue that a synthesis of findings across the various sub-areas of research in complex problem solving and consequently progress in theory building is hampered by an insufficient differentiation of complexity and difficulty. In the proposed framework of person, task, and situation (PTS), complexity is conceptualized as a quality that is determined by the cognitive demands that the characteristics of the task and the situation impose. Difficulty represents the quantifiable level of a person's success in dealing with such demands. We use the well-documented "semantic effect" as an exemplar for testing some of the conceptual assumptions derived from the PTS framework. We demonstrate how a differentiation between complexity and difficulty can help take beyond a potentially too narrowly defined psychometric perspective and subsequently gain a better understanding of the cognitive mechanisms behind this effect. In an empirical study a total of 240 university students were randomly allocated to one of four conditions. The four conditions resulted from contrasting the semanticity level of the variable labels used in the CPS system (high vs. low) and two instruction conditions for how to explore the CPS system's causal structure (starting with the assumption that all relationships between variables existed vs. starting with the assumption that none of the relationships existed). The variation in the instruction aimed at inducing knowledge acquisition processes of either (1) systematic elimination of presumptions, or (2) systematic compilation of a mental representation of the causal structure underpinning the system. Results indicate that (a) it is more complex to adopt a "blank slate" perspective under high semanticity as it requires processes of inhibiting prior assumptions, and (b) it seems more difficult to employ a systematic heuristic when testing against presumptions. In combination, situational characteristics, such as the semanticity of variable labels, have the potential to trigger qualitatively different tasks. Failing to differentiate between 'task' and 'situation' as independent sources of complexity and treating complexity and difficulty synonymously threaten the validity of performance scores obtained in CPS research.

Dataset Information

Usability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study.

Publications

Usability, Acceptability, and Effectiveness of Web-Based Conversational Agents to Facilitate Problem Solving in Older Adults: Controlled Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets