Dataset Information

A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study.

ABSTRACT:

Background

Approximately 60%-80% of the primary care visits have a psychological stress component, but only 3% of patients receive stress management advice during these visits. Given recent advances in natural language processing, there is renewed interest in mental health chatbots. Conversational agents that can understand a user's problems and deliver advice that mitigates the effects of daily stress could be an effective public health tool. However, such systems are complex to build and costly to develop.

Objective

To address these challenges, our aim is to develop and evaluate a fully automated mobile suite of shallow chatbots-we call them Popbots-that may serve as a new species of chatbots and further complement human assistance in an ecosystem of stress management support.

Methods

After conducting an exploratory Wizard of Oz study (N=14) to evaluate the feasibility of a suite of multiple chatbots, we conducted a web-based study (N=47) to evaluate the implementation of our prototype. Each participant was randomly assigned to a different chatbot designed on the basis of a proven cognitive or behavioral intervention method. To measure the effectiveness of the chatbots, the participants' stress levels were determined using self-reported psychometric evaluations (eg, web-based daily surveys and Patient Health Questionnaire-4). The participants in these studies were recruited through email and enrolled on the web, and some of them participated in follow-up interviews that were conducted in person or on the web (as necessary).

Results

Of the 47 participants, 31 (66%) completed the main study. The findings suggest that the users viewed the conversations with our chatbots as helpful or at least neutral and came away with increasingly positive sentiment toward the use of chatbots for proactive stress management. Moreover, those users who used the system more often (ie, they had more than or equal to the median number of conversations) noted a decrease in depression symptoms compared with those who used the system less often based on a Wilcoxon signed-rank test (W=91.50; Z=-2.54; P=.01; r=0.47). The follow-up interviews with a subset of the participants indicated that half of the common daily stressors could be discussed with chatbots, potentially reducing the burden on human coping resources.

Conclusions

Our work suggests that suites of shallow chatbots may offer benefits for both users and designers. As a result, this study's contributions include the design and evaluation of a novel suite of shallow chatbots for daily stress management, a summary of benefits and challenges associated with random delivery of multiple conversational interventions, and design guidelines and directions for future research into similar systems, including authoring chatbot systems and artificial intelligence-enabled recommendation algorithms.

SUBMITTER: Mauriello ML

PROVIDER: S-EPMC8479600 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundRegular physical activity (PA) is crucial for well-being; however, healthy habits are difficult to create and maintain. Interventions delivered via conversational agents (eg, chatbots or virtual agents) are a novel and potentially accessible way to promote PA. Thus, it is important to understand the evolving landscape of research that uses conversational agents.ObjectiveThis mixed methods systematic review aims to summarize the usability and effectiveness of conversational agents in promoting PA, describe common theories and intervention components used, and identify areas for further development.MethodsWe conducted a mixed methods systematic review. We searched seven electronic databases (PsycINFO, PubMed, Embase, CINAHL, ACM Digital Library, Scopus, and Web of Science) for quantitative, qualitative, and mixed methods studies that conveyed primary research on automated conversational agents designed to increase PA. The studies were independently screened, and their methodological quality was assessed using the Mixed Methods Appraisal Tool by 2 reviewers. Data on intervention impact and effectiveness, treatment characteristics, and challenges were extracted and analyzed using parallel-results convergent synthesis and narrative summary.ResultsIn total, 255 studies were identified, 7.8% (20) of which met our inclusion criteria. The methodological quality of the studies was varied. Overall, conversational agents had moderate usability and feasibility. Those that were evaluated through randomized controlled trials were found to be effective in promoting PA. Common challenges facing interventions were repetitive program content, high attrition, technical issues, and safety and privacy concerns.ConclusionsConversational agents hold promise for PA interventions. However, there is a lack of rigorous research on long-term intervention effectiveness and patient safety. Future interventions should be based on evidence-informed theories and treatment approaches and should address users' desires for program variety, natural language processing, delivery via mobile devices, and safety and privacy concerns.

Project description:BackgroundObesity is a public health issue worldwide. Conversational agents (CAs), also frequently called chatbots, are computer programs that simulate dialogue between people. Owing to better accessibility, cost-effectiveness, personalization, and compassionate patient-centered treatments, CAs are expected to have the potential to provide sustainable lifestyle counseling for weight management.ObjectiveThis systematic review aimed to critically summarize and evaluate clinical studies on the effectiveness and feasibility of CAs with unconstrained natural language input for weight management.MethodsPubMed, Embase, the Cochrane Library (CENTRAL), PsycINFO, and ACM Digital Library were searched up to December 2022. Studies were included if CAs were used for weight management and had a capability for unconstrained natural language input. No restrictions were imposed on study design, language, or publication type. The quality of the included studies was assessed using the Cochrane risk-of-bias assessment tool or the Critical Appraisal Skills Programme checklist. The extracted data from the included studies were tabulated and narratively summarized as substantial heterogeneity was expected.ResultsIn total, 8 studies met the eligibility criteria: 3 (38%) randomized controlled trials and 5 (62%) uncontrolled before-and-after studies. The CAs in the included studies were aimed at behavior changes through education, advice on food choices, or counseling via psychological approaches. Of the included studies, only 38% (3/8) reported a substantial weight loss outcome (1.3-2.4 kg decrease at 12-15 weeks of CA use). The overall quality of the included studies was judged as low.ConclusionsThe findings of this systematic review suggest that CAs with unconstrained natural language input can be used as a feasible interpersonal weight management intervention by promoting engagement in psychiatric intervention-based conversations simulating treatments by health care professionals, but currently there is a paucity of evidence. Well-designed rigorous randomized controlled trials with larger sample sizes, longer treatment duration, and follow-up focusing on CAs' acceptability, efficacy, and safety are warranted.

Project description:BackgroundMobile health and self-management interventions may positively affect behavioral change and reduce hospital admissions for patients with chronic obstructive pulmonary disease (COPD). However, not all patients qualify for these interventions, and systematic, comprehensive information on implementation- and compliance-related aspects of mobile self-management apps is lacking. Due to the tendency to target digital services to patients in stable phases of disease, it is especially relevant to focus on the use of these services in broad clinical practice for patients recently discharged from hospital.ObjectiveThis study aims to evaluate the effects of a mobile health and self-management app in clinical practice for recently discharged patients with COPD on use of the app, self-management, expectations, and experiences (technology acceptance); patients' and nurses' satisfaction; and hospital readmissions.MethodsA prototype of the app was pilot tested with 6 patients with COPD. The COPD app consisted of an 8-week program including the Lung Attack Action Plan, education, medication overview, video consultation, and questionnaires (monitored by nurses). In the feasibility study, adult patients with physician-diagnosed COPD, access to a mobile device, and proficiency of the Dutch language were included from a large teaching hospital during hospital admission. Self-management (Partners in Health Scale), technology acceptance (Unified Theory Acceptance and Use of Technology model), and satisfaction were assessed using questionnaires at baseline, after 8 weeks, and 20 weeks. Use was assessed with log data, and readmission rates were extracted from the electronic medical record.ResultsA total of 39 patients were included; 76.4% (133/174) of patients had to be excluded from participation, and 48.9% of those patients (65/133) were excluded because of lack of digital skills, access to a mobile device, or access to the internet. The COPD app was opened most often in the first week (median 6.0; IQR 3.5-10.0), but its use decreased over time. The self-management element knowledge and coping increased significantly over time (P=.04). The COPD app was rated on a scale of 1-10, with an average score by patients of 7.7 (SD 1.7) and by nurses of 6.3 (SD 1.2). Preliminary evidence about the readmission rate showed that 13% (5/39) of patients were readmitted within 30 days; 31% (12/39) of patients were readmitted within 20 weeks, compared with 14.1% (48/340) and 21.8% (74/340) in a preresearch cohort, respectively.ConclusionsThe use of a mobile self-management app after hospital discharge seems to be feasible only for a small number of patients with COPD. Patients were satisfied with the service; however, use decreased over time, and only knowledge and coping changed significantly over time. Therefore, future research on digital self-management interventions in clinical practice should focus on including more difficult subgroups of target populations, a multidisciplinary approach, technology-related aspects (such as acceptability), and fine-tuning its adoption in clinical pathways.Trial registrationClinicaltrials.gov NCT04540562; https://clinicaltrials.gov/ct2/show/NCT04540562.

Project description:IntroductionObesity and physical inactivity are major societal challenges and significant contributors to the global burden of disease and healthcare costs. Information and communication technologies are increasingly being used in interventions to promote behaviour change in diet and physical activity. In particular, social networking platforms seem promising for the delivery of weight control interventions.We intend to pilot test an intervention involving the use of a social networking mobile application and tracking devices (Fitbit Flex 2 and Fitbit Aria scale) to promote the social comparison of weight and physical activity, in order to evaluate whether mechanisms of social influence lead to changes in those outcomes over the course of the study.Methods and analysisMixed-methods study involving semi-structured interviews and a pre-post quasi-experimental pilot with one arm, where healthy participants in different body mass index (BMI) categories, aged between 19 and 35 years old, will be subjected to a social networking intervention over a 6-month period. The primary outcome is the average difference in weight before and after the intervention. Secondary outcomes include BMI, number of steps per day, engagement with the intervention, social support and system usability. Semi-structured interviews will assess participants' expectations and perceptions regarding the intervention.Ethics and disseminationEthics approval was granted by Macquarie University's Human Research Ethics Committee for Medical Sciences on 3 November 2016 (ethics reference number 5201600716).The social network will be moderated by a researcher with clinical expertise, who will monitor and respond to concerns raised by participants. Monitoring will involve daily observation of measures collected by the fitness tracker and the wireless scale, as well as continuous supervision of forum interactions and posts. Additionally, a protocol is in place to monitor for participant misbehaviour and direct participants-in-need to appropriate sources of help.

Project description:BackgroundEmbodied conversational agents (ECAs) are increasingly used in health care apps; however, their acceptability in type 2 diabetes (T2D) self-management apps has not yet been investigated.ObjectiveThis study aimed to evaluate the acceptability of the ECA (Laura) used to deliver diabetes self-management education and support in the My Diabetes Coach (MDC) app.MethodsA sequential mixed methods design was applied. Adults with T2D allocated to the intervention arm of the MDC trial used the MDC app over a period of 12 months. At 6 months, they completed questions assessing their interaction with, and attitudes toward, the ECA. In-depth qualitative interviews were conducted with a subsample of the participants from the intervention arm to explore their experiences of using the ECA. The interview questions included the participants' perceptions of Laura, including their initial impression of her (and how this changed over time), her personality, and human character. The quantitative and qualitative data were interpreted using integrated synthesis.ResultsOf the 93 intervention participants, 44 (47%) were women; the mean (SD) age of the participants was 55 (SD 10) years and the baseline glycated hemoglobin A1c level was 7.3% (SD 1.5%). Overall, 66 of the 93 participants (71%) provided survey responses. Of these, most described Laura as being helpful (57/66, 86%), friendly (57/66, 86%), competent (56/66, 85%), trustworthy (48/66, 73%), and likable (40/66, 61%). Some described Laura as not real (18/66, 27%), boring (26/66, 39%), and annoying (20/66, 30%). Participants reported that interacting with Laura made them feel more motivated (29/66, 44%), comfortable (24/66, 36%), confident (14/66, 21%), happy (11/66, 17%), and hopeful (8/66, 12%). Furthermore, 20% (13/66) of the participants were frustrated by their interaction with Laura, and 17% (11/66) of the participants reported that interacting with Laura made them feel guilty. A total of 4 themes emerged from the qualitative data (N=19): (1) perceived role: a friendly coach rather than a health professional; (2) perceived support: emotional and motivational support; (3) embodiment preference acceptability of a human-like character; and (4) room for improvement: need for greater congruence between Laura's words and actions.ConclusionsThese findings suggest that an ECA is an acceptable means to deliver T2D self-management education and support. A human-like character providing ongoing, friendly, nonjudgmental, emotional, and motivational support is well received. Nevertheless, the ECA can be improved by increasing congruence between its verbal and nonverbal communication and accommodating user preferences.Trial registrationAustralian New Zealand Clinical Trials Registry CTRN12614001229662; https://tinyurl.com/yxshn6pd.

Project description:BackgroundVarenicline is a pharmacological intervention for tobacco dependence that is safe and effective in facilitating smoking cessation. Enhanced adherence to varenicline augments the probability of prolonged smoking abstinence. However, research has shown that one-third of people who use varenicline are nonadherent by the second week. There is evidence showing that behavioral support helps with medication adherence. We have designed an artificial intelligence (AI) conversational agent or health bot, called "ChatV," based on evidence of what works as well as what varenicline is, that can provide these supports. ChatV is an evidence-based, patient- and health care provider-informed health bot to improve adherence to varenicline. ChatV has been programmed to provide medication reminders, answer questions about varenicline and smoking cessation, and track medication intake and the number of cigarettes.ObjectiveThis study aims to explore the feasibility of the ChatV health bot, to examine if it is used as intended, and to determine the appropriateness of proceeding with a randomized controlled trial.MethodsWe will conduct a mixed methods feasibility study where we will pilot-test ChatV with 40 participants. Participants will be provided with a standard 12-week varenicline regimen and access to ChatV. Passive data collection will include adoption measures (how often participants use the chatbot, what features they used, when did they use it, etc). In addition, participants will complete questionnaires (at 1, 4, 8, and 12 weeks) assessing self-reported smoking status and varenicline adherence, as well as questions regarding the acceptability, appropriateness, and usability of the chatbot, and participate in an interview assessing acceptability, appropriateness, fidelity, and adoption. We will use "stop, amend, and go" progression criteria for pilot studies to decide if a randomized controlled trial is a reasonable next step and what modifications are required. A health equity lens will be adopted during participant recruitment and data analysis to understand and address the differences in uptake and use of this digital health solution among diverse sociodemographic groups. The taxonomy of implementation outcomes will be used to assess feasibility, that is, acceptability, appropriateness, fidelity, adoption, and usability. In addition, medication adherence and smoking cessation will be measured to assess the preliminary treatment effect. Interview data will be analyzed using the framework analysis method.ResultsParticipant enrollment for the study will begin in January 2024.ConclusionsBy using predetermined progression criteria, the results of this preliminary study will inform the determination of whether to advance toward a larger randomized controlled trial to test the effectiveness of the health bot. Additionally, this study will explore the acceptability, appropriateness, fidelity, adoption, and usability of the health bot. These insights will be instrumental in refining the intervention and the health bot.Trial registrationClinicalTrials.gov NCT05997901; https://classic.clinicaltrials.gov/ct2/show/NCT05997901.International registered report identifier (irrid)PRR1-10.2196/53556.

Project description:BACKGROUND:Patients with chronic arthritis (CA) ideally apply self-management behaviors between consultations. This enduring, tedious task of keeping track of disease-related parameters, adhering to medication schemes, and engaging in physical therapy may be supported by using a mobile health (mHealth) app. However, further research is needed to determine which self-management features are valued most by adult patients with CA patients. OBJECTIVE:The aim of this study was to determine the preference of features for an mHealth app to support self-management behavior in patients with CA. In addition, we aimed to explore the motives behind these ratings. METHODS:A mixed-methods approach was used to gather information from 31 adult patients (14 females), aged 23 to 71 years (mean 51 [SD 12.16]), with CA. Structured interviews were conducted to gather data pertaining to preferences of app features. Interviews were analyzed qualitatively, whereas ratings for each of the 28 features studied were analyzed quantitatively. RESULTS:In general, patients with CA favored the use of features pertaining to supporting active and direct disease management, (eg, medication intake and detecting and alarming of bad posture), helping them to keep a close watch on their disease status and inform their health care professional (eg, providing a means to log and report disease-related data) and receiving personalized information (eg, offering tailored information based on the patient's health data). Patients strongly disliked features that provide a means of social interaction or provide incentivization for disease-related actions (eg, being able to compare yourself with other patients, cooperating toward a common goal, and receiving encouragement from friends and/or family). Driving these evaluations is the finding that every patient with CA hurts in his/her own way, the way the disease unfolds over time and manifests itself in the patient and social environment is different for every patient, and patients with CA are well aware of this. CONCLUSIONS:We have offered an insight into how patients with CA favor mHealth features for self-management apps. The results of this research can inform the design and development of prospective self-management apps for patients with CA.

Project description:BACKGROUND:Despite a large number of clinical trials aiming at evaluating the digital self-management of chronic diseases, there is little discussion about users' experiences with digital approaches. However, a good user experience is a critical factor for technology adoption. Understanding users' experiences can inform the design of approaches toward increased motivation for digital self-management. OBJECTIVE:This study aimed to evaluate the self-management of cystic fibrosis (CF) with a focus on gastrointestinal concerns and the care of young patients. Following a user-centered design approach, we developed a self-management app for patients and parents and a web tool for health care professionals (HCPs). To evaluate the proposed solutions, a 6-month clinical trial was conducted in 6 European CF competence centers. This paper analyzes the user acceptance of the technology and the benefits and disadvantages perceived by the trial participants. METHODS:A mixed methods approach was applied. Data were collected through 41 semistructured qualitative interviews of patients, parents, and HCPs involved in the clinical trial. In addition, data were collected through questionnaires embedded in the self-management app. RESULTS:Support for enzyme dose calculation and nutrition management was found to be particularly useful. Patients and parents rapidly strengthened their knowledge about the treatment and increased their self-efficacy. Reported benefits include reduced occurrence of symptoms and enhanced quality of life. Patients and parents had different skills, requiring follow-up by HCPs in an introductory phase. HCPs valued obtaining precise information about the patients, allowing for more personalized advice. However, the tight follow-up of several patients led to an increased workload. Over time, as patient self-efficacy increased, patient motivation for using the app decreased and the quality of the reported data was reduced. CONCLUSIONS:Self-management enfolds a collaboration between patients and HCPs. To be successful, a self-management approach should be accepted by both parties. Through understanding behaviors and experiences, this study defines recommendations for a complex case-the demanding treatment of CF. We identify target patient groups and situations for which the app is most beneficial and suggest focusing on these rather than motivating for regular app usage over a long time. We also advise the personalized supervision of patients during the introduction of the approach. Finally, we propose to develop guidance for HCPs to facilitate changes in practice. As personalization and technology literacy are factors found to influence the acceptance of digital self-management of other chronic diseases, it is relevant to consider the proposed recommendations beyond the case of CF.

Project description:BACKGROUND:Mobile interventions hold promise as an intervention modality to engage children in improving diabetes self-management education, attitudes, and behaviors. OBJECTIVE:This pilot study aimed to explore the usability, acceptability, and feasibility of delivering a mobile diabetes educational tool to parent-child pairs in a clinical setting. METHODS:This mixed methods pilot study comprised two concurrent phases with differing study participants. Phase 1 used user testing interviews to collect qualitative data on the usability and acceptability of the tool. Phase 2 used a single-arm pre- and poststudy design to quantitatively evaluate the feasibility and preliminary efficacy of the intervention. Study participants (English-speaking families with youth aged 5-14 years with insulin-dependent diabetes) were recruited from an urban hospital in Massachusetts, United States. In phase 1, parent-child pairs were invited to complete the intervention together and participate in 90-min user testing interviews assessing the tool's usability and acceptability. Interview transcripts were analyzed using a directed content analysis approach. In phase 2, parent-child pairs were invited to complete the intervention together in the clinical setting. Measures included parental and child knowledge, attitudes, and behaviors related to diabetes management (self-report surveys) and child hemoglobin A1c levels (medical record extractions); data were collected at baseline and 1-month follow-up. Pre- and postoutcomes were compared using paired t tests and the Fisher exact test. RESULTS:A total of 11 parent-child pairs (N=22) participated in phase 1 of the study, and 10 parent-child pairs (N=20) participated in phase 2 of the study. Participants viewed the mobile educational tool as acceptable (high engagement and satisfaction with the layout, activities, and videos) and identified the areas of improvement for tool usability (duration, directions, and animation). CONCLUSIONS:The findings from this pilot study suggest that the mobile educational tool is an informative, engaging, and feasible way to deliver diabetes self-management education to parents and children in an urban hospital setting. Data will inform future iterations of this mobile diabetes educational intervention to improve usability and test intervention efficacy.