Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study.
Ontology highlight
ABSTRACT: Objective
The objective of this study was to assess whether a version of the Smoke Free app with a supportive chatbot powered by artificial intelligence (versus a version without the chatbot) led to increased engagement and short-term quit success.Methods
Daily or non-daily smokers aged ?18 years who purchased the 'pro' version of the app and set a quit date were randomly assigned (unequal allocation) to receive the app with or without the chatbot. The outcomes were engagement (i.e. total number of logins over the study period) and self-reported abstinence at a one-month follow-up. Unadjusted and adjusted negative binomial and logistic regression models were fitted to estimate incidence rate ratios (IRRs) and odds ratios (ORs) for the associations of interest.Results
A total of 57,214 smokers were included (intervention: 9.3% (5339); control: 90.7% (51,875). The app with the chatbot compared with the standard version led to a 101% increase in engagement (IRRadj?=?2.01, 95% confidence interval (CI)?=?1.92-2.11, p?vs. 3,704/51,875, ORadj?=?2.38, 95% CI?=?2.19-2.58, p?vs. 3,704/5,050, ORadj?=?1.36, 95% CI?=?1.16-1.61, p?ConclusionThe addition of a supportive chatbot to a popular smoking cessation app more than doubled user engagement. In view of very low follow-up rates, there is low quality evidence that the addition also increased self-reported smoking cessation.
SUBMITTER: Perski O
PROVIDER: S-EPMC6775545 | biostudies-literature | 2019 Jan-Dec
REPOSITORIES: biostudies-literature
Publications
Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study.
Perski Olga O Crane David D Beard Emma E Brown Jamie J
Digital health 20190101
<h4>Objective</h4>The objective of this study was to assess whether a version of the Smoke Free app with a supportive chatbot powered by artificial intelligence (versus a version without the chatbot) led to increased engagement and short-term quit success.<h4>Methods</h4>Daily or non-daily smokers aged ≥18 years who purchased the 'pro' version of the app and set a quit date were randomly assigned (unequal allocation) to receive the app with or without the chatbot. The outcomes were engagement (i ...[more]
Similar Datasets
Project description:BackgroundGamification in smoking cessation apps has been found to improve cognitive outcomes associated with higher odds of quitting. Although some research has shown that gamification can also positively impact behavioral outcomes such as smoking cessation, studies have largely focused on physical activity and mental health. Only a few studies have explored the effects of gamification on smoking cessation outcomes, of which the majority have adopted qualitative methodologies and/or assessed engagement with apps using self-report.ObjectiveThis study aimed to explore levels of user engagement with gamification features in a smoking cessation app via in-app metrics. Specifically, the objective of this paper was to investigate whether higher engagement with gamification features is associated with the likelihood of quitting in the short term.MethodsData from a larger online study that recruited smokers seeking to quit were analyzed to address the objectives presented in this paper. The study took place between June 2019 and July 2020, and participants were primarily recruited via social media posts. Participants who met the eligibility criteria used 1 of 2 mobile apps for smoking cessation. In-app metrics shared by the developer of one of the smoking cessation apps, called Kwit, were used to assess engagement with gamification features. Out of 58 participants who used the Kwit app, 14 were excluded due to missing data or low engagement with the app (ie, not opening the app once a week). For the remaining 44 participants, mean (SD) values were calculated for engagement with the app using in-app metrics. A logistic regression model was used to investigate the association between engagement with gamification and 7-day smoking abstinence.ResultsIn total, data from 44 participants who used the Kwit app were analyzed. The majority of participants were male, married, and employed. Almost 30% (n=13) of participants self-reported successful 7-day abstinence at the end of the study. On average, the Kwit app was opened almost 31 (SD 39) times during the 4-week study period, with the diary feature used the most often (mean 22.8, SD 49.3). Moreover, it was found that each additional level unlocked was associated with approximately 22% higher odds of achieving 7-day abstinence after controlling for other factors such as age and gender (odds ratio 1.22, 95% CI 1.01-1.47).ConclusionsThis study highlights the likely positive effects of certain gamification elements such as levels and achievements on short-term smoking abstinence. Although more robust research with a larger sample size is needed, this research highlights the important role that gamification features integrated into mobile apps can play in facilitating and supporting health behavior change.
| S-EPMC9932870 | biostudies-literature
Project description:BackgroundConversational chatbots are an emerging digital intervention for smoking cessation. No studies have reported on the entire development process of a cessation chatbot.ObjectiveWe aim to report results of the user-centered design development process and randomized controlled trial for a novel and comprehensive quit smoking conversational chatbot called QuitBot.MethodsThe 4 years of formative research for developing QuitBot followed an 11-step process: (1) specifying a conceptual model; (2) conducting content analysis of existing interventions (63 hours of intervention transcripts); (3) assessing user needs; (4) developing the chat's persona ("personality"); (5) prototyping content and persona; (6) developing full functionality; (7) programming the QuitBot; (8) conducting a diary study; (9) conducting a pilot randomized controlled trial (RCT); (10) reviewing results of the RCT; and (11) adding a free-form question and answer (QnA) function, based on user feedback from pilot RCT results. The process of adding a QnA function itself involved a three-step process: (1) generating QnA pairs, (2) fine-tuning large language models (LLMs) on QnA pairs, and (3) evaluating the LLM outputs.ResultsWe developed a quit smoking program spanning 42 days of 2- to 3-minute conversations covering topics ranging from motivations to quit, setting a quit date, choosing Food and Drug Administration-approved cessation medications, coping with triggers, and recovering from lapses and relapses. In a pilot RCT with 96% three-month outcome data retention, QuitBot demonstrated high user engagement and promising cessation rates compared to the National Cancer Institute's SmokefreeTXT text messaging program, particularly among those who viewed all 42 days of program content: 30-day, complete-case, point prevalence abstinence rates at 3-month follow-up were 63% (39/62) for QuitBot versus 38.5% (45/117) for SmokefreeTXT (odds ratio 2.58, 95% CI 1.34-4.99; P=.005). However, Facebook Messenger intermittently blocked participants' access to QuitBot, so we transitioned from Facebook Messenger to a stand-alone smartphone app as the communication channel. Participants' frustration with QuitBot's inability to answer their open-ended questions led to us develop a core conversational feature, enabling users to ask open-ended questions about quitting cigarette smoking and for the QuitBot to respond with accurate and professional answers. To support this functionality, we developed a library of 11,000 QnA pairs on topics associated with quitting cigarette smoking. Model testing results showed that Microsoft's Azure-based QnA maker effectively handled questions that matched our library of 11,000 QnA pairs. A fine-tuned, contextualized GPT-3.5 (OpenAI) responds to questions that are not within our library of QnA pairs.ConclusionsThe development process yielded the first LLM-based quit smoking program delivered as a conversational chatbot. Iterative testing led to significant enhancements, including improvements to the delivery channel. A pivotal addition was the inclusion of a core LLM-supported conversational feature allowing users to ask open-ended questions.Trial registrationClinicalTrials.gov NCT03585231; https://clinicaltrials.gov/study/NCT03585231.
| S-EPMC11303891 | biostudies-literature
Project description:BACKGROUND:Smoking rates in the United States have been reduced in the past decades to 15% of the general population. However, up to 88% of people with psychiatric symptoms still smoke, leading to high rates of disease and mortality. Therefore, there is a great need to develop smoking cessation interventions that have adequate levels of usability and can reach this population. OBJECTIVE:The objective of this study was to report the rationale, ideation, design, user research, and final specifications of a novel smoking cessation app for people with serious mental illness (SMI) that will be tested in a feasibility trial. METHODS:We used a variety of user-centered design methods and materials to develop the tailored smoking cessation app. This included expert panel guidance, a set of design principles and theory-based smoking cessation content, development of personas and paper prototyping, usability testing of the app prototype, establishment of app's core vision and design specification, and collaboration with a software development company. RESULTS:We developed Learn to Quit, a smoking cessation app designed and tailored to individuals with SMI that incorporates the following: (1) evidence-based smoking cessation content from Acceptance and Commitment Therapy and US Clinical Practice Guidelines for smoking cessation aimed at providing skills for quitting while addressing mental health symptoms, (2) a set of behavioral principles to increase retention and comprehension of smoking cessation content, (3) a gamification component to encourage and sustain app engagement during a 14-day period, (4) an app structure and layout designed to minimize usability errors in people with SMI, and (5) a set of stories and visuals that communicate smoking cessation concepts and skills in simple terms. CONCLUSIONS:Despite its increasing importance, the design and development of mHealth technology is typically underreported, hampering scientific innovation. This report describes the systematic development of the first smoking cessation app tailored to people with SMI, a population with very high rates of nicotine addiction, and offers new design strategies to engage this population. mHealth developers in smoking cessation and related fields could benefit from a design strategy that capitalizes on the role visual engagement, storytelling, and the systematic application of behavior analytic principles to deliver evidence-based content.
| S-EPMC5790963 | biostudies-literature
Project description:OBJECTIVE:To assess the efficacy of an interactive smoking cessation decision-aid application (pp) compared with a smoking cessation static information app on continuous abstinence. DESIGN:Automated double-blind randomised controlled trial with 6 months follow-up (2014-2015). SETTING:Smartphone-based. PARTICIPANTS:684 participants (daily smokers of cigarettes, 18 years old or over) recruited passively from app stores in the USA, Australia, UK and Singapore, and randomised to one of two sub-apps. INTERVENTIONS:Behavioural, decision-aid, smartphone application. MAIN OUTCOMES:Continuous abstinence at 10 days, 1?month, 3 months and 6 months. RESULTS:Smokers who received the decision-aid app were more likely to be continuously abstinent at 1?month compared with the information-only app (28.5% vs 16.9%; relative risk (RR) 1.68; 95%?CI 1.25 to 2.28). The effect was sustained at 3 months (23.8% vs 10.2%; RR 2.08; 95%?CI 1.38 to 3.18) and 6 months (10.2% vs 4.8%; RR 2.02; 95%?CI 1.08 to 3.81). Participants receiving the decision-aid app were also more likely to have made an informed choice (31.9% vs 19.6%) and have lower decisional conflict (19.5% vs 3.9%). CONCLUSION:A smartphone decision-aid app with support features significantly increased smoking cessation and informed choice. With an increasing number of smokers attempting to quit, unassisted evidence-based decision-aid apps can provide an effective and user-friendly option to many who are making quit decisions without healthcare professionals. TRIAL REGISTRATION NUMBER:ACTRN12613000833763.
| S-EPMC5780681 | biostudies-other
Project description:IntroductionUsing WebQuit as a case study, a smoking cessation website grounded in Acceptance and Commitment Therapy, we aimed to identify sequence clusters of content usage and examine their associations with baseline characteristics, change to a key mechanism of action, and smoking cessation.MethodsParticipants were adult smokers allocated to the WebQuit arm in a randomized controlled trial (n = 1,313). WebQuit contains theory-informed content including goal setting, self-monitoring and feedback, and values- and acceptance-based exercises. Sequence analysis was used to temporally order 30-s website usage segments for each participant. Similarities between sequences were assessed with the optimal matching distance algorithm and used as input in an agglomerative hierarchical clustering analysis. Associations between sequence clusters and baseline characteristics, acceptance of cravings at 3 months and self-reported 30-day point prevalence abstinence at 12 months were examined with linear and logistic regression.ResultsThree qualitatively different sequence clusters were identified. "Disengagers" (576/1,313) almost exclusively used the goal-setting feature. "Tryers" (375/1,313) used goal setting and two of the values- and acceptance-based components ("Be Aware," "Be Willing"). "Committers" (362/1,313) primarily used two of the values- and acceptance-based components ("Be Willing," "Be Inspired"), goal setting, and self-monitoring and feedback. Compared with Disengagers, Committers demonstrated greater increases in acceptance of cravings (p = .01) and 64% greater odds of quit success (ORadj = 1.64, 95% CI = 1.18, 2.29, p = .003).DiscussionWebQuit users were categorized into Disengagers, Tryers, and Committers based on their qualitatively different content usage patterns. Committers saw increases in a key mechanism of action and greater odds of quit success.ImplicationsThis case study demonstrates how employing sequence and cluster analysis of usage data can help researchers and practitioners gain a better understanding of how users engage with a given eHealth intervention over time and use findings to test theory and/or to improve future iterations to the intervention. Future WebQuit users may benefit from being directed to the values- and acceptance-based and the self-monitoring and feedback components via reminders over the course of the program.
| S-EPMC8186423 | biostudies-literature
Project description:IntroductionMedication sampling is a clinically useful tool to engage smokers in the quitting process. Whether varenicline is suitable for sampling purposes is unclear. The purpose of this study was to examine the feasibility, uptake, and preliminary outcomes of varenicline sampling.MethodsSmokers (N = 99), both motivated to quit and not, were recruited and randomized to varenicline sampling versus not, with 12 week follow-up. The intervention consisted of mailing one-time samples of varenicline (lasting 2-4 wks), with minimally suggestive guidance on use.ResultsUptake of varenicline was strong, at 2 weeks (54% any use, 66% daily use) and 4 weeks (38%, 46%), with 58% of medication users seeking additional medication. Most users followed conventional titration patterns, self-titrating from 0.5 mg to 2 mg. Relative to control, varenicline sampling increased motivation (p = 0.006) and confidence to quit (p = 0.02), and decreased cigarette smoking (p = 0.02). Smokers receiving varenicline samples were significantly more likely to achieve 50% reduction in cigarettes per day (CPD), both immediately following the sampling exercise (Adjusted Odds Ratio [AOR] = 4.12; 95% CI: 1.39 to 12.17) and at final follow-up (AOR = 4.50; 95% CI: 1.56 to 13.01). Though cessation outcomes were not statistically significant, there was a 1.5 to 3-fold increase in quit attempts and abstinence from varenicline sampling throughout follow-up. These outcomes were comparable among smokers motivated to quit and not.ConclusionsUnguided, user-driven sampling of varenicline sampling is a concrete behavioral exercise that is feasible to do and seems to suggest clinical utility. Sampling is a pragmatic clinical approach to engage more smokers in quitting.ImplicationsUse of evidence-based pharmacotherapies for smoking cessation is low. Medication sampling is a pragmatic behavioral exercise that allows smokers to experience the benefits of using them, while promoting positive downstream effects towards quitting. While previous studies have shown that nicotine replacement therapy (NRT) sampling is viable and effective, whether this extends to varenicline is unclear. Results from this trial demonstrate that varenicline sampling is feasible, safe, and suggestive of clinically important steps toward quitting, deserving of a larger trial.Clinical trial registrationNCT #03742154.
| S-EPMC8150130 | biostudies-literature
Project description:The use of mobile health applications (apps) especially in the area of lifestyle behaviors has increased, thus providing unprecedented opportunities to develop health programs that can engage people in real-time and in the real-world. Yet, relatively little is known about which factors relate to the engagement of commercially available apps for health behaviors. This exploratory study examined behavioral engagement with a weight loss app, Lose It! and characterized higher versus lower engaged groups. Cross-sectional, anonymized data from Lose It! were analyzed (n = 12,427,196). This dataset was randomly split into 24 subsamples and three were used for this study (total n = 1,011,008). Classification and regression tree methods were used to identify subgroups of user engagement with one subsample, and descriptive analyses were conducted to examine other group characteristics associated with engagement. Data mining validation methods were conducted with two separate subsamples. On average, users engaged with the app for 29 days. Six unique subgroups were identified, and engagement for each subgroup varied, ranging from 3.5 to 172 days. Highly engaged subgroups were primarily distinguished by the customization of diet and exercise. Those less engaged were distinguished by weigh-ins and the customization of diet. Results were replicated in further analyses. Commercially-developed apps can reach large segments of the population, and data from these apps can provide insights into important app features that may aid in user engagement. Getting users to engage with a mobile health app is critical to the success of apps and interventions that are focused on health behavior change.
| S-EPMC5526821 | biostudies-other
Project description:BACKGROUND:Smoking remains a leading cause of preventable death and illness. Internet interventions for smoking cessation have the potential to significantly impact public health, given their broad reach and proven effectiveness. Given the dose-response association between engagement and behavior change, identifying strategies to promote engagement is a priority across digital health interventions. Text messaging is a proven smoking cessation treatment modality and a powerful strategy to increase intervention engagement in other areas of health, but it has not been tested as an engagement strategy for a digital cessation intervention. OBJECTIVE:This study examined the impact of 4 experimental text message design factors on adult smokers' engagement with an internet smoking cessation program. METHODS:We conducted a 2×2×2×2 full factorial screening experiment wherein 864 participants were randomized to 1 of 16 experimental conditions after registering with a free internet smoking cessation program and enrolling in its automated text message program. Experimental factors were personalization (on/off), integration between the web and text message platforms (on/off), dynamic tailoring of intervention content based on user engagement (on/off), and message intensity (tapered vs abrupt drop-off). Primary outcomes were 3-month measures of engagement (ie, page views, time on site, and return visits to the website) as well as use of 6 interactive features of the internet program. All metrics were automatically tracked; there were no missing data. RESULTS:Main effects were detected for integration and dynamic tailoring. Integration significantly increased interactive feature use by participants, whereas dynamic tailoring increased the number of features used and page views. No main effects were found for message intensity or personalization alone, although several synergistic interactions with other experimental features were observed. Synergistic effects, when all experimental factors were active, resulted in the highest rates of interactive feature use and the greatest proportion of participants at high levels of engagement. Measured in terms of standardized mean differences (SMDs), effects on interactive feature use were highest for Build Support System (SMD 0.56; 95% CI 0.27 to 0.81), Choose Quit Smoking Aid (SMD 0.38; 95% CI 0.10 to 0.66), and Track Smoking Triggers (SMD 0.33; 95% CI 0.05 to 0.61). Among the engagement metrics, the largest effects were on overall feature utilization (SMD 0.33; 95% CI 0.06 to 0.59) and time on site (SMD 0.29; 95% CI 0.01 to 0.57). As no SMD >0.30 was observed for main effects on any outcome, results suggest that for some outcomes, the combined intervention was stronger than individual factors alone. CONCLUSIONS:This factorial experiment demonstrates the effectiveness of text messaging as a strategy to increase engagement with an internet smoking cessation intervention, resulting in greater overall intervention dose and greater exposure to the core components of tobacco dependence treatment that can promote abstinence. TRIAL REGISTRATION:ClinicalTrials.gov NCT02585206; https://clinicaltrials.gov/ct2/show/NCT02585206. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID):RR2-10.1136/bmjopen-2015-010687.
| S-EPMC7386536 | biostudies-literature
Project description:BackgroundMotivational messaging is a frequently used digital intervention to promote positive health behavior changes, including smoking cessation. Typically, motivational messaging systems have not actively sought feedback on each message, preventing a closer examination of the user-system engagement. This study assessed the granular user-system engagement around a recommender system (a new system that actively sought user feedback on each message to improve message selection) for promoting smoking cessation and the impact of engagement on cessation outcome.MethodsWe prospectively followed a cohort of current smokers enrolled to use the recommender system for 6 months. The system sent participants motivational messages to support smoking cessation every 3 days and used machine learning to incorporate user feedback (i.e., user's rating on the perceived influence of each message, collected on a 5-point Likert scale with 1 indicating strong disagreement and 5 indicating strong agreement on perceiving the influence on quitting smoking) to improve the selection of the following message. We assessed user-system engagement by various metrics, including user response rate (i.e., the percent of times a user rated the messages) and the perceived influence of messages. We compared retention rates across different levels of user-system engagement and assessed the association between engagement and the 7-day point prevalence abstinence (missing outcome = smoking) by using multiple logistic regression.ResultsWe analyzed data from 731 participants (13% Black; 73% women). The user response rate was 0.24 (SD = 0.34) and user-perceived influence was 3.76 (SD = 0.84). The retention rate positively increased with the user response rate (trend test P < 0.001). Compared with non-response, six-month cessation increased with the levels of response rates: low response rate (odds ratio [OR] = 1.86, 95% confidence interval [CI]: 1.07-3.23), moderate response rate (OR = 2.30, 95% CI: 1.36-3.88), high response rate (OR = 2.69, 95% CI: 1.58-4.58). The association between perceived message influence and the outcome showed a similar pattern.ConclusionsHigh user-system engagement was positively associated with both high retention rate and smoking cessation, suggesting that investigation of methods to increase engagement may be crucial to increase the impact of the recommender system for smoking cessation.Trial registrationRegistration Identifier: NCT03224520 . Registration date: July 21, 2017.
| S-EPMC8465689 | biostudies-literature
Project description: Not available
| S-EPMC8323184 | biostudies-literature