Dataset Information

Evaluating the use of a recommender system for selecting optimal messages for smoking cessation: patterns and effects of user-system engagement.

ABSTRACT:

Background

Motivational messaging is a frequently used digital intervention to promote positive health behavior changes, including smoking cessation. Typically, motivational messaging systems have not actively sought feedback on each message, preventing a closer examination of the user-system engagement. This study assessed the granular user-system engagement around a recommender system (a new system that actively sought user feedback on each message to improve message selection) for promoting smoking cessation and the impact of engagement on cessation outcome.

Methods

We prospectively followed a cohort of current smokers enrolled to use the recommender system for 6 months. The system sent participants motivational messages to support smoking cessation every 3 days and used machine learning to incorporate user feedback (i.e., user's rating on the perceived influence of each message, collected on a 5-point Likert scale with 1 indicating strong disagreement and 5 indicating strong agreement on perceiving the influence on quitting smoking) to improve the selection of the following message. We assessed user-system engagement by various metrics, including user response rate (i.e., the percent of times a user rated the messages) and the perceived influence of messages. We compared retention rates across different levels of user-system engagement and assessed the association between engagement and the 7-day point prevalence abstinence (missing outcome = smoking) by using multiple logistic regression.

Results

We analyzed data from 731 participants (13% Black; 73% women). The user response rate was 0.24 (SD = 0.34) and user-perceived influence was 3.76 (SD = 0.84). The retention rate positively increased with the user response rate (trend test P < 0.001). Compared with non-response, six-month cessation increased with the levels of response rates: low response rate (odds ratio [OR] = 1.86, 95% confidence interval [CI]: 1.07-3.23), moderate response rate (OR = 2.30, 95% CI: 1.36-3.88), high response rate (OR = 2.69, 95% CI: 1.58-4.58). The association between perceived message influence and the outcome showed a similar pattern.

Conclusions

High user-system engagement was positively associated with both high retention rate and smoking cessation, suggesting that investigation of methods to increase engagement may be crucial to increase the impact of the recommender system for smoking cessation.

Trial registration

Registration Identifier: NCT03224520 . Registration date: July 21, 2017.

SUBMITTER: Chen J

PROVIDER: S-EPMC8465689 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:Smoking is one of the most avoidable health risk factors, and yet the quitting success rates are low. The usage of tailored health messages to support quitting has been proved to increase quitting success rates. Technology can provide convenient means to deliver tailored health messages. Health recommender systems are information-filtering algorithms that can choose the most relevant health-related items-for instance, motivational messages aimed at smoking cessation-for each user based on his or her profile. The goals of this study are to analyze the perceived quality of an mHealth recommender system aimed at smoking cessation, and to assess the level of engagement with the messages delivered to users via this medium. METHODS:Patients participating in a smoking cessation program will be provided with a mobile app to receive tailored motivational health messages selected by a health recommender system, based on their profile retrieved from an electronic health record as the initial knowledge source. Patients' feedback on the messages and their interactions with the app will be analyzed and evaluated following an observational prospective methodology to a) assess the perceived quality of the mobile-based health recommender system and the messages, using the precision and time-to-read metrics and an 18-item questionnaire delivered to all patients who complete the program, and b) measure patient engagement with the mobile-based health recommender system using aggregated data analytic metrics like session frequency and, to determine the individual-level engagement, the rate of read messages for each user. This paper details the implementation and evaluation protocol that will be followed. DISCUSSION:This study will explore whether a health recommender system algorithm integrated with an electronic health record can predict which tailored motivational health messages patients would prefer and consider to be of a good quality, encouraging them to engage with the system. The outcomes of this study will help future researchers design better tailored motivational message-sending recommender systems for smoking cessation to increase patient engagement, reduce attrition, and, as a result, increase the rates of smoking cessation. TRIAL REGISTRATION:The trial was registered at clinicaltrials.org under the ClinicalTrials.gov identifier NCT03206619 on July 2nd 2017. Retrospectively registered.

Project description:BackgroundSmoking cessation is the most common preventative for an array of diseases, including lung cancer and chronic obstructive pulmonary disease. Although there are many efforts advocating for smoking cessation, smoking is still highly prevalent. For instance, in the USA in 2015, 50% of all smokers attempted to quit smoking, and only 5-7% of them succeeded - with slight deviation depending on external assistance. Previous studies show that computer-tailored messages which support smoking abstinence are effective. The combination of health recommender systems and behavioral-change theories is becoming increasingly popular in computer-tailoring. The objective of this study is to evaluate patients's smoking cessation rates by means of two randomized controlled trials using computer-tailored motivational messages. A group of 100 patients will be recruited in medical centers in Taiwan (50 patients in the intervention group, and 50 patients in the control group), and a group of 1000 patients will be recruited on-line (500 patients in the intervention group, and 500 patients in the control group). The collected data will be made available to the public in an open-source data portal.MethodsOur study will gather data from two sources. The first source is a clinical pilot in which a group of patients from two Taiwanese medical centers will be randomly assigned to either an intervention or a control group. The intervention group will be provided with a mobile app that sends motivational messages selected by a recommender system that takes the user profile (including gender, age, motivations, and social context) and similar users' opinions. For 6 months, the patients' smoking activity will be followed up, and confirmed as "smoke-free" by using a test that measures expired carbon monoxide and urinary cotinine levels. The second source will be a public pilot in which Internet users wanting to quit smoking will be able to download the same mobile app as used in the clinical pilot. They will be randomly assigned to a control group that receives basic motivational messages or to an intervention group, that receives personalized messages by the recommender system. For 6 months, patients in the public pilot will be assessed periodically with self-reported questionnaires.DiscussionThis study will be the first to use the I-Change behavioral-change model in combination with a health recommender system and will, therefore, provide relevant insights into computer-tailoring for smoking cessation. If our hypothesis is validated, clinical practice for smoking cessation would benefit from the use of our mobile solution.Trial registrationClinicalTrials.gov, ID: NCT03108651 . Registered on 11 April 2017.

Project description:BACKGROUND:Smoking remains a leading cause of preventable death and illness. Internet interventions for smoking cessation have the potential to significantly impact public health, given their broad reach and proven effectiveness. Given the dose-response association between engagement and behavior change, identifying strategies to promote engagement is a priority across digital health interventions. Text messaging is a proven smoking cessation treatment modality and a powerful strategy to increase intervention engagement in other areas of health, but it has not been tested as an engagement strategy for a digital cessation intervention. OBJECTIVE:This study examined the impact of 4 experimental text message design factors on adult smokers' engagement with an internet smoking cessation program. METHODS:We conducted a 2×2×2×2 full factorial screening experiment wherein 864 participants were randomized to 1 of 16 experimental conditions after registering with a free internet smoking cessation program and enrolling in its automated text message program. Experimental factors were personalization (on/off), integration between the web and text message platforms (on/off), dynamic tailoring of intervention content based on user engagement (on/off), and message intensity (tapered vs abrupt drop-off). Primary outcomes were 3-month measures of engagement (ie, page views, time on site, and return visits to the website) as well as use of 6 interactive features of the internet program. All metrics were automatically tracked; there were no missing data. RESULTS:Main effects were detected for integration and dynamic tailoring. Integration significantly increased interactive feature use by participants, whereas dynamic tailoring increased the number of features used and page views. No main effects were found for message intensity or personalization alone, although several synergistic interactions with other experimental features were observed. Synergistic effects, when all experimental factors were active, resulted in the highest rates of interactive feature use and the greatest proportion of participants at high levels of engagement. Measured in terms of standardized mean differences (SMDs), effects on interactive feature use were highest for Build Support System (SMD 0.56; 95% CI 0.27 to 0.81), Choose Quit Smoking Aid (SMD 0.38; 95% CI 0.10 to 0.66), and Track Smoking Triggers (SMD 0.33; 95% CI 0.05 to 0.61). Among the engagement metrics, the largest effects were on overall feature utilization (SMD 0.33; 95% CI 0.06 to 0.59) and time on site (SMD 0.29; 95% CI 0.01 to 0.57). As no SMD >0.30 was observed for main effects on any outcome, results suggest that for some outcomes, the combined intervention was stronger than individual factors alone. CONCLUSIONS:This factorial experiment demonstrates the effectiveness of text messaging as a strategy to increase engagement with an internet smoking cessation intervention, resulting in greater overall intervention dose and greater exposure to the core components of tobacco dependence treatment that can promote abstinence. TRIAL REGISTRATION:ClinicalTrials.gov NCT02585206; https://clinicaltrials.gov/ct2/show/NCT02585206. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID):RR2-10.1136/bmjopen-2015-010687.

Project description:BackgroundTailored, web-assisted interventions can reach many smokers. Content from other smokers (peers) through crowdsourcing could enhance relevance.PurposeTo evaluate whether peers can generate tailored messages encouraging other smokers to use a web-assisted tobacco intervention (Decide2Quit.org).MethodsPhase 1: In 2009, smokers wrote messages in response to scenarios for peer advice. These smoker-to-smoker (S2S) messages were coded to identify themes. Phase 2: resulting S2S messages, and comparison expert messages, were then e-mailed to newly registered smokers. In 2012, subsequent Decide2Quit.org visits following S2S or expert-written e-mails were compared.ResultsPhase 1: a total of 39 smokers produced 2886 messages (message themes: attitudes and expectations, improvements in quality of life, seeking help, and behavioral strategies). For not-ready-to-quit scenarios, S2S messages focused more on expectations around a quit attempt and how quitting would change an individual's quality of life. In contrast, for ready-to-quit scenarios, S2S messages focused on behavioral strategies for quitting. Phase 2: In multivariable analysis, S2S messages were more likely to generate a return visit (OR=2.03, 95% CI=1.74, 2.35), compared to expert messages. A significant effect modification of this association was found, by time-from-registration and message codes (both interaction terms p<0.01). In stratified analyses, S2S codes that were related more to "social" and "real-life" aspects of smoking were driving the main association of S2S and increased return visits.ConclusionsS2S peer messages that increased longitudinal engagement in a web-assisted tobacco intervention were successfully collected and delivered. S2S messages expanded beyond the biomedical model to enhance relevance of messages.Trial registrationThis study is registered at www.clinicaltrials.gov NCT00797628 (web-delivered provider intervention for tobacco control [QUIT-PRIMO]) and NCT01108432 (DPBRN Hygienists Internet Quality Improvement in Tobacco Cessation [HiQuit]).

Project description:IntroductionUsing WebQuit as a case study, a smoking cessation website grounded in Acceptance and Commitment Therapy, we aimed to identify sequence clusters of content usage and examine their associations with baseline characteristics, change to a key mechanism of action, and smoking cessation.MethodsParticipants were adult smokers allocated to the WebQuit arm in a randomized controlled trial (n = 1,313). WebQuit contains theory-informed content including goal setting, self-monitoring and feedback, and values- and acceptance-based exercises. Sequence analysis was used to temporally order 30-s website usage segments for each participant. Similarities between sequences were assessed with the optimal matching distance algorithm and used as input in an agglomerative hierarchical clustering analysis. Associations between sequence clusters and baseline characteristics, acceptance of cravings at 3 months and self-reported 30-day point prevalence abstinence at 12 months were examined with linear and logistic regression.ResultsThree qualitatively different sequence clusters were identified. "Disengagers" (576/1,313) almost exclusively used the goal-setting feature. "Tryers" (375/1,313) used goal setting and two of the values- and acceptance-based components ("Be Aware," "Be Willing"). "Committers" (362/1,313) primarily used two of the values- and acceptance-based components ("Be Willing," "Be Inspired"), goal setting, and self-monitoring and feedback. Compared with Disengagers, Committers demonstrated greater increases in acceptance of cravings (p = .01) and 64% greater odds of quit success (ORadj = 1.64, 95% CI = 1.18, 2.29, p = .003).DiscussionWebQuit users were categorized into Disengagers, Tryers, and Committers based on their qualitatively different content usage patterns. Committers saw increases in a key mechanism of action and greater odds of quit success.ImplicationsThis case study demonstrates how employing sequence and cluster analysis of usage data can help researchers and practitioners gain a better understanding of how users engage with a given eHealth intervention over time and use findings to test theory and/or to improve future iterations to the intervention. Future WebQuit users may benefit from being directed to the values- and acceptance-based and the self-monitoring and feedback components via reminders over the course of the program.

Project description:Patient self-management interventions for smoking cessation are effective but underused. Health care providers do not routinely refer smokers to these interventions.The objective of our study was to uncover barriers and facilitators to the use of an e-referral system that will be evaluated in a community-based randomized trial. The e-referral system will allow providers to refer smokers to an online smoking intervention during routine clinical care.We devised a four-step development and pilot testing process: (1) system conceptualization using Delphi to identify key functionalities that would overcome barriers in provider referrals for smoking cessation, (2) Web system programming using agile software development and best programming practices with usability refinement using think-aloud testing, (3) implementation planning using the nominal group technique for the effective integration of the system into the workflow of practices, and (4) pilot testing to identify practice recruitment and system-use barriers in real-world settings.Our Delphi process (step 1) conceptualized three key e-referral functions: (1) Refer Your Smokers, allowing providers to e-refer patients at the point of care by entering their emails directly into the system, (2) practice reports, providing feedback regarding referrals and impact of smoking-cessation counseling, and (3) secure messaging, facilitating provider-patient communication. Usability testing (step 2) suggested the system was easy to use, but implementation planning (step 3) suggested several important approaches to encourage use (eg, proactive email cues to encourage practices to participate). Pilot testing (step 4) in 5 practices had limited success, with only 2 patients referred; we uncovered important recruitment and system-use barriers (eg, lack of study champion, training, and motivation, registration difficulties, and forgetting to refer).Implementing a system to be used in a clinical setting is complex, as several issues can affect system use. In our ongoing large randomized trial, preliminary analysis with the first 50 practices using the system for 3 months demonstrated that our rigorous preimplementation evaluation helped us successfully identify and overcome these barriers before the main trial.Clinicaltrials.gov NCT00797628; http://clinicaltrials.gov/ct2/show/NCT00797628 (Archived by WebCite at http://www.webcitation.org/61feCfjCy).