Dataset Information

Text Message Analysis Using Machine Learning to Assess Predictors of Engagement With Mobile Health Chronic Disease Prevention Programs: Content Analysis.

ABSTRACT:

Background

SMS text messages as a form of mobile health are increasingly being used to support individuals with chronic diseases in novel ways that leverage the mobility and capabilities of mobile phones. However, there are knowledge gaps in mobile health, including how to maximize engagement.

Objective

This study aims to categorize program SMS text messages and participant replies using machine learning (ML) and to examine whether message characteristics are associated with premature program stopping and engagement.

Methods

We assessed communication logs from SMS text message-based chronic disease prevention studies that encouraged 1-way (SupportMe/ITM) and 2-way (TEXTMEDS [Text Messages to Improve Medication Adherence and Secondary Prevention]) communication. Outgoing messages were manually categorized into 5 message intents (informative, instructional, motivational, supportive, and notification) and replies into 7 groups (stop, thanks, questions, reporting healthy, reporting struggle, general comment, and other). Grid search with 10-fold cross-validation was implemented to identify the best-performing ML models and evaluated using nested cross-validation. Regression models with interaction terms were used to compare the association of message intent with premature program stopping and engagement (replied at least 3 times and did not prematurely stop) in SupportMe/ITM and TEXTMEDS.

Results

We analyzed 1550 messages and 4071 participant replies. Approximately 5.49% (145/2642) of participants responded with stop, and 11.7% (309/2642) of participants were engaged. Our optimal ML model correctly classified program message intent with 76.6% (95% CI 63.5%-89.8%) and replies with 77.8% (95% CI 74.1%-81.4%) balanced accuracy (average area under the curve was 0.95 and 0.96, respectively). Overall, supportive (odds ratio [OR] 0.53, 95% CI 0.35-0.81) messages were associated with reduced chance of stopping, as were informative messages in SupportMe/ITM (OR 0.35, 95% CI 0.20-0.60) but not in TEXTMEDS (for interaction, P<.001). Notification messages were associated with a higher chance of stopping in SupportMe/ITM (OR 5.76, 95% CI 3.66-9.06) but not TEXTMEDS (for interaction, P=.01). Overall, informative (OR 1.76, 95% CI 1.46-2.12) and instructional (OR 1.47, 95% CI 1.21-1.80) messages were associated with higher engagement but not motivational messages (OR 1.18, 95% CI 0.82-1.70; P=.37). For supportive messages, the association with engagement was opposite with SupportMe/ITM (OR 1.77, 95% CI 1.21-2.58) compared with TEXTMEDS (OR 0.77, 95% CI 0.60-0.98; for interaction, P<.001). Notification messages were associated with reduced engagement in SupportMe/ITM (OR 0.07, 95% CI 0.05-0.10) and TEXTMEDS (OR 0.28, 95% CI 0.20-0.39); however, the strength of the association was greater in SupportMe/ITM (for interaction P<.001).

Conclusions

ML models enable monitoring and detailed characterization of program messages and participant replies. Outgoing message intent may influence premature program stopping and engagement, although the strength and direction of association appear to vary by program type. Future studies will need to examine whether modifying message characteristics can optimize engagement and whether this leads to behavior change.

SUBMITTER: Klimis H

PROVIDER: S-EPMC8663456 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundInterest in mobile health (mHealth) has increased recently, and research suggests that mHealth devices can enhance end-user engagement, especially when used in conjunction with brief message content.ObjectiveThis research aims to explore the stages of engagement framework for mHealth devices and develop a method to generate brief message content to promote sustained user engagement. This study uses the framework by O'Brien and Toms as a point of departure, where engagement is defined as the uptake or the use of an mHealth device. The framework is a linear repeatable process, including point of engagement, period of engagement, disengagement, and re-engagement. Each stage is characterized by attributes related to a person's technology experience. Although the literature has identified stages of engagement for health-related technology, few studies explore mHealth engagement. Furthermore, little research has determined a method for creating brief message content at each stage in this engagement journey.MethodsInterviews and observations from 19 participants who used mHealth technologies (apps, devices, or wellness websites) in a solo capacity were recruited for sample group 1. In sample group 2, interviews, and observations from 25 participants using mHealth technologies in a group capacity through the Global Corporate Challenge were used. These samples were investigated at 3 time points in both research contexts. The results underwent deductive-inductive thematic analysis for the engagement stages' framework and attributes.ResultsIn addition to the 4 stages identified by O'Brien and Toms, 2 additional stages, self-management and limited engagement, were identified. Self-management captures where users had disengaged from their technology but were still engaged with their health activity. Limited engagement captures where group mHealth users had minimal interaction with their mHealth technology but continued to engage in a group fitness activity. The results revealed that mHealth engagement stages were nonlinear and embedded in a wider engagement context and that each stage was characterized by a combination of 49 attributes that could be organized into 8 themes. Themes documented the total user experience and included technology usability, technology features, technology aesthetics, use motivations, health awareness, goal setting, social support, and interruptions. Different themes were found to have more relevance at different engagement stages. Knowing themes and attributes at all engagement stages allows technology developers and health care professionals to generate relevant brief message content informed by a person-centered approach.ConclusionsThis research extends an existing engagement stages framework and identifies attributes and themes relevant to mHealth technology users' total user experience and incorporates concepts derived from health, business studies, and information systems literature. In addition, we offer a practical 5-step process based on a person-centered approach to develop mHealth technology brief message content for sustained engagement.

Project description:IntroductionMany studies have now demonstrated the efficacy of text messaging in positively changing behaviours. We aimed to identify features and factors that explain the effectiveness of a successful text messaging program in terms of user engagement, perceived usefulness, behavior change and program delivery preferences.MethodsMixed methods qualitative design combining four data sources; (i) analytic data extracted directly from the software system, (ii) participant survey, (iii) focus groups to identify barriers and enablers to implementation and mechanisms of effect and (iv) recruitment screening logs and text message responses to examine engagement. This evaluation was conducted within the TEXT ME trial-a parallel design, single-blind randomized controlled trial (RCT) of 710 patients with coronary heart disease (CHD). Qualitative data were interpreted using inductive thematic analysis.Results307/352 (87% response rate) of recruited patients with CHD completed the program evaluation survey at six months and 25 participated in a focus group. Factors increasing engagement included (i) ability to save and share messages, (ii) having the support of providers and family, (iii) a feeling of support through participation in the program, (iv) the program being initiated close to the time of a cardiovascular event, (v) personalization of the messages, (vi) opportunity for initial face-to-face contact with a provider and (vii) that program and content was perceived to be from a credible source. Clear themes relating to program delivery were that diet and physical activity messages were most valued, four messages per week was ideal and most participants felt program duration should be provided for at least for six months or longer.ConclusionsThis study provides context and insight into the factors influencing consumer engagement with a text message program aimed at improving health-related behavior. The study suggests program components that may enhance potential success but will require integration at the development stage to optimize up-scaling.Trial registrationAustralia and New Zealand Clinical Trials Registry, ACTRN12611000161921.

Project description:BackgroundImproved routine immunization (RI) coverage is recommended as the priority public health strategy to decrease vaccine-preventable diseases and eradicate polio in Pakistan and worldwide.ObjectiveThe objective of this study was to ascertain whether customized, automated, one-way text messaging (short message service, SMS) reminders delivered to caregivers via mobile phones when a child is due for an RI visit can improve vaccination uptake and timelines in Pakistan.MethodsThis was a randomized controlled trial, conducted in an urban squatter settlement area of Karachi, Pakistan. Infants less than 2 weeks of age with at least one family member who had a valid mobile phone connection and was comfortable receiving and reading SMS text messages were included. Participants were randomized to the intervention (standard care + one-way SMS reminder) or control (standard care) groups. The primary outcome was to compare the proportion of children immunized up to date at 18 weeks of age. Vaccine given at 6, 10, and 14 weeks schedule includes DPT-Hep-B-Hib vaccine (ie, diphtheria, pertussis, and tetanus; hepatitis B; and Haemophilus influenza type b) and oral poliovirus vaccine (OPV). Data were analyzed using chi-square tests of independence and tested for both per protocol (PP) and intention-to-treat (ITT) analyses.ResultsOut of those approached, 84.3% (300/356) of the participants were eligible for enrollment and 94.1% (318/338) of the participants had a working mobile phone. Only children in the PP analyses, who received an SMS reminder for vaccine uptake at 6 weeks visit, showed a statistically significant difference (96.0%, 86/90 vs 86.4%, 102/118; P=.03).The immunization coverage was consistently higher in the intervention group according to ITT analyses at the 6 weeks scheduled visit (76.0% vs 71.3%, P=.36). The 10 weeks scheduled visit (58.7% vs 52.7%, P=.30) and the 14 weeks scheduled visit (31.3% vs 26.0%, P=.31), however, were not statistically significant.ConclusionsAutomated simple one-way SMS reminders in local languages might be feasible for improving routine vaccination coverage. Whether one-way SMS reminders alone can have a strong impact on parental attitudes and behavior for improvement of RI coverage and timeliness needs to be further evaluated by better-powered studies and by comparing different types and content of text messages in low-and middle-income countries (LMICs).Trial registrationClinicalTrials.gov NCT01859546; https://clinicaltrials.gov/ct2/show/NCT01859546 (Archived by WebCite at http://www.webcitation.org/6xFr57AOc).

Project description:BackgroundConsidering the extreme shortage of smoking cessation services in China, and the acceptability, feasibility and efficacy of mobile phone-based text message interventions for quitting smoking in other countries, here we propose a study of "the efficacy of mobile phone-based text message interventions ('Happy Quit') for smoking cessation in China". The primary objective of this proposed project is to assess whether a program of widely accessed mobile phone-based text message interventions ('Happy Quit') will be effective at helping people in China who smoke, to quit. Based on the efficacy of previous studies in smoking cessation, we hypothesize that 'Happy Quit' will be an effective, feasible and affordable smoking cessation program in China.Methods/designIn this single-blind, randomized trial, undertaken in China, about 2000 smokers willing to make a quit attempt will be randomly allocated, using an independent telephone randomization system that includes a minimization algorithm balancing for sex (male, female), age (19-34 or >34 years), educational level (≤ or >12 years), and Fagerstrom score for nicotine addiction (≤5, >5), to 'Happy Quit', comprising motivational messages and behavioral-change support, or to a control group that receives text messages unrelated to quitting. Messages will be developed to be suitable for Chinese. A pilot study will be conducted before the intervention to modify the library of messages and interventions. The primary outcome will be self-reported continuous smoking abstinence. A secondary outcome will be point prevalence of abstinence. Abstinence will be assessed at six time points (4, 8, 12, 16, 20 and 24 weeks post-intervention). A third outcome will be reductions in number of cigarettes smoked per day.Discussion/implicationsThe results will provide valuable insights into bridging the gap between need and services received for smoking cessation interventions and tobacco use prevention in China. It will also serve as mHealth model for extending the public health significance of other interventions, such as mental health interventions.Trial registrationNCT02693626 (Registration data April 11, 2016).

Project description:BackgroundEngagement with digital health interventions (DHIs) may be regarded as a prerequisite for the intervention to achieve positive health or behavior change outcomes. One method employed to promote engagement is the use of prompts such as emails and text messages. However, little is known about the characteristics of prompts that promote engagement. This study explored the association between the content and delivery mode of prompts and the users' engagement with HeLP-Diabetes (Healthy Living for People with type 2 Diabetes), a DHI that aimed to promote self-management in adults with type 2 diabetes.ObjectiveThe objective of this study was to identify the characteristics of prompts, specifically the content and delivery mode, which were associated with increased engagement.MethodsThis was a mixed-methods study. Email and text message prompts were sent to the registered users of HeLP-Diabetes. Use of the intervention was recorded and examined to identify which email and text message prompts were associated with subsequent visits to the DHI. Characteristics of prompts that were identified as particularly effective or ineffective were explored through think-aloud interviews with the participants.ResultsOf a total of 39 email prompts, 49% (19/39) prompts showed a significant association with subsequent visits to the DHI. However, none of the text message prompts were associated with subsequent visits to the DHI. Furthermore, think-aloud interviews were carried out with 6 experienced participants with type 2 diabetes. The findings suggest that these participants preferred email prompts that were clear, relatively short, and empowering; used nondirective advice; included health professional references; were visually appealing; and contained news and updates.ConclusionsThe findings of this study contribute to the existing evidence supporting the role of email prompts in promoting and maintaining engagement with DHIs. This study described the content of prompts that may be engaging. However, the results should be interpreted with caution, as prompts may be context-specific interventions and the results may not be generalizable across other DHIs or other types of interventions targeting self-management of type 2 diabetes.

Project description:BackgroundHeavy episodic (binge) drinking is common among young adults and can lead to injury and illness. Young adults who seek care in the Emergency Department (ED) may be disproportionately affected with binge drinking behavior, therefore provide an opportunity to reduce future risk through screening, brief intervention and referral to treatment (SBIRT). Mobile phone text messaging (SMS) is a common form of communication among young adults and has been shown to be effective at providing behavioral support to young adult drinkers after ED discharge. Efficacy of SMS programs to reduce binge drinking remains unknown.Methods/designWe will conduct a three parallel arm, randomized trial. A convenience sample of adults aged 18 to 25 years attending three EDs in Pittsburgh, PA and willing to participate in the study will be screened for hazardous alcohol consumption. Participants identified as hazardous drinkers will then be allocated to either 12 weeks of weekly SMS drinking assessments with feedback (SA+F), SMS drinking assessments without feedback (SA), or a control group. Randomization will be via an independent and remote computerized randomization and will be stratified by study site. The SA+F group will be asked to provide pre-weekend drinking intention as well as post-weekend consumption via SMS and will receive feedback messages focused on health consequences of alcohol consumption, personalized normative feedback, protective drinking strategies and goal setting. Follow-up data on alcohol use and injury related to alcohol will be collected through a password-protected website three, six and nine months later. The primary outcome for the study is binge drinking days (≥4 drinks for women; ≥5 drinks for men) during the previous month, and the main secondary outcome is the proportion of participants who report any injury related to alcohol in the prior three months.DiscussionThis study will test the hypothesis that a mobile phone text-messaging program will result in immediate and durable reductions in binge drinking among at-risk young adults. By testing an intervention group to an assessment-only and control group, we will be able to separate the effect of assessment reactivity. By collecting pre-weekend drinking intentions and post-weekend consumption data in the SA+F group, we will be able to better understand mechanism of change.Trial registrationClinicaltrials.gov NCT01688245.