Dataset Information

Optimizing Adaptive Notifications in Mobile Health Interventions Systems: Reinforcement Learning from a Data-driven Behavioral Simulator.

ABSTRACT: Mobile health (mHealth) intervention systems can employ adaptive strategies to interact with users. Instead of designing such complex strategies manually, reinforcement learning (RL) can be used to adaptively optimize intervention strategies concerning the user's context. In this paper, we focus on the issue of overwhelming interactions when learning a good adaptive strategy for the user in RL-based mHealth intervention agents. We present a data-driven approach integrating psychological insights and knowledge of historical data. It allows RL agents to optimize the strategy of delivering context-aware notifications from empirical data when counterfactual information (user responses when receiving notifications) is missing. Our approach also considers a constraint on the frequency of notifications, which reduces the interaction burden for users. We evaluated our approach in several simulation scenarios using real large-scale running data. The results indicate that our RL agent can deliver notifications in a manner that realizes a higher behavioral impact than context-blind strategies.

SUBMITTER: Wang S

PROVIDER: S-EPMC8523513 | biostudies-literature | 2021 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Optimizing Adaptive Notifications in Mobile Health Interventions Systems: Reinforcement Learning from a Data-driven Behavioral Simulator.

Wang Shihan S Zhang Chao C Kröse Ben B van Hoof Herke H

Journal of medical systems 20211018 12

Mobile health (mHealth) intervention systems can employ adaptive strategies to interact with users. Instead of designing such complex strategies manually, reinforcement learning (RL) can be used to adaptively optimize intervention strategies concerning the user's context. In this paper, we focus on the issue of overwhelming interactions when learning a good adaptive strategy for the user in RL-based mHealth intervention agents. We present a data-driven approach integrating psychological insights ...[more]

PMID: 34664120

Similar Datasets

Project description:BackgroundA just-in-time adaptive intervention (JITAI) is "designed to address the dynamically changing needs of individuals via the provision of the type or amount of support needed, at the right time when needed." If and how rumination-focused cognitive behavioral therapy (RFCBT), the gold standard, blocks emotional cascades underlying rumination is unclear. Furthermore, cognitive behavioral therapy has been successfully used as a mobile variant, but RFCBT has not been adapted for a mobile variant (mobile RFCBT [MRFCBT]) or for a JITAI variant.ObjectiveThis study aimed to pilot-test a fully automated JITAI leveraging RFCBT and ways to identify and block cascading depressive rumination.MethodsPatients in therapy for clinical depression were recruited for a randomized controlled trial (RCT). After consenting to be part of the RCT, they were randomly assigned to either of the 2 mobile versions of the RFCBT conditions personalized to the individual's rumination timing patterns (JITAI-MRFCBT) or a no-treatment control condition through a double-blind procedure. Although the initial design was to have a 3-armed trial with 2 JITAI conditions (a JITAI and a narrative JITAI condition), we later opted to collapse those 2 conditions into 1 JITAI condition because of the low number of participants. All participants were recruited and participated through their smartphones, receiving 5 SMS text message reminders on each of the 35 days to self-report their rumination-related symptoms (eg, rumination episodes and duration). In the JITAI-MRFCBT condition, they also received treatment materials. The first 7 days provided a rumination baseline, and the last 7 days provided a postintervention rumination value. In total, 42% (25/59) of volunteers were eligible and provided their phone numbers, 20% (5/25) of whom never replied to the SMS text message reminding them to start the RCT. A total of 90% (18/20) of volunteers completed it (ie, finishing, as prespecified, 80% of the questionnaires and training tasks) and, therefore, were included in the analysis.ResultsUsing independent 2-tailed t tests with bootstrapping, results showed that participants in the JITAI-MRFCBT condition, compared with those in the control condition, reported a greater reduction in counts of rumination episodes (mean -25.28, SD 14.50 vs mean 1.44, SD 4.12, P<.001) and greater reduced average time (minutes) spent in rumination (mean -21.53, SD 17.6 vs mean 1.47, SD 1.5; P=.04). Results also suggest that, compared with those in the control group, those in treatment reduced ruminative carryover from one episode to the next.ConclusionsThe results suggest that JITAI-MRFCBT may reduce negative rumination by providing RFCBT just in time following rumination, thereby blocking the next rumination episode using the same trigger. This study supports a subsequent, full-scale JITAI and the importance of leveraging mobile smartphone technology with MRFCBT to curb depressive symptoms.Trial registrationClinicalTrials.gov NCT04554706; https://clinicaltrials.gov/ct2/show/NCT04554706.

Project description:BackgroundMobile health (mHealth) interventions are increasingly being designed to facilitate health-related behavior change. Integrating insights from behavioral science and design science can help support the development of more effective mHealth interventions. Behavioral Design (BD) and Design Thinking (DT) have emerged as best practice approaches in their respective fields. Until now, little work has been done to examine how BD and DT can be integrated throughout the mHealth design process.ObjectiveThe aim of this scoping review was to map the evidence on how insights from BD and DT can be integrated to guide the design of mHealth interventions. The following questions were addressed: (1) what are the main characteristics of studies that integrate BD and DT during the mHealth design process? (2) what theories, models, and frameworks do design teams use during the mHealth design process? (3) what methods do design teams use to integrate BD and DT during the mHealth design process? and (4) what are key design challenges, implementation considerations, and future directions for integrating BD and DT during mHealth design?MethodsThis review followed the Joanna Briggs Institute reviewer manual and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist. Studies were identified from MEDLINE, PsycINFO, Embase, CINAHL, and JMIR by using search terms related to mHealth, BD, and DT. Included studies had to clearly describe their mHealth design process and how behavior change theories, models, frameworks, or techniques were incorporated. Two independent reviewers screened the studies for inclusion and completed the data extraction. A descriptive analysis was conducted.ResultsA total of 75 papers met the inclusion criteria. All studies were published between 2012 and 2021. Studies integrated BD and DT in notable ways, which can be referred to as "Behavioral Design Thinking." Five steps were followed in Behavioral Design Thinking: (1) empathize with users and their behavior change needs, (2) define user and behavior change requirements, (3) ideate user-centered features and behavior change content, (4) prototype a user-centered solution that supports behavior change, and (5) test the solution against users' needs and for its behavior change potential. The key challenges experienced during mHealth design included meaningfully engaging patient and public partners in the design process, translating evidence-based behavior change techniques into actual mHealth features, and planning for how to integrate the mHealth intervention into existing clinical systems.ConclusionsBest practices from BD and DT can be integrated throughout the mHealth design process to ensure that mHealth interventions are purposefully developed to effectively engage users. Although this scoping review clarified how insights from BD and DT can be integrated during mHealth design, future research is needed to identify the most effective design approaches.

Dataset Information

Optimizing Adaptive Notifications in Mobile Health Interventions Systems: Reinforcement Learning from a Data-driven Behavioral Simulator.

Publications

Optimizing Adaptive Notifications in Mobile Health Interventions Systems: Reinforcement Learning from a Data-driven Behavioral Simulator.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets