Dataset Information

Assessing Digital Health Implementation for a Pediatric Chronic Pain Intervention: Comparing the RE-AIM and BIT Frameworks Against Real-World Trial Data and Recommendations for Future Studies.

ABSTRACT: BACKGROUND:Digital health interventions have demonstrated efficacy for several conditions including for pediatric chronic pain. However, the process of making interventions available to end users in an efficient and sustained way is challenging and remains a new area of research. To advance this field, comprehensive frameworks have been created. OBJECTIVE:The aim of this study is to compare the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) and Behavior Interventions using Technology (BIT) frameworks with data collected from the web-based management of adolescent pain (WebMAP Mobile; WMM) randomized controlled trial (RCT). METHODS:We conducted a hybrid effectiveness-implementation cluster RCT with a stepped wedge design in which the intervention was sequentially implemented in 8 clinics, following a usual care period. Participants were 143 youths (mean age 14.5 years, SD 1.9; 117/143, 81.8% female) with chronic pain, from which 73 were randomized to receive the active intervention. Implementation outcomes were assessed using the RE-AIM and BIT frameworks. RESULTS:According to the RE-AIM framework, the WMM showed excellent reach, recruiting a sample 19% larger than the size originally planned and consenting 79.0% (143/181) of eligible referred adolescents. Effectiveness was limited, with only global impression of change showing significantly greater improvements in the treatment group; however, greater treatment engagement was associated with greater reductions in pain and disability. Adoption was excellent (all the invited clinics participated and referred patients). Implementation was acceptable, showing good user engagement and moderate adherence and positive attitudes of providers. Costs were similar to planned, with a 7% increase in funds needed to make the WMM publicly available. Maintenance was evidenced by 56 new patients downloading the app during the maintenance period and by all clinics agreeing to continue making referrals and all, but one, making new referrals. According to the BIT, 82% (60/73) of adolescents considered the treatment acceptable. In terms of adoption, 93% (68/73) downloaded the app, and all of them used it after their first log-in. In terms of appropriateness at the user level, 2 participants were unable to download the app. Perceptions of the appearance, navigation, and theme were positive. Providers perceived the WMM as a good fit for their clinic, beneficial, helpful, and resource efficient. In terms of feasibility, no technical issues were reported. In terms of fidelity, 40% (29/73) completed the treatment. Implementation costs were 7% above the budget. With regard to penetration, 56 new users accessed the app during the maintenance period. In terms of sustainability, 88% (7/8) of clinics continued recommending the WMM after the end of the study. CONCLUSIONS:For the first time, a real-world digital health intervention was used as a proof of concept to test all the domains in the RE-AIM and BIT frameworks, allowing for comparisons. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID):RR2-10.1016/j.cct.2018.10.003.

SUBMITTER: de la Vega R

PROVIDER: S-EPMC7492980 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing Digital Health Implementation for a Pediatric Chronic Pain Intervention: Comparing the RE-AIM and BIT Frameworks Against Real-World Trial Data and Recommendations for Future Studies.

de la Vega Rocio R Ritterband Lee L Palermo Tonya M TM

Journal of medical Internet research 20200901 9

<h4>Background</h4>Digital health interventions have demonstrated efficacy for several conditions including for pediatric chronic pain. However, the process of making interventions available to end users in an efficient and sustained way is challenging and remains a new area of research. To advance this field, comprehensive frameworks have been created.<h4>Objective</h4>The aim of this study is to compare the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) and Behavior I ...[more]

PMID: 32870158

Similar Datasets

Project description:Background: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) and CFIR (Consolidated Framework for Implementation Research) dissemination and implementation frameworks define theory-based domains associated with the adoption, implementation and maintenance of evidence-based interventions. Used together, the two frameworks identify metrics for evaluating implementation success, i.e., high reach and effectiveness resulting in sustained practice change (RE-AIM), and modifiable factors that explain and enhance implementation outcomes (CFIR). We applied both frameworks to study the implementation planning process for a technology-delivered asthma care intervention called Breathewell within an integrated care organization. The goal of the Breathewell intervention is to increase the efficiency of delivering resource-intensive asthma care services. Methods: We reviewed historical documents (i.e., meeting agendas; minutes) from 14 months of planning to evaluate alignment of implementation team priorities with RE-AIM domains. Key content was extracted and analyzed on topics, frequency and amount of discussion within each RE-AIM domain. Implementation team members were interviewed using questions adapted from the CFIR Interview Guide Tool to focus their reflection on the process and contextual factors considered during pre-implementation planning. Documents and transcripts were initially coded using RE-AIM domain definitions, and recoded using CFIR constructs, with intent to help explain how team decisions and actions can contribute to adoption, implementation and maintenance outcomes. Results: Qualitative analysis of team documents and interviews demonstrated strong alignment with the RE-AIM domains: Reach, Effectiveness, and Implementation; and with the CFIR constructs: formal inclusion of provider and staff stakeholders in implementation planning, compatibility of the intervention with workflows and systems, and alignment of the intervention with organizational culture. Focus on these factors likely contributed to RE-AIM outcomes of high implementation fidelity. However, team members expressed low confidence that Breathewell would be adopted and maintained post-trial. A potential explanation was weak alignment with several CFIR constructs, including tension for change, relative priority, and leadership engagement that contribute to organizational receptivity and motivation to sustain change. Conclusions: While RE-AIM provides a practical framework for planning and evaluating practice change interventions to assure their external validity, CFIR explains why implementation succeeded or failed, and when used proactively, identifies relevant modifiable factors that can promote or undermine adoption, implementation, and maintenance.

Project description:Background: The RE-AIM framework has been widely used in health research but it is unclear the extent to which this framework is also used for planning and evaluating health-related programs in clinical and community settings. Our objective was to evaluate how RE-AIM is used in the "real-world" and identify opportunities for improving use outside of research contexts. Methods: We used purposive and snowball sampling to identify clinical and community health programs that used RE-AIM for planning and/or evaluation. Recruitment methods included surveys with email follow-up to funders, implementers, and RE-AIM working group members. We identified 17 programs and conducted structured in-depth interviews with key informants (n = 18). Across RE-AIM dimensions, respondents described motivations, uses, and measures; rated understandability and usefulness; discussed benefits and challenges, strategies to overcome challenges, and resources used. We used descriptive statistics for quantitative ratings, and content analysis for qualitative data. Results: Program content areas included chronic disease management and prevention, healthy aging, mental health, or multiple, often behavioral health-related topics. During planning, most programs considered reach (n = 9), adoption (n = 11), and implementation (n = 12) while effectiveness (n = 7) and maintenance (n = 6) were considered less frequently. In contrast, most programs evaluated all RE-AIM dimensions, ranging from 13 programs assessing maintenance to 15 programs assessing implementation and effectiveness. On five-point scales, all RE-AIM dimensions were rated as easy to understand (Overall M = 4.7 ± 0.5), but obtaining data was rated as somewhat challenging (Overall M = 3.4 ± 0.9). Implementation was the most frequently used dimension to inform program design (M = 4.7 ± 0.6) relative to the other dimensions (3.0-3.9). All dimensions were considered similarly important for decision-making (average M = 4.1 ± 1.4), with the exception of maintenance (M = 3.4 ± 1.7). Qualitative corresponded to the quantitative findings in that RE-AIM was reported to be a practical, easy to understand, and well-established implementation science framework. Challenges included understanding differences among RE-AIM dimensions and data acquisition. Valuable resources included the RE-AIM website and collaborating with an expert. Discussion: RE-AIM is an efficient framework for planning and evaluation of clinical and community-based projects. It provides structure to systematically evaluate health program impact. Programs found planning for and assessing maintenance difficult, providing opportunities for further refinement.

Project description:Background:Recent reviews of the use and application of implementation frameworks in implementation efforts highlight the limited use of frameworks, despite the value in doing so. As such, this article aims to provide recommendations to enhance the application of implementation frameworks, for implementation researchers, intermediaries, and practitioners. Discussion:Ideally, an implementation framework, or multiple frameworks should be used prior to and throughout an implementation effort. This includes both in implementation science research studies and in real-world implementation projects. To guide this application, outlined are ten recommendations for using implementation frameworks across the implementation process. The recommendations have been written in the rough chronological order of an implementation effort; however, we understand these may vary depending on the project or context: (1) select a suitable framework(s), (2) establish and maintain community stakeholder engagement and partnerships, (3) define issue and develop research or evaluation questions and hypotheses, (4) develop an implementation mechanistic process model or logic model, (5) select research and evaluation methods (6) determine implementation factors/determinants, (7) select and tailor, or develop, implementation strategy(s), (8) specify implementation outcomes and evaluate implementation, (9) use a framework(s) at micro level to conduct and tailor implementation, and (10) write the proposal and report. Ideally, a framework(s) would be applied to each of the recommendations. For this article, we begin by discussing each recommendation within the context of frameworks broadly, followed by specific examples using the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Summary:The use of conceptual and theoretical frameworks provides a foundation from which generalizable implementation knowledge can be advanced. On the contrary, superficial use of frameworks hinders being able to use, learn from, and work sequentially to progress the field. Following the provided ten recommendations, we hope to assist researchers, intermediaries, and practitioners to improve the use of implementation science frameworks.

Project description:Promoting walking for the journey to/from work and during the working day is one potential approach to increase physical activity in adults. Walking Works was a practice-led, whole-workplace walking programme delivered by employees (walking champions). This study aimed to evaluate the implementation of Walking Works using the RE-AIM framework and provide recommendations for future delivery of whole-workplace walking programmes.Two cross sectional surveys were conducted; 1544 (28%) employees completed the baseline survey and 918 employees (21%) completed the follow-up survey. Effectiveness was assessed using baseline and follow-up data; reach, implementation and maintenance were assessed using follow-up data only. For categorical data, Chi square tests were conducted to assess differences between surveys or groups. Continuous data were analysed to test for significant differences using a Mann-Whitney U test. Telephone interviews were conducted with the lead organisation co-ordinator, eight walking champions and three business representatives at follow-up. Interviews were transcribed verbatim and analysed to identify key themes related to adoption, implementation and maintenance.Adoption: Five workplaces participated in Walking Works. Reach: 480 (52.3%) employees were aware of activities and 221 (24.1%) participated.A variety of walking activities were delivered. Some programme components were not delivered as planned which was partly due to barriers in using walking champions to deliver activities. These included the walking champions' capacity, skills, support needs, ability to engage senior management, and the number and type of activities they could deliver. Other barriers included lack of management support, difficulties communicating information about activities and challenges embedding the programme into normal business activities. Effectiveness: No significant changes in walking to/from work or walking during the working day were observed. Maintenance: Plans to continue activities were mainly dependent on identifying continued funding.RE-AIM provided a useful framework for evaluating Walking Works. No changes in walking behaviour were observed. This may have been due to barriers in using walking champions to deliver activities, programme components not being delivered as intended, the types of activities delivered, or lack of awareness and participation by employees. Recommendations are provided for researchers and practitioners implementing future whole-workplace walking programmes.

Project description:BackgroundThe National Diabetes Prevention Program (National DPP) is rapidly expanding in an effort to help those at high risk of type 2 diabetes prevent or delay the disease. In 2012, the Centers for Disease Control and Prevention funded six national organizations to scale and sustain multistate delivery of the National DPP lifestyle change intervention (LCI). This study aims to describe reach, adoption, and maintenance during the 4-year funding period and to assess associations between site-level factors and program effectiveness regarding participant attendance and participation duration.MethodsThe Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework was used to guide the evaluation from October 2012 to September 2016. Multilevel linear regressions were used to examine associations between participant-level demographics and site-level strategies and number of sessions attended, attendance in months 7-12, and duration of participation.ResultsThe six funded national organizations increased the number of participating sites from 68 in 2012 to 164 by 2016 across 38 states and enrolled 14,876 eligible participants. By September 2016, coverage for the National DPP LCI was secured for 42 private insurers and 7 public payers. Nearly 200 employers were recruited to offer the LCI on site to their employees. Site-level strategies significantly associated with higher overall attendance, attendance in months 7-12, and longer participation duration included using self-referral or word of mouth as a recruitment strategy, providing non-monetary incentives for participation, and using cultural adaptations to address participants' needs. Sites receiving referrals from healthcare providers or health systems also had higher attendance in months 7-12 and longer participation duration. At the participant level, better outcomes were achieved among those aged 65+ (vs. 18-44 or 45-64), those who were overweight (vs. obesity), those who were non-Hispanic white (vs. non-Hispanic black or multiracial/other races), and those eligible based on a blood test or history of gestational diabetes mellitus (vs. screening positive on a risk test).ConclusionsIn a time of rapid dissemination of the National DPP LCI the findings of this evaluation can be used to enhance program implementation and translate lessons learned to similar organizations and settings.

Project description:BackgroundA fundamental challenge of implementation is identifying contextual determinants (i.e., barriers and facilitators) and determining which implementation strategies will address them. Numerous conceptual frameworks (e.g., the Consolidated Framework for Implementation Research; CFIR) have been developed to guide the identification of contextual determinants, and compilations of implementation strategies (e.g., the Expert Recommendations for Implementing Change compilation; ERIC) have been developed which can support selection and reporting of implementation strategies. The aim of this study was to identify which ERIC implementation strategies would best address specific CFIR-based contextual barriers.MethodsImplementation researchers and practitioners were recruited to participate in an online series of tasks involving matching specific ERIC implementation strategies to specific implementation barriers. Participants were presented with brief descriptions of barriers based on CFIR construct definitions. They were asked to rank up to seven implementation strategies that would best address each barrier. Barriers were presented in a random order, and participants had the option to respond to the barrier or skip to another barrier. Participants were also asked about considerations that most influenced their choices.ResultsFour hundred thirty-five invitations were emailed and 169 (39%) individuals participated. Respondents had considerable heterogeneity in opinions regarding which ERIC strategies best addressed each CFIR barrier. Across the 39 CFIR barriers, an average of 47 different ERIC strategies (SD?=?4.8, range 35 to 55) was endorsed at least once for each, as being one of seven strategies that would best address the barrier. A tool was developed that allows users to specify high-priority CFIR-based barriers and receive a prioritized list of strategies based on endorsements provided by participants.ConclusionsThe wide heterogeneity of endorsements obtained in this study's task suggests that there are relatively few consistent relationships between CFIR-based barriers and ERIC implementation strategies. Despite this heterogeneity, a tool aggregating endorsements across multiple barriers can support taking a structured approach to consider a broad range of strategies given those barriers. This study's results point to the need for a more detailed evaluation of the underlying determinants of barriers and how these determinants are addressed by strategies as part of the implementation planning process.

Project description:Tobacco cessation after cancer diagnosis leads to better patient outcomes. However, tobacco treatment services are frequently unavailable in cancer care settings, and multilevel implementation challenges can impede uptake of new programs. The National Cancer Institute (NCI) dedicated Cancer Moonshot funding through the Cancer Center Cessation Initiative (C3I) for NCI-Designated Cancer Centers to implement or enhance the implementation of tobacco treatment services. We examined a pragmatic application of the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance) to evaluate tobacco treatment programs implemented within Cancer Centers funded through C3I. Using three C3I-funded Centers as examples, we describe how each RE-AIM construct was operationalized to evaluate the implementation of a wide range of cessation services (e.g., tobacco use screening, counseling, Quitline referral, pharmacotherapy) in this heterogeneous group of cancer care settings. We discuss the practical challenges encountered in assessing RE-AIM constructs in real world situations, including using the electronic health record (EHR) to aid in assessment. Reach and effectiveness evaluation required that Centers define the setting(s) where cessation services were implemented (to determine the "denominator"), enumerate the patient population, report current patient tobacco use, patient engagement in tobacco treatment, and 6-month cessation outcomes. To reduce site heterogeneity, increase data accuracy, and reduce burden, reach was frequently captured via standardized EHR enhancements that improved the identification of current smokers and tobacco treatment referrals. Effectiveness was determined by cessation outcomes (30-day point prevalence abstinence at 6-months post-engagement) assessed through a variety of data collection approaches. Adoption was measured by the characteristics and proportion of targeted cancer care settings and clinicians engaged in cessation service delivery. Implementation was assessed by examining the delivery of tobacco screening assessments and intervention components across sites, and provider-level implementation consistency. Maintenance assessments identified whether tobacco treatment services continued in the setting after implementation and documented the sustainability plan and organizational commitment to continued delivery. In sum, this paper demonstrates a pragmatic approach to using RE-AIM as an evaluation framework that yields relevant outcomes on common implementation metrics across widely differing tobacco treatment approaches and settings.

Project description:Background: Few studies of hospital-based implementation assess sustainability or collect formal implementation outcomes, in part because the emphasis is often on initial adoption and rapid cycles of improvement. The purpose of this process evaluation was to assess the implementation of a pharmacy-led, hospital-wide program and contribute to the literature by collecting formal implementation outcomes, including sustainability. Methods: This was a qualitative process evaluation of a program that delivers discharge medications and related education to hospitalized patients' bedside prior to discharge. Semi-structured interviews were conducted with the program's key stakeholders to assess the program's implementation barriers and facilitators as well as its potential for sustainability. An interview guide was created based on the RE-AIM constructs of Reach, Adoption, Implementation, and Maintenance. Effectiveness was not assessed due to an ongoing effectiveness evaluation by another team. Each interview was coded by two independent coders and any discrepancy was adjudicated by a third, independent coder. Results: Twelve stakeholders were approached and all agreed to be interviewed. Related to providers' decisions to adopt the program, key themes emerged around the different priorities of nurses and physicians, which has implications for how program leadership promoted the program to these different stakeholder groups. Key implementation barriers included the nature of hospital provider rotations and turnover, which led to confusion on who could use the program and to whom providers should direct program-related questions. Key implementation facilitators included the enthusiasm of program staff and identified champions on the units. Themes related to maintenance or sustainability included the need to continually generate buy-in and educate providers about the program and allowing program staff and leadership to remain nimble and adapt their operations to meet evolving needs. Conclusions: The results suggest that in an environment in which rapidly achieving improvement is often the focus more than maintaining that improvement, strategies to achieve successful implementation may not be sufficient to achieve successful sustainment. New strategies are likely needed to address the unique barriers to sustaining a program once initial adoption and implementation is complete.

Dataset Information

Assessing Digital Health Implementation for a Pediatric Chronic Pain Intervention: Comparing the RE-AIM and BIT Frameworks Against Real-World Trial Data and Recommendations for Future Studies.

Publications

Assessing Digital Health Implementation for a Pediatric Chronic Pain Intervention: Comparing the RE-AIM and BIT Frameworks Against Real-World Trial Data and Recommendations for Future Studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure