Dataset Information

Operationalizing the reach, effectiveness, adoption, implementation, maintenance (RE-AIM) framework to evaluate the collective impact of autonomous community programs that promote health and well-being.

ABSTRACT:

Background

The RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework is a useful tool for evaluating the impact of programs in community settings. RE-AIM has been applied to evaluate individual programs but seldom used to evaluate the collective impact of community-based, public health programming developed and delivered by multiple autonomous organizations. The purposes of this paper were to (a) demonstrate how RE-AIM can be operationalized and applied to evaluate the collective impact of similar autonomous programs that promote health and well-being and (b) provide preliminary data on the collective impact of Canadian spinal cord injury (SCI) peer mentorship programs on the delivery of peer mentorship services.

Methods

Criteria from all five RE-AIM dimensions were operationalized to evaluate multiple similar community-based programs. For this study, nine provincial organizations that serve people with SCI were recruited from across Canada. Organizations completed a structured self-report questionnaire and participated in a qualitative telephone interview to examine different elements of their peer mentorship program. Data were analyzed using summary statistics.

Results

Having multiple indicators to assess RE-AIM dimensions provided a broad evaluation of the impact of Canadian SCI peer mentorship programs. Peer mentorship programs reached 1.63% of the estimated Canadian SCI population. The majority (67%) of organizations tracked the effectiveness of peer mentorship through testimonials and reports. Setting-level adoption rates were high with 100% of organizations offering peer mentorship in community and hospital settings. On average, organizations allocated 10.4% of their operating budget and 9.8% of their staff to implement peer mentorship and 89% had maintained their programming for over 10 years. Full interpretation of the collective impact of peer mentorship programs was limited as complete data were only collected for 52% of survey questions.

Conclusions

The lack of available organizational data highlights a significant challenge when using RE-AIM to evaluate the collective impact of multiple programs that promote health and well-being. Although researchers are encouraged to use RE-AIM to evaluate the collective impact of programs delivered by different organizations, documenting limitations and providing recommendations should be done to further the understanding of how best to operationalize RE-AIM in these contexts.

SUBMITTER: Shaw RB

PROVIDER: S-EPMC6591988 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Introduction: The use of models and frameworks to design and evaluate strategies to improve delivery of evidence-based interventions is a foundational element of implementation science. To date, however, evaluative implementation science frameworks such as Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) have not been widely employed to examine environmental health interventions. We take advantage of a unique opportunity to utilize and iteratively adapt the RE-AIM framework to guide NIH-funded case studies of the implementation of clean cooking fuel programs in eleven low- and middle-income countries. Methods: We used existing literature and expert consultation to translate and iteratively adapt the RE-AIM framework across several stages of the NIH Clean Cooking Implementation Science case study project. Checklists and templates to guide investigators were developed at each stage. Results: The RE-AIM framework facilitated identification of important emerging issues across this set of case studies, in particular highlighting the fact that data associated with certain important outcomes related to health and welfare are chronically lacking in clean fuel programs. Monitoring of these outcomes should be prioritized in future implementation efforts. As RE-AIM was not originally designed to evaluate household energy interventions, employing the framework required adaptation. Specific adaptations include the broadening of Effectiveness to encompass indicators of success toward any stated programmatic goal, and expansion of Adoption to include household-level uptake of technology. Conclusions: The RE-AIM implementation science framework proved to be a useful organizing schema for 11 case studies of clean fuel cooking programs, in particular highlighting areas requiring emphasis in future research and evaluation efforts. The iterative approach used here to adapt an implementation science framework to a specific programmatic goal may be of value to other multi-country program efforts, such as those led by international development agencies. The checklists and templates developed for this project are publicly available for others to use and/or further modify.

Project description:Tobacco cessation after cancer diagnosis leads to better patient outcomes. However, tobacco treatment services are frequently unavailable in cancer care settings, and multilevel implementation challenges can impede uptake of new programs. The National Cancer Institute (NCI) dedicated Cancer Moonshot funding through the Cancer Center Cessation Initiative (C3I) for NCI-Designated Cancer Centers to implement or enhance the implementation of tobacco treatment services. We examined a pragmatic application of the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance) to evaluate tobacco treatment programs implemented within Cancer Centers funded through C3I. Using three C3I-funded Centers as examples, we describe how each RE-AIM construct was operationalized to evaluate the implementation of a wide range of cessation services (e.g., tobacco use screening, counseling, Quitline referral, pharmacotherapy) in this heterogeneous group of cancer care settings. We discuss the practical challenges encountered in assessing RE-AIM constructs in real world situations, including using the electronic health record (EHR) to aid in assessment. Reach and effectiveness evaluation required that Centers define the setting(s) where cessation services were implemented (to determine the "denominator"), enumerate the patient population, report current patient tobacco use, patient engagement in tobacco treatment, and 6-month cessation outcomes. To reduce site heterogeneity, increase data accuracy, and reduce burden, reach was frequently captured via standardized EHR enhancements that improved the identification of current smokers and tobacco treatment referrals. Effectiveness was determined by cessation outcomes (30-day point prevalence abstinence at 6-months post-engagement) assessed through a variety of data collection approaches. Adoption was measured by the characteristics and proportion of targeted cancer care settings and clinicians engaged in cessation service delivery. Implementation was assessed by examining the delivery of tobacco screening assessments and intervention components across sites, and provider-level implementation consistency. Maintenance assessments identified whether tobacco treatment services continued in the setting after implementation and documented the sustainability plan and organizational commitment to continued delivery. In sum, this paper demonstrates a pragmatic approach to using RE-AIM as an evaluation framework that yields relevant outcomes on common implementation metrics across widely differing tobacco treatment approaches and settings.

Project description:Background: Explored the role of public health centers in the delivery of physical activity programs to older Brazilians. Methods: Total of 114 older adults (81% women) from public health centers across the city of Florianopolis, Brazil, were randomized into three groups: behavior change group (n = 36), traditional exercise group (n = 52), and control group (n = 26). The behavioral change group included 12 weekly meetings (2 h each). The traditional exercise group offered a 12-week exercise class. Individuals in the control group participated only in measurements. Program evaluation included a mixed-methods approach following the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance). Trained interviewers conducted 12 focus groups and 32 interviews with participants in the program, professionals delivering the programs, community health workers, and local and city administrators overseeing public health centers. Participants completed health, quality of life, and fitness assessments at four time points. Results: The study reached 11.5% of the eligible population in the community. Older adults' resistance to change and limited understanding of behavior change science by public health center staff hindered program reach. Physician encouraging patient participation and personal invitations by community health workers were perceived as favorable factors. Results of program effectiveness and maintenance suggest that behavior change strategies may be better suited than traditional exercise classes for decreasing sedentary time and increasing moderate-to-vigorous physical activity, as well as improving participants' quality of life. Only 14% of public health centers in the city adopted the programs. Heavy workload of health educators delivering the programs and limited physical space for program delivery were barriers for adoption. The fidelity of program delivery was high and indicates that the programs are culturally-appropriate for the Brazilian context and feasible for implementation by local health educators. Conclusions: Our findings support the potential for dissemination of behavior change and traditional exercise programs to older adults through public health centers in Brazil. REBEC: RBR-9pkxn2 (retrospectively registered) Register April 20, 2019.

Project description:ObjectivesThe purpose of this review was to use RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) to assess the extent to which weight gain prevention studies targeting young adults reported on elements of external validity.DesignSystematic review.Eligibility criteriaArticles of interest included a lifestyle/behavioral intervention targeting weight gain prevention. Eligibility criteria included the following: study design of randomized controlled trials, quasi-randomized control trials, or natural experiments; average participant age between 18 and 35 years; study duration of at least 12 months; and published in English between January 2008 and May 2018. Studies had to report weight or body mass index as a measured outcome and were excluded if they were paired with smoking cessation programs, were conducted in specific groups (ie, pregnant women, breast cancer survivors), or were follow-ups to weight loss studies.Study selectionAfter removing duplicates, the search yielded 11426 articles. Titles and abstracts were screened by 1 reviewer; 144 articles were assessed in a full-text review by 2 reviewers. Discrepancies were resolved by consensus. Nine studies (13 articles) were included in the review.Main outcomes measuresReported elements of the RE-AIM framework.ResultsA total of 9 studies met the selection criteria. All studies lacked full reporting on external validity elements. Of the total of 60 RE-AIM reporting criteria, 8 were reported by all 9 studies, 26 criteria were reported by fewer than 4 studies, and 22 criteria were not reported by any of the studies.DiscussionThere remains inadequate reporting of elements of external validity and generalizability in weight gain prevention studies. This is a significant scientific constraint that limits the information required to disseminate and implement prevention of weight gain interventions for population impact. Standardized reporting may be needed to ensure results that demonstrate not only internal validity but also external validity and generalizability are needed to promote public health impact.

Project description:BackgroundLung ultrasound (LUS) is a clinician-performed evidence-based imaging modality that has multiple advantages in the evaluation of dyspnea caused by multiple disease processes, including COVID-19. Despite these advantages, few hospitalists have been trained to perform LUS. The aim of this study was to increase adoption and implementation of LUS during the 2020 COVID-19 pandemic by using recurrent assessments of RE-AIM outcomes to iteratively revise our implementation strategies.MethodsIn an academic hospital, we implemented guidelines for the use of LUS in patients with COVID-19 in July 2020. Using a novel "RE-AIM dashboard," we used an iterative process of evaluating the high-priority outcomes of Reach, Adoption, and Implementation at twice monthly intervals to inform revisions of our implementation strategies for LUS delivery (i.e., Iterative RE-AIM process). Using a convergent mixed methods design, we integrated quantitative RE-AIM outcomes with qualitative hospitalist interview data to understand the dynamic determinants of LUS Reach, Adoption, and Implementation.ResultsOver the 1-year study period, 453 LUSs were performed in 298 of 12,567 eligible inpatients with COVID-19 (Reach = 2%). These 453 LUS were ordered by 43 out of 86 eligible hospitalists (LUS order adoption = 50%). However, the LUSs were performed/supervised by only 8 of these 86 hospitalists, 4 of whom were required to complete LUS credentialing as members of the hospitalist procedure service (proceduralist adoption 75% vs 1.2% non-procedural hospitalists adoption). Qualitative and quantitative data obtained to evaluate this Iterative RE-AIM process led to the deployment of six sequential implementation strategies and 3 key findings including (1) there were COVID-19-specific barriers to LUS adoption, (2) hospitalists were more willing to learn to make clinical decisions using LUS images than obtain the images themselves, and (3) mandating the credentialing of a strategically selected sub-group may be a successful strategy for improving Reach.ConclusionsMandating use of a strategically selected subset of clinicians may be an effective strategy for improving Reach of LUS. Additionally, use of Iterative RE-AIM allowed for timely adjustments to implementation strategies, facilitating higher levels of LUS Adoption and Reach. Future studies should explore the replicability of these preliminary findings.

Project description:BackgroundProlonged caregiving of an older adult can cause family caregivers to be overwhelmed, potentially affecting the well-being of both the caregivers and their care-recipients. Carer Matters is a holistic hospital-to-home programme, centred on caregivers' needs as their care-recipients transit from hospital to home. The programme was piloted to support caregivers through caregivers needs assessment, tailored resources, tele-support, training courses, and community support network. This study aimed to examine the feasibility of Carer Matters in a tertiary hospital in Singapore.MethodsThis feasibility study was conducted on the pilot implementation from January to December 2021, during the Covid-19 pandemic. It adopted the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. The study highlighted quantitative data collected from key process indicators, such as number of caregivers screened, assessed on their needs and provided with assistance. Additionally, qualitative data was collected from in-depth interviews with 51 stakeholders involved in the implementation to examine their perspectives and experiences. These included family caregivers, clinician caregiver support nurses, hospital leaders and community partners.ResultsDuring the pilot, 550 caregivers were enrolled. All caregivers received educational resources when they completed the needs assessment, while 69 of them who reported high burden were given tele-support and 252 attended our caregiver training courses. Despite initial recruitment challenges and obstacles to adoption, stakeholders interviewed found Carer Matters to be effective in providing caregivers with emotional support, knowledge and skills that improved their caregiving abilities, and reduced their sense of isolation and caregiving stress. Among caregivers, the training courses were effective with majority of caregivers agreeing that the courses addressed their needs (99%) and improved their knowledge of the relevant disease conditions (97%). Programme maintenance considered among stakeholders included strategies such as multipronged approach in recruiting caregivers and inviting caregiver advocates to share their experiences.ConclusionThis feasibility study highlights that Carer Matters is a valuable component to the ecosystem of support for family caregivers and their care recipients. Carer Matters extends the current patient-centric care model to a more holistic post-discharge continuity of care for both caregivers and their care-recipients, improving and maintaining their overall well-being to better allow transition from hospital-to home.Trial registrationFeasibility Study of Project Carer Matters for Family Caregivers of Persons with Dementia (NCT number: NCT05205135 ).

Project description:Background: The RE-AIM framework has been widely used in health research but it is unclear the extent to which this framework is also used for planning and evaluating health-related programs in clinical and community settings. Our objective was to evaluate how RE-AIM is used in the "real-world" and identify opportunities for improving use outside of research contexts. Methods: We used purposive and snowball sampling to identify clinical and community health programs that used RE-AIM for planning and/or evaluation. Recruitment methods included surveys with email follow-up to funders, implementers, and RE-AIM working group members. We identified 17 programs and conducted structured in-depth interviews with key informants (n = 18). Across RE-AIM dimensions, respondents described motivations, uses, and measures; rated understandability and usefulness; discussed benefits and challenges, strategies to overcome challenges, and resources used. We used descriptive statistics for quantitative ratings, and content analysis for qualitative data. Results: Program content areas included chronic disease management and prevention, healthy aging, mental health, or multiple, often behavioral health-related topics. During planning, most programs considered reach (n = 9), adoption (n = 11), and implementation (n = 12) while effectiveness (n = 7) and maintenance (n = 6) were considered less frequently. In contrast, most programs evaluated all RE-AIM dimensions, ranging from 13 programs assessing maintenance to 15 programs assessing implementation and effectiveness. On five-point scales, all RE-AIM dimensions were rated as easy to understand (Overall M = 4.7 ± 0.5), but obtaining data was rated as somewhat challenging (Overall M = 3.4 ± 0.9). Implementation was the most frequently used dimension to inform program design (M = 4.7 ± 0.6) relative to the other dimensions (3.0-3.9). All dimensions were considered similarly important for decision-making (average M = 4.1 ± 1.4), with the exception of maintenance (M = 3.4 ± 1.7). Qualitative corresponded to the quantitative findings in that RE-AIM was reported to be a practical, easy to understand, and well-established implementation science framework. Challenges included understanding differences among RE-AIM dimensions and data acquisition. Valuable resources included the RE-AIM website and collaborating with an expert. Discussion: RE-AIM is an efficient framework for planning and evaluation of clinical and community-based projects. It provides structure to systematically evaluate health program impact. Programs found planning for and assessing maintenance difficult, providing opportunities for further refinement.

Project description:BackgroundCognitive impairment (CI) is a risk factor for falls due to environmental or living settings, balance, gait and vision impairments, as well as medications. While previous systematic reviews have focused on the effectiveness of fall prevention programs in adults with cognitive impairment, very limited information is available on their implementation. This review examines what aspects of fall prevention interventions for community-dwelling adults with CI have been reported using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to support successful implementation.MethodsWe examined the included studies from our systematic review, which searched 7 databases for primary and secondary fall prevention interventions involving community-dwelling adults ≥50 years with mild to moderate CI. Reviewers screened citations and extracted data for study characteristics and the 5 dimensions (62 criteria) of the RE-AIM framework.ResultsTwelve randomized or clinical controlled trials (RCTs/CCTs) consisting of 8 exercise interventions, 3 multifactorial interventions, and 1 medication treatment were included in the review. Only 4 of 62 criteria were reported by all 12 included studies and 29 criteria were not reported by any of the studies. Five of the included studies reported on 20 or more of the 62 possible RE-AIM criteria and 3 of these studies self-identified as "feasibility" studies. While Reach was the best-reported construct by the included studies, followed by Effectiveness and Implementation, the criteria within the Adoption and Maintenance constructs were rarely mentioned by these studies. In general, there was also wide variation in how each of the criteria were reported on by study authors.ConclusionBased on the reporting of RE-AIM components in this review, we are unable to make connections to successful intervention components and thus practice-based recommendations for fall prevention in those with CI. The lack of detail regarding implementation approaches greatly limits the interpretation and comparisons across studies to fully inform future research efforts.