Development and Evaluation of a Component Level Implementation Fidelity Rating System for the GenerationPMTO Intervention.
Ontology highlight
ABSTRACT:
SUBMITTER: Holtrop K
PROVIDER: S-EPMC8032561 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Background Implementation of clinical medication reviews in daily practice is scarcely evaluated. The Opti-Med intervention applied a structured approach with external expert teams (pharmacist and physician) to conduct medication reviews. The intervention was effective with respect to resolving drug related problems, but did not improve quality of life. Objective The objective of this process evaluation was to gain more insight into the implementation fidelity of the intervention. Setting Process evaluation alongside a cluster randomized trial in 22 general practices and 518 patients of 65 years and over. Method A mixed methods design using quantitative and qualitative data and the conceptual framework for implementation fidelity was used. Implementation fidelity is defined as the degree to which the various components of an intervention are delivered as intended. Main outcome measure Implementation fidelity for key components of the Opti-Med intervention. Results Patient selection and preparation of the medication analyses were carried out as planned, although mostly by the Opti-Med researchers instead of practice nurses. Medication analyses by expert teams were performed as planned, as well as patient consultations and patient involvement. 48% of the proposed changes in the medication regime were implemented. Cooperation between expert teams members and the use of an online decision-support medication evaluation facilitated implementation. Barriers for implementation were time constraints in daily practice, software difficulties with patient selection and incompleteness of medical files. The degree of embedding of the intervention was found to influence implementation fidelity. The total time investment for healthcare professionals was 94 min per patient. Conclusion Overall, the implementation fidelity was moderate to high for all key components of the Opti-Med intervention. The absence of its effectiveness with respect to quality of life could not be explained by insufficient implementation fidelity.
| S-EPMC5984963 | biostudies-literature
Project description:Background"We Act" is a health-promoting school intervention comprising an educational, a parental and a school component. The intervention was implemented in 4 Danish public schools with 4 control schools. The objectives were to improve pupils' dietary habits, physical activity, well-being and social capital using the Investigation, Vision, Action & Change (IVAC) health educational methodology. The target group was pupils in grades 5-6. The purpose of this study was to evaluate implementation fidelity and interacting context factors in the intervention schools.MethodsThe Medical Research Council's new guidance for process evaluation was used as a framework. Data were collected concurrently and evenly at the 4 intervention schools through field visits (n?=?43?days), questionnaires (n?=?17 teachers, 52 parents), and interviews (n?=?9 teachers, 4 principals, 52 pupils). The data were analysed separately and via triangulation.ResultsA total of 289 pupils participated, and 22 teachers delivered the educational component in 12 classes. In all schools, the implementation fidelity to the educational methodology was high for the Investigation and Vision phases as the teachers delivered the proposed lessons and activities. However, the implementation fidelity to the Action & Change phase was low, and little change occurred in the schools. The pupils' presentation of their visions did not work as intended as an impact mechanism to prompt actions. The implementation of the parental and the school components was weak. The main context factors influencing implementation fidelity were a poor fit into the school-year plan and weak management support.ConclusionsAlthough 'We Act' was designed to comply with evidence- and theory-based requirements, IVAC and the health-promoting school approach did not result in change. The time dedicated to schools' preparation and competence development may have been too low. This must be considered in future process evaluation research on health-promoting schools and by school health promotion administrators when planning future school interventions.Trial registrationISRCTN85203017.
| S-EPMC6307196 | biostudies-literature
Project description:BACKGROUND:Despite improvements in diagnostics and therapy, the majority of lung tumours are diagnosed at advanced stage IV with a poor prognosis. Due to the nature of an incurable disease, patients need to engage in shared decision making on advance care planning. To implement this in clinical practice, effective communication between patients, caregivers and healthcare professionals is essential. The Heidelberg Milestones Communication Approach (MCA) is delivered by a specifically trained interprofessional tandem and consists of four milestone conversations (MCs) at pivotal times in the disease trajectory. MC 1 (Diagnosis): i.e. prognosis; MC 2 (Stable disease): i.e. prognostic awareness; MC 3 (Progression): i.e. reassessment; MC 4 (Best supportive care): i.e. end of treatment. In between MCs, follow-up calls are carried out to sustain communication. This study aimed to assess to what extent the MCA was implemented as planned and consolidated in specialized oncology practice. METHODS:A prospective observational process evaluation study was conducted, which focused on the implementation fidelity of the MCA. All MCs during two assessment periods were included. We analysed all written records of the conversations, which are part of the routine documentation during MCs and follow-up calls. Adherence to key aspects of the manual was documented on structured checklists at the beginning of the implementation of the MCA and after 6?months. The analysis was descriptive. Differences between the two assessment periods are analysed with chi-square tests. RESULTS:A total of 133 MCs and 54 follow-up-calls (t1) and of 172 MCs and 92 follow-up calls (t2) were analysed. MC 2 were the most frequently completed conversations (n =?51 and n =?47). Advance care planning was discussed in 26 and 13% of MC 2 in the respective assessment periods; in 31 and 47% of MC 2, prognostic awareness was recorded. The most frequently documented topic in the follow-up calls was the physical condition in patients (82 and 83%). CONCLUSION:The implementation of a trajectory-specific communication concept was largely successful. Additional studies are needed to understand how fidelity could be further improved. TRIAL REGISTRATION:DRKS00013469 / Date of registration: 22.12.2017.
| S-EPMC7029500 | biostudies-literature
Project description:BackgroundIn pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music & MemorySM (M&M), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs).MethodsIntervention NHs (N = 27) implemented M&M using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core M&M components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to M&M at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity).ResultsThe 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was τb = 0.11 (p = 0.59) and with Duration was τb = - 0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer's unit, suggesting face validity.ConclusionsGuided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach.Trial registrationClinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844 .
| S-EPMC8761354 | biostudies-literature
Project description:Burkina Faso implemented the seasonal malaria chemoprevention (SMC) in 2014 in seven pilot health districts, following the new recommendation by the WHO in 2012 for the prevention of the disease in children under five years old, for areas of highly seasonal malaria transmission.The objective of this study was to assess the implementation fidelity of the seasonal malaria chemoprevention strategy in one of the districts, Kaya Health District.We conducted a case study, with a quantitative and qualitative mixed methods. Data were collected after two campaigns of implementation of the intervention, in 2014 and 2015, through a review of specific documents of SMC intervention, and individual interview with key informants (n = 21) involved at various levels in the implementation of the strategy and a household survey with the parents (n = 284) of eligible children for the SMC strategy in 2015 in the Kaya health district. The analysis framework focused on the fidelity of the intervention's content, its coverage, and its schedule, as well as the potential moderating factors, using the model proposed by Hasson, originally from Carroll.All components of the intervention were implemented. Villages and sectors were covered at 100%. In terms of intervention doses received, less than one-third of eligible children (32.3%) received the recommended four doses in 2015. Implementation of the strategy faced some difficulties due to insufficient training of community distributors, inadequate supply of inputs and insufficient financial resources for remuneration, advocacy and supervision, but also because of the contextual constraints due to the rainy season. Moreover, an interaction between the different moderating factors, influencing the degree of implementation of the strategy was noted.Taking into account the moderating factors of the implementation is necessary for achieving the highest possible degree of implementation fidelity and then, reach the expected beneficial effects.
| S-EPMC5706718 | biostudies-literature
Project description:BackgroundFidelity of implementation (FOI) reflects whether an intervention was implemented in clinical practice according to the originally developed manual and is a key aspect in understanding intervention effectiveness. To illustrate this process of developing a fidelity measure, this study uses the Mini-AFTERc, a brief psychological intervention aimed at managing breast cancer patients' fear of cancer recurrence, as an example.ObjectivesTo illustrate the development of an FOI measure through (1) applying this process to the Mini-AFTERc intervention, by including the design of a scoring system and rating criteria; (2) content validating the FOI measure using thematic framework analysis as a qualitative approach; (3) testing consistency of the FOI measure using interrater reliability.MethodsThe FOI measure was developed, its scoring system modified and the rating criteria defined. Thematic framework analysis was conducted to content validate the FOI measure using nine intervention discussions between four specialist cancer nurses and four breast cancer patients, and one simulated breast cancer patient. Intraclass-correlation was conducted to assess interrater reliability.ResultsThe qualitative findings suggested that the Mini-AFTERc FOI measure has content validity as it was able to measure all five components of the Mini-AFTERc intervention. The interrater reliability suggested a moderate to excellent degree of reliability among three raters, r ICC = 0.84, 95% CI [0.51, 0.96].ConclusionThe study has illustrated the steps that an FOI measure can be developed through a systematic approach applied to the Mini-AFTERc intervention. The FOI measure was found to have content validity and was consistently applied, independently, by three researchers familiar with the Mini-AFTERc intervention. Future studies should determine whether similar levels of interrater reliability can be obtained by distributing written and/or video instructions to researchers who are unfamiliar with the FOI measure, using a larger sample. Employing developed and validated FOI measures such as the one presented for the Mini-AFTERc would facilitate implementation of interventions in the FCR field in clinical practice as intended.Clinical trial registrationwww.ClinicalTrials.gov, identifier: NCT03763825.
| S-EPMC7723987 | biostudies-literature
Project description:BackgroundAlong with the increasing prevalence of chronic illness has been an increase in interventions, such as nurse case management programs, to improve outcomes for patients with chronic illness. Evidence supports the effectiveness of such interventions in reducing patient morbidity, mortality, and resource utilization, but other studies have produced equivocal results. Often, little is known about how implementation of an intervention actually occurs in clinical practice. While studies often assume that interventions are used in clinical practice exactly as originally designed, this may not be the case. Thus, fidelity of an intervention's implementation reflects how an intervention is, or is not, used in clinical practice and is an important factor in understanding intervention effectiveness and in replicating the intervention in dissemination efforts. The purpose of this paper is to contribute to the understanding of implementation science by (a) proposing a methodology for measuring fidelity of implementation (FOI) and (b) testing the measure by examining the association between FOI and intervention effectiveness.MethodsWe define and measure FOI based on organizational members' level of commitment to using the distinct components that make up an intervention as they were designed. Semistructured interviews were conducted among 18 organizational members in four medical centers, and the interviews were analyzed qualitatively to assess three dimensions of commitment to use--satisfaction, consistency, and quality--and to develop an overall rating of FOI. Mixed methods were used to explore the association between FOI and intervention effectiveness (inpatient resource utilization and mortality).ResultsPredictive validity of the FOI measure was supported based on the statistical significance of FOI as a predictor of intervention effectiveness. The strongest relationship between FOI and intervention effectiveness was found when an alternative measure of FOI was utilized based on individual intervention components that had the greatest variation across medical centers.ConclusionsIn addition to contextual factors, implementation research needs to consider FOI as an important factor in influencing intervention effectiveness. Our proposed methodology offers a systematic means for understanding organizational members' use of distinct intervention components, assessing the reasons for variation in use across components and organizations, and evaluating the impact of FOI on intervention effectiveness.
| S-EPMC3161382 | biostudies-literature
Project description:BackgroundBlunt chest wall injuries can lead to complications, especially without early intervention. A blunt Chest Injury Protocol (ChIP) was developed to help improve the consistency of evidence-based care following admission to the emergency department. Implementation strategy fidelity is the extent to which the strategies of implementation are delivered in line with the intended plan. The aim of this study was to assess fidelity to the strategies of the implementation plan developed for ChIP.MethodsA retrospective evaluation of strategies used for implementation was performed, specifically the behaviour change techniques (BCTs). BCTs were used as part of an implementation plan derived based on the Behaviour Change Wheel from results from a staff survey at two hospitals. Levels of implementation or adaptation for BCTs were scored by implementers as follows: 'Were the behaviour change interventions implemented?' (0 = 'not implemented', 1 = partially implemented, and 2 = fully implemented); 'Were adaptations made to the implementation plan?', scored 1 (many changes from plan) to 4 (just as planned). Free text explanation to their responses was also collected with supporting evidence and documentation (such as emails, implementation checklists, audit reports, and incident reports).ResultsThere was high overall fidelity of 97.6% for BCTs partially or fully implemented. More than three quarters (32/42, 76.2%) of the BCTs were fully implemented with an additional 9/42 (21.4%) partially implemented. BCTs that were not fully implemented were social support, feedback on behaviour, feedback on outcomes of behaviour, adding objects to the environment, and restructuring the environment. The modes of delivery with poorer implementation or increased adaptations were clinical champions and audit/feedback.ConclusionsThis study describes the evaluation of implementation strategy fidelity in the acute care context. The systematic use and application of the behaviour change wheel was used to develop an implementation plan and was associated with high implementation strategy fidelity. A fidelity checklist developed during the implementation process may help implementers assess fidelity.Trial registrationTrial registered on ANZCTR. Registration number ACTRN12618001548224 , date approved 17/09/2018.
| S-EPMC8353870 | biostudies-literature
Project description:BackgroundStand and Move at Work was a 12-month, multicomponent, peer-led (intervention delivery personnel) worksite intervention to reduce sedentary time. Although successful, the magnitude of reduced sedentary time varied by intervention worksite. The purpose of this study was to use a qualitative comparative analysis approach to examine potential explanatory factors that could distinguish higher from lower performing worksites based on reduced sedentary time.MethodsWe assessed 12-month changes in employee sedentary time objectively using accelerometers at 12 worksites. We ranked worksites based on the magnitude of change in sedentary time and categorized sites as higher vs. lower performing. Guided by the integrated-Promoting Action on Research Implementation in Health Services framework, we created an indicator of intervention fidelity related to adherence to the protocol and competence of intervention delivery personnel (i.e., implementer). We then gathered information from employee interviews and surveys as well as delivery personnel surveys. These data were aggregated, entered into a truth table (i.e., a table containing implementation construct presence or absence), and used to examine differences between higher and lower performing worksites.ResultsThere were substantive differences in the magnitude of change in sedentary time between higher (-75.2 min/8 h workday, CI95: -93.7, -56.7) and lower (-30.3 min/8 h workday, CI95: -38.3, -22.7) performing worksites. Conditions that were present in all higher performing sites included implementation of indoor/outdoor walking route accessibility, completion of delivery personnel surveys, and worksite culture supporting breaks (i.e., adherence to protocol). A similar pattern was found for implementer willingness to continue role and employees using face-to-face interaction/stair strategies (i.e., delivery personnel competence). However, each of these factors were also present in some of the lower performing sites suggesting we were unable to identify sufficient conditions to predict program success.ConclusionsHigher intervention adherence and implementer competence is necessary for greater program success. These findings illustrate the need for future research to identify what factors may influence intervention fidelity, and in turn, effectiveness.Trial registrationClinicalTrials.gov Identifier: NCT02566317 . Registered 2 October 2015, first participant enrolled 11 January 2016.
| S-EPMC9158295 | biostudies-literature
Project description:BackgroundDiabetes is becoming a very important health issue in rapidly developing nations and there is an urgent need to improve overall diabetes self-management education in these countries. Although e-health is an emerging theme, only a few successful web-based studies on diabetes self-management have been reported.PurposeWe describe the development, implementation, and process evaluation of an Internet-delivered dietary intervention program (myDIDeA) for diabetic patients in a developing country.MethodSpecific dietary components to be included in the intervention module were first identified through a comprehensive review of literature and guidelines. The lesson plans and the study website were then developed based on the evidence, Transtheoretical Model's Stages of Change and user-centered design approach. Finally, the effectiveness of the website was tested through a randomized-controlled trial to promote dietary change in patients with type 2 diabetes. The participants in the intervention group (n = 66) were given access to myDIDeA for 6 months. Process evaluation in form of intervention adherence and program reception were conducted at post intervention.ResultsThe response rate for the process evaluation was 89%. On average, each participant logged in at least once for each lesson plan and spent almost 12 min on the site. The participants' content satisfaction, acceptability, and usability scores were satisfactory. The primary outcome of the trial, Dietary Knowledge, Attitude, and Behavior score was strongly correlated with content satisfaction (r = 0.826, p < 0.001), acceptability (r = 0.793, p < 0.001) and usability of the website (r = 0.724, p < 0.001), and moderately correlated with frequency of log-in (r = 0.501, p < 0.05) and duration spent in the website (r = 0.399, p < 0.05).ConclusionThe process evaluation of myDIDeA demonstrates its feasibility, and future studies should identify the possibility of extending the use of Internet-based intervention programs to other health behaviors and issues related to self-management of chronic conditions. In addition, interactivity, peer support via social media, and other means to stimulate the interest of participants can be explored.
| S-EPMC4559524 | biostudies-literature