Dataset Information

Fidelity of implementation: development and testing of a measure.

ABSTRACT:

Background

Along with the increasing prevalence of chronic illness has been an increase in interventions, such as nurse case management programs, to improve outcomes for patients with chronic illness. Evidence supports the effectiveness of such interventions in reducing patient morbidity, mortality, and resource utilization, but other studies have produced equivocal results. Often, little is known about how implementation of an intervention actually occurs in clinical practice. While studies often assume that interventions are used in clinical practice exactly as originally designed, this may not be the case. Thus, fidelity of an intervention's implementation reflects how an intervention is, or is not, used in clinical practice and is an important factor in understanding intervention effectiveness and in replicating the intervention in dissemination efforts. The purpose of this paper is to contribute to the understanding of implementation science by (a) proposing a methodology for measuring fidelity of implementation (FOI) and (b) testing the measure by examining the association between FOI and intervention effectiveness.

Methods

We define and measure FOI based on organizational members' level of commitment to using the distinct components that make up an intervention as they were designed. Semistructured interviews were conducted among 18 organizational members in four medical centers, and the interviews were analyzed qualitatively to assess three dimensions of commitment to use--satisfaction, consistency, and quality--and to develop an overall rating of FOI. Mixed methods were used to explore the association between FOI and intervention effectiveness (inpatient resource utilization and mortality).

Results

Predictive validity of the FOI measure was supported based on the statistical significance of FOI as a predictor of intervention effectiveness. The strongest relationship between FOI and intervention effectiveness was found when an alternative measure of FOI was utilized based on individual intervention components that had the greatest variation across medical centers.

Conclusions

In addition to contextual factors, implementation research needs to consider FOI as an important factor in influencing intervention effectiveness. Our proposed methodology offers a systematic means for understanding organizational members' use of distinct intervention components, assessing the reasons for variation in use across components and organizations, and evaluating the impact of FOI on intervention effectiveness.

SUBMITTER: Keith RE

PROVIDER: S-EPMC3161382 | biostudies-literature | 2010 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Fidelity of implementation: development and testing of a measure.

Keith Rosalind E RE Hopp Faith P FP Subramanian Usha U Wiitala Wyndy W Lowery Julie C JC

Implementation science : IS 20101230

<h4>Background</h4>Along with the increasing prevalence of chronic illness has been an increase in interventions, such as nurse case management programs, to improve outcomes for patients with chronic illness. Evidence supports the effectiveness of such interventions in reducing patient morbidity, mortality, and resource utilization, but other studies have produced equivocal results. Often, little is known about how implementation of an intervention actually occurs in clinical practice. While stu ...[more]

PMID: 21192817

Similar Datasets

Project description:BackgroundFidelity of implementation (FOI) reflects whether an intervention was implemented in clinical practice according to the originally developed manual and is a key aspect in understanding intervention effectiveness. To illustrate this process of developing a fidelity measure, this study uses the Mini-AFTERc, a brief psychological intervention aimed at managing breast cancer patients' fear of cancer recurrence, as an example.ObjectivesTo illustrate the development of an FOI measure through (1) applying this process to the Mini-AFTERc intervention, by including the design of a scoring system and rating criteria; (2) content validating the FOI measure using thematic framework analysis as a qualitative approach; (3) testing consistency of the FOI measure using interrater reliability.MethodsThe FOI measure was developed, its scoring system modified and the rating criteria defined. Thematic framework analysis was conducted to content validate the FOI measure using nine intervention discussions between four specialist cancer nurses and four breast cancer patients, and one simulated breast cancer patient. Intraclass-correlation was conducted to assess interrater reliability.ResultsThe qualitative findings suggested that the Mini-AFTERc FOI measure has content validity as it was able to measure all five components of the Mini-AFTERc intervention. The interrater reliability suggested a moderate to excellent degree of reliability among three raters, r ICC = 0.84, 95% CI [0.51, 0.96].ConclusionThe study has illustrated the steps that an FOI measure can be developed through a systematic approach applied to the Mini-AFTERc intervention. The FOI measure was found to have content validity and was consistently applied, independently, by three researchers familiar with the Mini-AFTERc intervention. Future studies should determine whether similar levels of interrater reliability can be obtained by distributing written and/or video instructions to researchers who are unfamiliar with the FOI measure, using a larger sample. Employing developed and validated FOI measures such as the one presented for the Mini-AFTERc would facilitate implementation of interventions in the FCR field in clinical practice as intended.Clinical trial registrationwww.ClinicalTrials.gov, identifier: NCT03763825.

Project description:BackgroundIn healthcare and allied healthcare settings, leadership that supports effective implementation of evidenced-based practices (EBPs) is a critical concern. However, there are no empirically validated measures to assess implementation leadership. This paper describes the development, factor structure, and initial reliability and convergent and discriminant validity of a very brief measure of implementation leadership: the Implementation Leadership Scale (ILS).MethodsParticipants were 459 mental health clinicians working in 93 different outpatient mental health programs in Southern California, USA. Initial item development was supported as part of a two United States National Institutes of Health (NIH) studies focused on developing implementation leadership training and implementation measure development. Clinician work group/team-level data were randomly assigned to be utilized for an exploratory factor analysis (n?=?229; k?=?46 teams) or for a confirmatory factor analysis (n?=?230; k?=?47 teams). The confirmatory factor analysis controlled for the multilevel, nested data structure. Reliability and validity analyses were then conducted with the full sample.ResultsThe exploratory factor analysis resulted in a 12-item scale with four subscales representing proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership. Confirmatory factor analysis supported an a priori higher order factor structure with subscales contributing to a single higher order implementation leadership factor. The scale demonstrated excellent internal consistency reliability as well as convergent and discriminant validity.ConclusionsThe ILS is a brief and efficient measure of unit level leadership for EBP implementation. The availability of the ILS will allow researchers to assess strategic leadership for implementation in order to advance understanding of leadership as a predictor of organizational context for implementation. The ILS also holds promise as a tool for leader and organizational development to improve EBP implementation.

Project description:BackgroundHypertension (HTN) is the single leading risk factor for human mortality worldwide, and more prevalent in sub-Saharan Africa than any other region [1]-although resources for HTN screening, treatment, and control are few. Most regional pilot studies to leverage HIV programs for HTN control have achieved blood pressure control in half of participants or fewer [2,3,4]. But this control gap may be due to inconsistent delivery of services, rather than ineffective underlying interventions.MethodsWe sought to evaluate the consistency of HTN program delivery within the SEARCH study (NCT01864603) among 95,000 adults in 32 rural communities in Uganda and Kenya from 2013-2016. To achieve this objective, we designed and performed a fidelity evaluation of the step-by-step process (cascade) of HTN care within SEARCH, calculating rates of HTN screening, linkage to care, and follow-up care. We evaluated SEARCH's assessment of each participant's HTN status against measured blood pressure and HTN history.FindingsSEARCH completed blood pressure screens on 91% of participants. SEARCH HTN screening was 91% sensitive and over 99% specific for HTN relative to measured blood pressure and patient history. 92% of participants screened HTN+ received clinic appointments, and 42% of persons with HTN linked to subsequent care. At follow-up, 82% of SEARCH clinic participants received blood pressure checks; 75% received medication appropriate for their blood pressure; 66% remained in care; and 46% had normal blood pressure at their most recent visit.ConclusionThe SEARCH study's consistency in delivering screening and treatment services for HTN was generally high, but SEARCH could improve effectiveness in linking patients to care and achieving HTN control. Its model for implementing population-scale HTN testing and care through an existing HIV test-and-treat program-and protocol for evaluating the intervention's stepwise fidelity and care outcomes-may be adapted, strengthened, and scaled up for use across multiple resource-limited settings.

Project description:BackgroundDeveloping pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery.ApproachWe describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians (n = 6) and consultation with CBT experts (n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising.ConclusionOur community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems.Plain language summaryDeveloping brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such "pragmatic" measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts.

Project description:BackgroundThere exists low uptake of Human Immunodeficiency Virus (HIV) testing among Tuberculosis (TB) patients through Provider-Initiated HIV Testing and Counseling (PITC) under the national TB control program in Nepal. The degree and quality of program delivery were explored through determining whether the PITC program is currently implemented as intended. This study aimed to assess three major components of the program's implementation fidelity: adherence to PITC service, exposure, and quality of program delivery in order to optimize and standardize PITC implementation by exploring its barriers and enablers.MethodsThis research used a sequential explanatory mixed method design. Retrospective cross-sectional study of TB patients enrolled in five TB treatment centers of the Kathmandu district from July 1, 2016, to June 30, 2017 was done to assess PITC adherence to Direct Observed Treatment-Short Course (DOTS) protocols. The centers' TB-DOTS readiness was assessed using the WHO Service Availability and Readiness Assessment checklist. A qualitative study was conducted to explore the barriers and enablers of PITC service implementation.ResultsFrom a total of 643 TB patients registered, 591 (92.1%) patients were offered HIV test counseling. Amongst those, 571 (96.6%) accepted and 523 (91.5%) were tested. Service providers' HIV knowledge was found to be good although only 2/5 (40%) had participated in PITC training. The key barriers experienced by service providers were: patients feeling offended, stigmatization and lack of human resources in DOTS centers. The main enablers for PITC were national TB program commitment, health workers' motivation, collaboration between stakeholders and external development partners' promotion of program implementation.ConclusionIn the selected study sites, PITC services are well integrated into the routine TB control program with a high uptake of HIV testing among registered TB patients. This achievement should be sustained by addressing the identified barriers mainly in the quality of the PITC program delivery.

Project description:Patient empowerment is viewed by policy makers and health care practitioners as a mechanism to help patients with long-term conditions better manage their health and achieve better outcomes. However, assessing the role of empowerment is dependent on effective measures of empowerment. Although many measures of empowerment exist, no measure has been developed specifically for patients with long-term conditions in the primary care setting. This study presents preliminary data on the development and validation of such a measure.We conducted two empirical studies. Study one was an interview study to understand empowerment from the perspective of patients living with long-term conditions. Qualitative analysis identified dimensions of empowerment, and the qualitative data were used to generate items relating to these dimensions. Study two was a cross-sectional postal study involving patients with different types of long-term conditions recruited from general practices. The survey was conducted to test and validate our new measure of empowerment. Factor analysis and regression were performed to test scale structure, internal consistency and construct validity.Sixteen predominately elderly patients with different types of long-term conditions described empowerment in terms of 5 dimensions (identity, knowledge and understanding, personal control, personal decision-making, and enabling other patients). One hundred and ninety seven survey responses were received from mainly older white females, with relatively low levels of formal education, with the majority retired from paid work. Almost half of the sample reported cardiovascular, joint or diabetes long-term conditions. Factor analysis identified a three factor solution (positive attitude and sense of control, knowledge and confidence in decision making and enabling others), although the structure lacked clarity. A total empowerment score across all items showed acceptable levels of internal consistency and relationships with other measures were generally supportive of its construct validity.Initial analyses suggest that the new empowerment measure meets basic psychometric criteria. Reasons concerning the failure to confirm the hypothesized factor structure are discussed alongside further developments of the scale.

Dataset Information

Fidelity of implementation: development and testing of a measure.

Background

Methods

Results

Conclusions

Publications

Fidelity of implementation: development and testing of a measure.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets