Dataset Information

Pragmatic applications of implementation science frameworks to regulatory science: an assessment of FDA Risk Evaluation and Mitigation Strategies (REMS) (2014-2018).

ABSTRACT:

Background

A Risk Evaluation and Mitigation Strategy (REMS) is a drug safety program for certain medications with serious safety concerns required by the U.S. Food and Drug Administration (FDA) of manufacturers to implement to help ensure the benefits of the medication outweigh its risks. FDA is encouraging "the research community to develop novel methods for assessing REMS," conveying the unmet need for a standardized evaluation method of these regulatory-mandated healthcare programs. The objective of this research is to evaluate FDA REMS assessment plans using established implementation science frameworks and identify opportunities for strengthening REMS evaluation.

Methods

A content analysis was conducted of publicly available assessment plans for all REMS programs (N = 23) approved 1/1/2014-12/31/2018 for new drug applications (NDAs) and biologics license applications (BLAs) requiring FDA-mandated Elements to Assure Safe Use (ETASU). Blinded reviewers critically appraised REMS assessment measures (n = 674) using three established implementation science frameworks: RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance); PRECEDE-PROCEED (Predisposing, Reinforcing, and Enabling Constructs in Educational/Environmental Diagnosis and Evaluation - Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development); and CFIR (Consolidated Framework for Implementation Research). Framework constructs were mapped to REMS Assessment categories as defined by FDA Guidance for Industry to evaluate congruence.

Results

REMS assessment measures demonstrated strong congruence (> 90% mapping rate) with the evaluative constructs of RE-AIM, PRECEDE-PROCEED, and CFIR. Application of the frameworks revealed that REMS assessment measures heavily emphasize implementation and operations, focus less on health outcomes, and do not evaluate program context and design assumptions.

Conclusions

Implementation science frameworks have utility for evaluating FDA-mandated drug safety programs including the selection of primary measures to determine whether REMS goals are being met and of secondary measures to evaluate contextual factors affecting REMS effectiveness in varying organizational settings.

SUBMITTER: Huynh L

PROVIDER: S-EPMC8348874 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Malnutrition in all its forms has risen on global and national agendas in recent years because of the recognition of its magnitude and its consequences for a wide range of human, social, and economic outcomes. Although the WHO, national governments, and other organizations have endorsed targets and identified appropriate policies, programs, and interventions, a major challenge lies in implementing these with the scale and quality needed to achieve population impact. This paper presents an approach to implementation science in nutrition (ISN) that builds upon concepts developed in other policy domains and addresses critical gaps in linking knowledge to effective action. ISN is defined here as an interdisciplinary body of theory, knowledge, frameworks, tools, and approaches whose purpose is to strengthen implementation quality and impact. It includes a wide range of methods and approaches to identify and address implementation bottlenecks; means to identify, evaluate, and scale up implementation innovations; and strategies to enhance the utilization of existing knowledge, tools, and frameworks based on the evolving science of implementation. The ISN framework recognizes that quality implementation requires alignment across 5 domains: the intervention, policy, or innovation being implemented; the implementing organization(s); the enabling environment of policies and stakeholders; the individuals, households, and communities of interest; and the strategies and decision processes used at various stages of the implementation process. The success of aligning these domains through implementation research requires a culture of inquiry, evaluation, learning, and response among program implementers; an action-oriented mission among the research partners; continuity of funding for implementation research; and resolving inherent tensions between program implementation and research. The Society for Implementation Science in Nutrition is a recently established membership society to advance the science and practice of nutrition implementation at various scales and in varied contexts.

Project description:Background:Implementation science has been growing as discipline in the past decades, producing an increasing number of models in the area. On the other hand, most frameworks are intended to guide the implementation of programs, focusing on identifying elements and stages that increase their success. This article aims to structure this discussion, proposing a simplified tool that synthesizes common elements of other frameworks, and highlight the usefulness to use implementation science not only in identifying successful implementation strategies but as a tool to assess gaps in global health initiatives. Methods:The study was carried out through a combined methodology that included an initial search of implementation science frameworks, experts' opinions, and the use of references in frameworks to elaborate a list of articles to be reviewed. A total of 52 articles were analyzed, identifying their definitions of implementation science and the elements of different frameworks. Results:The analysis of articles allowed identifying the main goals and definitions of implementation science. In a second stage, frameworks were classified into "time-based", "component-based" and "mixed", and common elements of each type of model were used to propose a synthetic framework with six elements: Diagnosis, Intervention provider/ system, Intervention, Recipient, Environment, and Evaluation. Finally, this simplified framework was used to identify gaps in global health was using The Lancet Global Health Series. Potential areas of intervention arise for five different global health issues: malaria, non-communicable diseases, maternal and child health, HIV/AIDS, and tuberculosis. Prioritization strategies differ for the different health issues, and the proposed framework can help identify and classify all these different proposals. Conclusions:There is a huge variety of definitions and models in implementation science. The analysis showed the usefulness of applying an implementation science approach to identify and prioritize gaps in implementation strategies in global health.

Project description:BackgroundThe relevance of context in implementation science is reflected in the numerous theories, frameworks, models and taxonomies that have been proposed to analyse determinants of implementation (in this paper referred to as determinant frameworks). This scoping review aimed to investigate and map how determinant frameworks used in implementation science were developed, what terms are used for contextual determinants for implementation, how the context is conceptualized, and which context dimensions that can be discerned.MethodsA scoping review was conducted. MEDLINE and EMBASE were searched from inception to October 2017, and supplemented with implementation science text books and known published overviews. Publications in English that described a determinant framework (theory, model, taxonomy or checklist), of which context was one determinant, were eligible. Screening and inclusion were done in duplicate. Extracted data were analysed to address the study aims. A qualitative content analysis with an inductive approach was carried out concerning the development and core context dimensions of the frameworks. The review is reported according to the PRISMA guidelines.ResultsThe database searches yielded a total of 1113 publications, of which 67 were considered potentially relevant based on the predetermined eligibility criteria, and retrieved in full text. Seventeen unique determinant frameworks were identified and included. Most were developed based on the literature and/or the developers' implementation experiences. Six of the frameworks explicitly referred to "context", but only four frameworks provided a specific definition of the concept. Instead, context was defined indirectly by description of various categories and sub-categories that together made up the context. Twelve context dimensions were identified, pertaining to different aggregation levels. The most widely addressed context dimensions were organizational support, financial resources, social relations and support, and leadership.ConclusionsThe findings suggest variation with regard to how the frameworks were developed and considerable inconsistency in terms used for contextual determinants, how context is conceptualized, and which contextual determinants are accounted for in frameworks used in implementation science. Common context dimensions were identified, which can facilitate research that incorporates a theory of context, i.e. assumptions about how different dimensions may influence each other and affect implementation outcomes. A thoughtful application of the concept and a more consistent terminology would enhance transparency, simplify communication among researchers, and facilitate comparison across studies.

Project description:BackgroundThe uptake, adoption and integration of new medicines and treatment regimens within healthcare delivery can take a decade or more. Increasingly, implementation science (IS) research is being used to bridge this gap between the availability of new therapeutic evidence and its actual application in clinical practice. Little is known, however, about the quality of IS research in this area, including the degree to which theories, models and frameworks (TMFs) are being used. The objective of this study was to conduct a scoping review of the use of TMFs in implementation research involving medicinal products.MethodsA search was conducted for English language abstracts and manuscripts describing the application of TMFs in IS studies for medicinal products. Eligible publications were those published between 1 January 1974 and 12 December 2022. All records were screened at the title and abstract stage; included full-text papers were abstracted using data extraction tables designed for the study. Study quality was appraised using the Implementation Research Development Tool.ResultsThe initial scoping search identified 2697 publications, of which 9 were ultimately eligible for inclusion in the review. Most studies were published after 2020 and varied in their objectives, design and therapeutic area. Most studies had sample sizes of fewer than 50 participants, and all focused on the post-marketing phase of drug development. The TMF most frequently used was the Consolidated Framework for Implementation Research (CFIR). Although most studies applied all TMF domains, TMF use was limited to instrument development and/or qualitative analysis. Quality appraisals indicated the need for engaging patients and other stakeholders in the implementation research, reporting on the cost of implementation strategies, and evaluating the unintended consequences of implementation efforts.ConclusionsWe found that few IS studies involving medicinal products reported using TMFs. Those that did encompassed a wide variety of therapeutic indications and medicinal products; all were in the post-marketing phase and involved limited application of the TMFs. Researchers should consider conducting IS in earlier phases of drug development and integrating the TMFs throughout the research process. More consistent and in-depth use of TMFs may help advance research in this area.

Project description:BackgroundThere is mounting evidence supporting the effectiveness of task-shifted mental health interventions in low- and middle-income countries (LMIC). However, there has been limited systematic scale-up or sustainability of these programs, indicating a need to study implementation. One barrier to progress is a lack of locally relevant and valid implementation measures. We adapted an existing brief dissemination and implementation (D&I) measure which includes scales for acceptability, appropriateness, feasibility and accessibility for local use and studied its validity and reliability among a sample of consumers in Ukraine.MethodsLocal qualitative data informed adaptation of the measure and development of vignettes to test the reliability and validity. Participants were veterans and internally displaced persons (IDPs) recruited as part of a separate validity study of adapted mental health instruments. We examined internal consistency reliability, test-retest reliability, and construct and criterion validity for each scale on the measure. We randomly assigned half the participants to respond to a vignette depicting existing local psychiatric services which we knew were not well regarded, while the other half was randomized to a vignette describing a potentially more well-implemented mental health service. Criterion validity was assessed by comparing scores on each scale by vignette and by overall summary ratings of the programs described in the vignettes.ResultsN = 169 participated in the qualitative study and N = 153 participated in the validity study. Qualitative findings suggested the addition of several items to the measure and indicated the importance of addressing professionalism/competency of providers in both the scales and the vignettes. Internal consistency reliabilities ranged from α = 0.85 for feasibility to α = 0.91 for appropriateness. Test-rest reliabilities were acceptable to good for all scales (rho: 0.61-0.79). All scales demonstrated substantial and significant differences in average scores by vignette assignment (ORs: 2.21-5.6) and overall ratings (ORs: 5.1-14.47), supporting criterion validity.ConclusionsThis study represents an innovative mixed-methods approach to testing an implementation science measure in contexts outside the United States. Results support the reliability and validity of most scales for consumers in Ukraine. Challenges included large amounts of missing data due to participants' difficulties responding to questions about a hypothetical program.

Project description:BackgroundA proliferation of theories, models, and frameworks (TMFs) have been developed in the implementation science field to facilitate the implementation process. The basic features of these TMFs have been identified by several reviews. However, systematic appraisals on the quality of these TMFs are inadequate. To fill this gap, this study aimed to assess the usability, applicability, and testability of the current TMFs in a structured way.MethodsA scoping review method was employed. Electronic databases were searched to locate English and Chinese articles published between January 2000 and April 2022. Search terms were specific to implementation science. Additionally, hand searches were administered to identify articles from related reviews. Purpose and characteristics such as the type of TMF, analytical level, and observation unit were extracted. Structured appraisal criteria were adapted from Birken et al.'s Theory Comparison and Selection Tool (T-CaST) to conduct an in-depth analysis of the TMFs' usability, applicability, and testability.ResultsA total of 143 TMFs were included in this analysis. Among them, the most common purpose was to identify barriers and facilitators. Most TMFs applied the descriptive method to summarize the included constructs or the prescriptive method to propose courses of implementation actions. TMFs were mainly mid-range theories built on existing conceptual frameworks or demonstrated grand theories. The usability of the TMFs needs to be improved in terms of the provision of conceptually matched strategies to barriers and facilitators and instructions on the TMFs usage. Regarding the applicability, little attention was paid to the constructs of macro-level context, stages of scale-up and sustainability, and implementation outcomes like feasibility, cost, and penetration. Also, fewer TMFs could propose recommended research and measurement methods to apply the TMFs. Lastly, explicit hypotheses or propositions were lacking in most of the TMFs, and empirical evidence was lacking to support the claimed mechanisms between framework elements in testability.ConclusionsCommon limitations were found in the usability, application, and testability of the current TMFs. The findings of this review could provide insights for developers of TMFs for future theoretical advancements.