Dataset Information

Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews.

ABSTRACT:

Background

How to assess the impact of research is of growing interest to funders, policy makers and researchers mainly to understand the value of investments and to increase accountability. Broadly speaking the term "research impact" refers to the contribution of research activities to achieve desired societal outcomes. The aim of this overview is to identify the most common approaches to research impact assessment, categories of impact and their respective indicators.

Methods

We systematically searched the relevant literature (PubMed, The Cochrane Library (1990-2009)) and funding agency websites. We included systematic reviews, theoretical and methodological papers, and empirical case-studies on how to evaluate research impact. We qualitatively summarised the included reports, as well the conceptual frameworks.

Results

We identified twenty-two reports belonging to four systematic reviews and 14 primary studies. These publications reported several theoretical frameworks and methodological approaches (bibliometrics, econometrics, ad hoc case studies). The "payback model" emerged as the most frequently used. Five broad categories of impact were identified: a) advancing knowledge, b) capacity building, c) informing decision-making, d) health benefits, e) broad socio-economic benefits. For each proposed category of impact we summarized a set of indicators whose pros and cons are presented and briefly discussed.

Conclusions

This overview is a comprehensive, yet descriptive, contribution to summarize the conceptual framework and taxonomy of an heterogeneous and evolving area of research. A shared and comprehensive conceptual framework does not seem to be available yet and its single components (epidemiologic, economic, and social) are often valued differently in different models.

SUBMITTER: Banzi R

PROVIDER: S-EPMC3141787 | biostudies-literature | 2011 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews.

Banzi Rita R Moja Lorenzo L Pistotti Vanna V Facchini Andrea A Liberati Alessandro A

Health research policy and systems 20110624

<h4>Background</h4>How to assess the impact of research is of growing interest to funders, policy makers and researchers mainly to understand the value of investments and to increase accountability. Broadly speaking the term "research impact" refers to the contribution of research activities to achieve desired societal outcomes. The aim of this overview is to identify the most common approaches to research impact assessment, categories of impact and their respective indicators.<h4>Methods</h4>We ...[more]

PMID: 21702930

Similar Datasets

Project description:BackgroundPerforming cancer research relies on substantial financial investment, and contributions in time and effort from patients. It is therefore important that this research has real life impacts which are properly evaluated. The optimal approach to cancer research impact evaluation is not clear. The aim of this study was to undertake a systematic review of review articles that describe approaches to impact assessment, and to identify examples of cancer research impact evaluation within these reviews.MethodsIn total, 11 publication databases and the grey literature were searched to identify review articles addressing the topic of approaches to research impact assessment. Information was extracted on methods for data collection and analysis, impact categories and frameworks used for the purposes of evaluation. Empirical examples of impact assessments of cancer research were identified from these literature reviews. Approaches used in these examples were appraised, with a reflection on which methods would be suited to cancer research impact evaluation going forward.ResultsIn total, 40 literature reviews were identified. Important methods to collect and analyse data for impact assessments were surveys, interviews and documentary analysis. Key categories of impact spanning the reviews were summarised, and a list of frameworks commonly used for impact assessment was generated. The Payback Framework was most often described. Fourteen examples of impact evaluation for cancer research were identified. They ranged from those assessing the impact of a national, charity-funded portfolio of cancer research to the clinical practice impact of a single trial. A set of recommendations for approaching cancer research impact assessment was generated.ConclusionsImpact evaluation can demonstrate if and why conducting cancer research is worthwhile. Using a mixed methods, multi-category assessment organised within a framework, will provide a robust evaluation, but the ability to perform this type of assessment may be constrained by time and resources. Whichever approach is used, easily measured, but inappropriate metrics should be avoided. Going forward, dissemination of the results of cancer research impact assessments will allow the cancer research community to learn how to conduct these evaluations.

Project description:BackgroundImplementation research may play an important role in reducing research waste by identifying strategies that support translation of evidence into practice. Implementation of healthcare interventions is influenced by multiple factors including the organisational context, implementation strategies and features of the intervention as perceived by people delivering and receiving the intervention. Recently, concepts relating to perceived features of interventions have been gaining traction in published literature, namely, acceptability, fidelity, feasibility, scalability and sustainability. These concepts may influence uptake of healthcare interventions, yet there seems to be little consensus about their nature and impact. The aim of this paper is to develop a testable conceptual framework of implementability of healthcare interventions that includes these five concepts.MethodsA multifaceted approach was used to develop and refine a conceptual framework of implementability of healthcare interventions. An overview of reviews identified reviews published between January 2000 and March 2021 that focused on at least one of the five concepts in relation to a healthcare intervention. These findings informed the development of a preliminary framework of implementability of healthcare interventions which was presented to a panel of experts. A nominal group process was used to critique, refine and agree on a final framework.ResultsA total of 252 publications were included in the overview of reviews. Of these, 32% were found to be feasible, 4% reported sustainable changes in practice and 9% were scaled up to other populations and/or settings. The expert panel proposed that scalability and sustainability of a healthcare intervention are dependent on its acceptability, fidelity and feasibility. Furthermore, acceptability, fidelity and feasibility require re-evaluation over time and as the intervention is developed and then implemented in different settings or with different populations. The final agreed framework of implementability provides the basis for a chronological, iterative approach to planning for wide-scale, long-term implementation of healthcare interventions.ConclusionsWe recommend that researchers consider the factors acceptability, fidelity and feasibility (proposed to influence sustainability and scalability) during the preliminary phases of intervention development, evaluation and implementation, and iteratively check these factors in different settings and over time.

Project description:BackgroundBillions of dollars are lost annually in health research that fails to create meaningful benefits for patients. Engaging in research co-design - the meaningful involvement of end-users in research - may help address this research waste. This rapid overview of reviews addressed three related questions, namely (1) what approaches to research co-design exist in health settings? (2) What activities do these research co-design approaches involve? (3) What do we know about the effectiveness of existing research co-design approaches? The review focused on the study planning phase of research, defined as the point up to which the research question and study design are finalised.MethodsReviews of research co-design were systematically identified using a rapid overview of reviews approach (PROSPERO: CRD42019123034). The search strategy encompassed three academic databases, three grey literature databases, and a hand-search of the journal Research Involvement and Engagement. Two reviewers independently conducted the screening and data extraction and resolved disagreements through discussion. Disputes were resolved through discussion with a senior author (PB). One reviewer performed quality assessment. The results were narratively synthesised.ResultsA total of 26 records (reporting on 23 reviews) met the inclusion criteria. Reviews varied widely in their application of 'research co-design' and their application contexts, scope and theoretical foci. The research co-design approaches identified involved interactions with end-users outside of study planning, such as recruitment and dissemination. Activities involved in research co-design included focus groups, interviews and surveys. The effectiveness of research co-design has rarely been evaluated empirically or experimentally; however, qualitative exploration has described the positive and negative outcomes associated with co-design. The research provided many recommendations for conducting research co-design, including training participating end-users in research skills, having regular communication between researchers and end-users, setting clear end-user expectations, and assigning set roles to all parties involved in co-design.ConclusionsResearch co-design appears to be widely used but seldom described or evaluated in detail. Though it has rarely been tested empirically or experimentally, existing research suggests that it can benefit researchers, practitioners, research processes and research outcomes. Realising the potential of research co-design may require the development of clearer and more consistent terminology, better reporting of the activities involved and better evaluation.

Project description:BackgroundAddressing deficiencies in the dissemination and transfer of research-based knowledge into routine clinical practice is high on the policy agenda both in the UK and internationally.However, there is lack of clarity between funding agencies as to what represents dissemination. Moreover, the expectations and guidance provided to researchers vary from one agency to another. Against this background, we performed a systematic scoping to identify and describe any conceptual/organising frameworks that could be used by researchers to guide their dissemination activity.MethodsWe searched twelve electronic databases (including MEDLINE, EMBASE, CINAHL, and PsycINFO), the reference lists of included studies and of individual funding agency websites to identify potential studies for inclusion. To be included, papers had to present an explicit framework or plan either designed for use by researchers or that could be used to guide dissemination activity. Papers which mentioned dissemination (but did not provide any detail) in the context of a wider knowledge translation framework, were excluded. References were screened independently by at least two reviewers; disagreements were resolved by discussion. For each included paper, the source, the date of publication, a description of the main elements of the framework, and whether there was any implicit/explicit reference to theory were extracted. A narrative synthesis was undertaken.ResultsThirty-three frameworks met our inclusion criteria, 20 of which were designed to be used by researchers to guide their dissemination activities. Twenty-eight included frameworks were underpinned at least in part by one or more of three different theoretical approaches, namely persuasive communication, diffusion of innovations theory, and social marketing.ConclusionsThere are currently a number of theoretically-informed frameworks available to researchers that can be used to help guide their dissemination planning and activity. Given the current emphasis on enhancing the uptake of knowledge about the effects of interventions into routine practice, funders could consider encouraging researchers to adopt a theoretically-informed approach to their research dissemination.

Project description:BackgroundThere are increasing expectations for researchers and knowledge users in the health system to use a research partnership approach, such as integrated knowledge translation, to increase the relevance and use of research findings in health practice, programmes and policies. However, little is known about how health research trainees engage in research partnership approaches such as IKT. In response, the purpose of this scoping review was to map and characterize the evidence related to using an IKT or other research partnership approach from the perspective of health research trainees in thesis and/or postdoctoral work.MethodsWe conducted this scoping review following the Joanna Briggs Institute methodology and Arksey and O'Malley's framework. We searched the following databases in June 2020: MEDLINE, Embase, CINAHL and PsycINFO. We also searched sources of unpublished studies and grey literature. We reported our findings in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews.ResultsWe included 74 records that described trainees' experiences using an IKT or other research partnership approach to health research. The majority of studies involved collaboration with knowledge users in the research question development, recruitment and data collection stages of the research process. Intersecting barriers to IKT or other research partnerships at the individual, interpersonal and organizational levels were reported, including lack of skills in partnership research, competing priorities and trainees' "outsider" status. We also identified studies that evaluated their IKT approach and reported impacts on partnership formation, such as valuing different perspectives, and enhanced relevance of research.ConclusionOur review provides insights for trainees interested in IKT or other research partnership approaches and offers guidance on how to apply an IKT approach to their research. The review findings can serve as a basis for future reviews and primary research focused on IKT principles, strategies and evaluation. The findings can also inform IKT training efforts such as guideline development and academic programme development.

Dataset Information

Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews.

Background

Methods

Results

Conclusions

Publications

Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets