Dataset Information

Strengthening and measuring research impact in global health: lessons from applying the FAIT framework.

ABSTRACT:

Background

To date, efforts to measure impact have largely focused on health research in high-income countries, reflecting where the majority of health research funding is spent. Nevertheless, there is a growing body of health and medical research being undertaken in low- and middle-income countries (LMICs), supported by both development aid and established research funders. The Framework to Assess the Impact of Translational health research (FAIT) combines three approaches to measuring research impact (Payback, economic assessment and case study narrative). Its aim is to strengthen the focus on translation and impact measurement in health research. FAIT has been used by several Australian research initiatives; however, it has not been used in LMICs. Our aim was to apply FAIT in an LMIC context and evaluate its utility.

Methods

We retrospectively applied all three FAIT methods to two LMIC studies using available data, supplemented with group discussion and further economic analyses. Results were presented in a scorecard format.

Results

FAIT helped clarify pathways of impact for the projects and provided new knowledge on areas of impact in several domains, including capacity-building for research, policy development and economic impact. However, there were constraints, particularly associated with calculating the return on investment in the LMIC context. The case study narrative provided a layperson's summary of the research that helped to explain outcomes and succinctly communicate lessons learnt.

Conclusion

Use of FAIT to assess the impact of LMIC research was both feasible and useful. We make recommendations related to prospective use, identification of metrics to support use of the Payback framework, and simplification of the economic assessment, which may facilitate further application in LMIC environments.

SUBMITTER: Dodd R

PROVIDER: S-EPMC6501392 | biostudies-literature | 2019 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Strengthening and measuring research impact in global health: lessons from applying the FAIT framework.

Dodd Rebecca R Ramanathan Shanthi S Angell Blake B Peiris David D Joshi Rohina R Searles Andrew A Webster Jacqui J

Health research policy and systems 20190506 1

<h4>Background</h4>To date, efforts to measure impact have largely focused on health research in high-income countries, reflecting where the majority of health research funding is spent. Nevertheless, there is a growing body of health and medical research being undertaken in low- and middle-income countries (LMICs), supported by both development aid and established research funders. The Framework to Assess the Impact of Translational health research (FAIT) combines three approaches to measuring ...[more]

PMID: 31060617

Similar Datasets

Project description:Impactful academic research plays a stellar role in society, pressing to ask the question of how one measures the impact created by different areas of academic research. Measuring the societal, cultural, economic and scientific impact of research is currently the priority of the National Science Foundation, European Commission and several research funding agencies. The recently concluded United Kingdom's national research quality exercise, the Research Excellence Framework (REF) 2014, which piloted impact assessment as part of the overall evaluation offers a lens to view how impact of research in different disciplines can be measured. Overall research quality was assessed through quality of outputs, 'impact' and research environment. We performed two studies using the REF 2014 as a case study. The first study on 363 Impact Case Studies (ICSs) submitted in 5 research areas (UoAs) reveals that, in general, the impact scores were constructed upon a combination of factors i.e. quantity of quartile-one (Q1) publications, quantity and value of grants/income, number of researchers stated in the ICSs, spin-offs created, discoveries/patents and presentation of esteem data, informing researchers/ academics of the factors to consider in order to achieve a better impact score in research impact assessments. However, there were differences among disciplines in terms of the role played by the factors in achieving their overall scores for the ICSs. The outcome of this study is thus a set of impact indicators, and their relationship with the overall score of impact of research in different disciplines as determined in REF2014, which would in the first instance provide some answers to impact measures that would be useful for researchers in different disciplines. The second study extracts the general themes of impact reported by universities by performing a word frequency analysis in all the ICSs submitted in the five chosen research areas, which were substantially varied owing to their fields.

Project description:ObjectivesTo (1) apply the Framework to Assess the Impact from Translational health research (FAIT) to Lessons from the Best to Better the Rest (LFTB), (2) report on impacts from LFTB and (3) assess the feasibility and outcomes from a retrospective application of FAIT.SettingThree Indigenous primary healthcare (PHC) centres in the Northern Territory, Australia; project coordinating centre distributed between Townsville, Darwin and Cairns and the broader LFTB learning community across Australia.ParticipantsLFTB research team and one representative from each PHC centre.Primary and secondary outcome measuresImpact reported as (1) quantitative metrics within domains of benefit using a modified Payback Framework, (2) a cost-consequence analysis given a return on investment was not appropriate and (3) a narrative incorporating qualitative evidence of impact. Data were gathered through in-depth stakeholder interviews and a review of project documentation, outputs and relevant websites.ResultsLFTB contributed to knowledge advancement in Indigenous PHC service delivery; enhanced existing capacity of health centre staff, researchers and health service users; enhanced supportive networks for quality improvement; and used a strengths-based approach highly valued by health centres. LFTB also leveraged between $A1.4 and $A1.6 million for the subsequent Leveraging Effective Ambulatory Practice (LEAP) Project to apply LFTB learnings to resource development and creation of a learning community to empower striving PHC centres.ConclusionRetrospective application of FAIT to LFTB, although not ideal, was feasible. Prospective application would have allowed Indigenous community perspectives to be included. Greater appreciation of the full benefit of LFTB including a measure of return on investment will be possible when LEAP is complete. Future assessments of impact need to account for the limitations of fully capturing impact when intermediate/final impacts have not yet been realised and captured.

Project description:BackgroundHealth and medical research funding agencies are increasingly interested in measuring the impact of funded research. We present a research impact case study for the first four years of an Australian National Health and Medical Research Council funded Centre of Research Excellence in Cardiovascular Outcomes Improvement (2016-2020). The primary aim of this paper was to explore the application of a research impact matrix to assess the impact of cardiovascular outcomes improvement research.MethodsWe applied a research impact matrix developed from a systematic review of existing methodological frameworks used to measure research impact. This impact matrix was used as a bespoke tool to identify and understand various research impacts over different time frames. Data sources included a review of existing internal documentation from the research centre and publicly available information sources, informal iterative discussions with 10 centre investigators, and confirmation of information from centre grant and scholarship recipients.ResultsBy July 2019, the impact on the short-term research domain category included over 41 direct publications, which were cited over 87 times (median journal impact factor of 2.84). There were over 61 conference presentations, seven PhD candidacies, five new academic collaborations, and six new database linkages conducted. The impact on the mid-term research domain category involved contributions towards the development of a national cardiac registry, cardiovascular guidelines, application for a Medicare Benefits Schedule reimbursement item number, introduction of patient-reported outcome measures into several databases, and the establishment of nine new industry collaborations. Evidence of long-term impacts were described as the development and use of contemporary management for aortic stenosis, a cardiovascular risk prediction model and prevention targets in several data registries, and the establishment of cost-effectiveness for stenting compared to surgery.ConclusionsWe considered the research impact matrix a feasible tool to identify evidence of academic and policy impact in the short- to midterm; however, we experienced challenges in capturing long-term impacts. Cost containment and broader economic impacts represented another difficult area of impact to measure.

Project description:BackgroundProviding evidence of impact highlights the benefits of medical research to society. Such evidence is increasingly requested by research funders and commonly relies on citation analysis. However, other indicators may be more informative. Although frameworks to demonstrate the impact of clinical research have been reported, no complementary framework exists for methodological research. Therefore, we assessed the impact of methodological research projects conducted or completed between 2009 and 2012 at the UK Medical Research Council Clinical Trials Unit Hub for Trials Methodology Research Hub, with a view to developing an appropriate framework.MethodsVarious approaches to the collection of data on research impact were employed. Citation rates were obtained using Web of Science (http://www.webofknowledge.com/) and analyzed descriptively. Semistructured interviews were conducted to obtain information on the rates of different types of research output that indicated impact for each project. Results were then pooled across all projects. Finally, email queries pertaining to methodology projects were collected retrospectively and their content analyzed.ResultsSimple citation analysis established the citation rates per year since publication for 74 methodological publications; however, further detailed analysis revealed more about the potential influence of these citations. Interviews that spanned 20 individual research projects demonstrated a variety of types of impact not otherwise collated, for example, applications and further developments of the research; release of software and provision of guidance materials to facilitate uptake; formation of new collaborations and broad dissemination. Finally, 194 email queries relating to 6 methodological projects were received from 170 individuals across 23 countries. They provided further evidence that the methodologies were impacting on research and research practice, both nationally and internationally. We have used the information gathered in this study to adapt an existing framework for impact of clinical research for use in methodological research.ConclusionsGathering evidence on research impact of methodological research from a variety of sources has enabled us to obtain multiple indicators and thus to demonstrate broad impacts of methodological research. The adapted framework developed can be applied to future methodological research and thus provides a tool for methodologists to better assess and report research impacts.

Project description:BackgroundMillions of dollars are invested annually under the umbrella of national health systems strengthening. Global health initiatives provide funding for low- and middle-income countries through disease-oriented programmes while maintaining that the interventions simultaneously strengthen systems. However, it is as yet unclear which, and to what extent, system-level interventions are being funded by these initiatives, nor is it clear how much funding they allocate to disease-specific activities - through conventional 'vertical-programming' approach. Such funding can be channelled to one or more of the health system building blocks while targeting disease(s) or explicitly to system-wide activities.MethodsWe operationalized the World Health Organization health system framework of the six building blocks to conduct a detailed assessment of Global Fund health system investments. Our application of this framework framework provides a comprehensive quantification of system-level interventions. We applied this systematically to a random subset of 52 of the 139 grants funded in Round 8 of the Global Fund to Fight AIDS, Tuberculosis and Malaria (totalling approximately US$1 billion).ResultsAccording to the analysis, 37% (US$ 362 million) of the Global Fund Round 8 funding was allocated to health systems strengthening. Of that, 38% (US$ 139 million) was for generic system-level interventions, rather than disease-specific system support. Around 82% of health systems strengthening funding (US$ 296 million) was allocated to service delivery, human resources, and medicines & technology, and within each of these to two to three interventions. Governance, financing, and information building blocks received relatively low funding.ConclusionsThis study shows that a substantial portion of Global Fund's Round 8 funds was devoted to health systems strengthening. Dramatic skewing among the health system building blocks suggests opportunities for more balanced investments with regard to governance, financing, and information system related interventions. There is also a need for agreement, by researchers, recipients, and donors, on keystone interventions that have the greatest system-level impacts for the cost-effective use of funds. Effective health system strengthening depends on inter-agency collaboration and country commitment along with concerted partnership among all the stakeholders working in the health system.

Dataset Information

Strengthening and measuring research impact in global health: lessons from applying the FAIT framework.

Background

Methods

Results

Conclusion

Publications

Strengthening and measuring research impact in global health: lessons from applying the FAIT framework.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets