Dataset Information

Frameworks for evaluating health research capacity strengthening: a qualitative study.

ABSTRACT:

Background

Health research capacity strengthening (RCS) projects are often complex and hard to evaluate. In order to inform health RCS evaluation efforts, we aimed to describe and compare key characteristics of existing health RCS evaluation frameworks: their process of development, purpose, target users, structure, content and coverage of important evaluation issues. A secondary objective was to explore what use had been made of the ESSENCE framework, which attempts to address one such issue: harmonising the evaluation requirements of different funders.

Methods

We identified and analysed health RCS evaluation frameworks published by seven funding agencies between 2004 and 2012, using a mixed methods approach involving structured qualitative analyses of documents, a stakeholder survey and consultations with key contacts in health RCS funding agencies.

Results

The frameworks were intended for use predominantly by the organisations themselves, and most were oriented primarily towards funders' internal organisational performance requirements. The frameworks made limited reference to theories that specifically concern RCS. Generic devices, such as logical frameworks, were typically used to document activities, outputs and outcomes, but with little emphasis on exploring underlying assumptions or contextual constraints. Usage of the ESSENCE framework appeared limited.

Conclusions

We believe that there is scope for improving frameworks through the incorporation of more accessible information about how to do evaluation in practice; greater involvement of stakeholders, following evaluation capacity building principles; greater emphasis on explaining underlying rationales of frameworks; and structuring frameworks so that they separate generic and project-specific aspects of health RCS evaluation. The third and fourth of these improvements might assist harmonisation.

SUBMITTER: Boyd A

PROVIDER: S-EPMC3878679 | biostudies-literature | 2013 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Frameworks for evaluating health research capacity strengthening: a qualitative study.

Boyd Alan A Cole Donald C DC Cho Dan-Bi DB Aslanyan Garry G Bates Imelda I

Health research policy and systems 20131214

<h4>Background</h4>Health research capacity strengthening (RCS) projects are often complex and hard to evaluate. In order to inform health RCS evaluation efforts, we aimed to describe and compare key characteristics of existing health RCS evaluation frameworks: their process of development, purpose, target users, structure, content and coverage of important evaluation issues. A secondary objective was to explore what use had been made of the ESSENCE framework, which attempts to address one such ...[more]

PMID: 24330628

Similar Datasets

Project description:Strengthening research capacity in low- and middle-income countries is one of the most effective ways of advancing their health and development but the complexity and heterogeneity of health research capacity strengthening (RCS) initiatives means it is difficult to evaluate their effectiveness. Our study aimed to enhance understanding about these difficulties and to make recommendations about how to make health RCS evaluations more effective. Through discussions and surveys of health RCS funders, including the ESSENCE on Health Research initiative, we identified themes that were important to health RCS funders and used these to guide a systematic analysis of their evaluation reports. Eighteen reports, produced between 2000 and 2013, representing 12 evaluations, were purposefully selected from 54 reports provided by the funders to provide maximum variety. Text from the reports was extracted independently by two authors against a pre-designed framework. Information about the health RCS approaches, tensions and suggested solutions was re-constructed into a narrative. Throughout the process contacts in the health RCS funder agencies were involved in helping us to validate and interpret our results. The focus of the health RCS evaluations ranged from individuals and institutions to national, regional and global levels. Our analysis identified tensions around how much stakeholders should participate in an evaluation, the appropriate balance between measuring and learning and between a focus on short-term processes vs longer-term impact and sustainability. Suggested solutions to these tensions included early and ongoing stakeholder engagement in planning and evaluating health RCS, modelling of impact pathways and rapid assimilation of lessons learned for continuous improvement of decision making and programming. The use of developmental approaches could improve health RCS evaluations by addressing common tensions and promoting sustainability. Sharing learning about how to do robust and useful health RCS evaluations should happen alongside, not after, health RCS efforts.

Project description:BackgroundBuilding the capacity of allied health professionals to engage in research has been recognised as a priority due to the many benefits it brings for patients, healthcare professionals, healthcare organisations and society more broadly. There is increasing recognition of the need for a coordinated multi-strategy approach to building research capacity. The aim of this systematic review was to identify existing integrated models and frameworks which guide research capacity building for allied health professionals working in publicly funded secondary and tertiary healthcare organisations.MethodsA systematic review was undertaken searching five databases (Medline, CINAHL, Embase, AustHealth and Web of Science) using English language restrictions. Two authors independently screened and reviewed studies, extracted data and performed quality assessments using the Mixed Methods Appraisal Tool. Content and thematic analysis methods were used to code and categorise the data.ResultsA total of 8492 unique records were screened by title and abstract, of which 20 were reviewed in full-text. One quantitative study and five qualitative studies were included, each of which describing a research capacity building framework. Three interconnected and interdependent themes were identified as being essential for research capacity building, including 'supporting clinicians in research', 'working together' and 'valuing research for excellence'.ConclusionsThe findings of this systematic review have been synthesised to develop a succinct and integrated framework for research capacity building which is relevant for allied health professionals working in publicly funded secondary and tertiary healthcare organisations. This framework provides further evidence to suggest that research capacity building strategies are interlinked and interdependent and should be implemented as part of an integrated 'whole of system' approach, with commitment and support from all levels of leadership and management. Future directions for research include using behaviour change and knowledge translation theories to guide the implementation and evaluation of this new framework.Trial registrationThe protocol for this systematic review has been registered with PROSPERO. The registration number is CRD42018087476 .

Project description:BACKGROUND:Strengthening research capacity in low-and-middle-income countries is essential to drive socioeconomic development and to achieve the Sustainable Development Goals. Understanding strengths and weaknesses in institutions' research capacity can guide effective targeting of investments and resources. This study assessed the capacity of institutions undertaking research in natural science topics in Africa to identify priority capacity gaps for future investment. METHODS:Assessments were conducted in eight African institutions that were partners in a UK-Africa programme to strengthen research capacity in renewable energy, soil-related science, and water and sanitation. Assessments involved eighty-six interviews and three focus group discussions to identify institutions' research capacity strengths and gaps against an evidence-informed benchmark. Use of the same interview guides and data collection processes across all institutions meant that findings could be compared. RESULTS:Common research capacity gaps were: lack of, or poorly maintained, equipment; unreliable, slow procurement systems; insufficient opportunities for developing the skills of research support staff such as administrators and technicians; dysfunctional institutional email communication systems; insufficient focus on the development of 'soft' researcher skills such as ethics, academic writing and, in non-Anglophone countries, English language. Programme strengths were the South-South and South-North partnerships for sharing and cascading expertise and resources, joint writing of proposals and publications, and improved individual and institutional visibility. CONCLUSION:There were many similarities in research capacity gaps irrespective of the institutions' natural sciences research focus, and these were similar to those reported in the health sector. Common capacity needs are improving the skills of technicians and administrators to support research activities, soft skills training for researchers, and more effective pan-institutional e-communication systems. These could be strategic investment targets for the joint efforts of national governments and international organisations that fund programmes for strengthening research capacity in low- and middle-income countries.

Project description:BackgroundConflicts pose new challenges for health systems, requiring rapid and practical approaches to meet emerging needs on the ground. Lebanon has been highly influenced by surrounding conflicts in the Middle East and North Africa (MENA) region, especially the Syrian crisis. Strengthening research capacity to collect evidence on conflict in the MENA region and beyond is crucial to inform healthcare policy and practice. For targeted capacity strengthening interventions, the main objective of this paper is to present key findings of a needs assessment of conflict and health research in Lebanon. This will support recent efforts to scale up context-specific policies, interventions to strengthen the country's health system, and research capacity.MethodsThe study is based on 30 semi-structured interviews with key informants such as specialist academics, humanitarian workers and public sector officials.ResultsDespite being ranked third in the number of publications on biomedical and health research per capita in MENA and in hosting reputable universities which are considered central academic hubs in the region, lack of nationwide research culture, insufficient funding and limited access to data were reported to be major challenges for health researchers in Lebanon. Even with the ongoing efforts, poor impact of research on policy continues to be a persistent gap. Large disparities in research capacities and taught skills were reported between different universities in Lebanon, with a disproportionate emphasis on quantitative over qualitative skills. Most medical students are not trained to conduct research or to practice in conflict settings. Concerns were also expressed regarding the ethics of research conducted, specifically by local non-governmental organizations.RecommendationsTo conduct contextualized trainings on research skills with a stronger focus on qualitative approaches, medical practice, and ethical research in conflict. To better involve policymakers in designing research agendas by organizing multiple stakeholder meetings.ConclusionThe study indicates that health research in Lebanon is characterized by considerable strengths in terms of human capital and research capacities of certain universities. However, the Lebanese research infrastructure needs further development in terms of ensuring sustainable funding, providing access to data, teaching qualitative research skills, conducting ethical and multidisciplinary research, and promoting cross-sectoral knowledge transfer.

Project description:Background: TDR, The Special Programme for Research and Training hosted at the World Health Organization, has long supported Low- and Middle-Income Countries in strengthening research capacity through three training programmes: the Postgraduate Training Scheme (PGTS), the Clinical Research and Development Fellowship (CRDF), and the Structured Operational Research Training InitiaTive (SORT IT). In the advent of the COVID-19 pandemic, we assessed whether those trained through these programmes were involved in the COVID-19 response and if so, in which area(s) of the emergency response they were applying their skills. Methods: From the records for each training programme, we identified the individuals who had completed training during the relevant timespan of each programme: 1999-2018 for the CRDF scheme, 2015-2020 for PGTS, and 2009-2019 for SORT-IT. Between March and April 2020, we sent trainees an online questionnaire by e-mail. Results: Out of 1254 trained, 1143 could be contacted and 699 responded to the survey. Of the latter, 411 were involved with the COVID-19 response, of whom 315 (77%) were applying their acquired skills in 85 countries. With some overlap between programmes, 84% of those trained through CRDF were applying their skills in 27 countries, 91% of those trained through PGTS were applying their skills in 19 countries, and through SORT IT, this was 73% in 62 countries. Skills were being applied in various areas of the emergency response, including: emergency preparedness, situation analysis/surveillance, infection control and clinical management, data generation, mitigating the effect of COVID on the health system, and research. Depending on the type of training programme, 26-74% were involved in implementation, operational or clinical research. Conclusion: Research training programmes build research capacity and equip health workers with transferable core competencies and skillsets prior to epidemics. This becomes invaluable in building health system resilience at a time of pandemics.

Project description:ObjectivesDespite an estimated 300 000 mobile health apps on the market, there remains no consensus around helping patients and clinicians select safe and effective apps. In 2018, our team drew on existing evaluation frameworks to identify salient categories and create a new framework endorsed by the American Psychiatric Association (APA). We have since created a more expanded and operational framework Mhealth Index and Navigation Database (MIND) that aligns with the APA categories but includes objective and auditable questions (105). We sought to survey the existing space, conducting a review of all mobile health app evaluation frameworks published since 2018, and demonstrate the comprehensiveness of this new model by comparing it to existing and emerging frameworks.DesignWe conducted a scoping review of mobile health app evaluation frameworks.Data sourcesReferences were identified through searches of PubMed, EMBASE and PsychINFO with publication date between January 2018 and October 2020.Eligibility criteriaPapers were selected for inclusion if they meet the predetermined eligibility criteria-presenting an evaluation framework for mobile health apps with patient, clinician or end user-facing questions.Data extraction and synthesisTwo reviewers screened the literature separately and applied the inclusion criteria. The data extracted from the papers included: author and dates of publication, source affiliation, country of origin, name of framework, study design, description of framework, intended audience/user and framework scoring system. We then compiled a collection of more than 1701 questions across 79 frameworks. We compared and grouped these questions using the MIND framework as a reference. We sought to identify the most common domains of evaluation while assessing the comprehensiveness and flexibility-as well as any potential gaps-of MIND.ResultsNew app evaluation frameworks continue to emerge and expand. Since our 2019 review of the app evaluation framework space, more frameworks include questions around privacy (43) and clinical foundation (57), reflecting an increased focus on issues of app security and evidence base. The majority of mapped frameworks overlapped with at least half of the MIND categories. The results of this search have informed a database (apps.digitalpsych.org) that users can access today.ConclusionAs the number of app evaluation frameworks continues to rise, it is becoming difficult for users to select both an appropriate evaluation tool and to find an appropriate health app. This review provides a comparison of what different app evaluation frameworks are offering, where the field is converging and new priorities for improving clinical guidance.

Dataset Information

Frameworks for evaluating health research capacity strengthening: a qualitative study.

Background

Methods

Results

Conclusions

Publications

Frameworks for evaluating health research capacity strengthening: a qualitative study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets