Dataset Information

Evaluation of stakeholder views on peer review of NIHR applications for funding: a qualitative study.

ABSTRACT: OBJECTIVES:Innovations resulting from research have both national and global impact, so selecting the most promising research studies to fund is crucial. Peer review of research funding applications is part of the selection process, and requires considerable resources. This study aimed to elicit stakeholder opinions about which factors contribute to and influence effective peer review of funding applications to the UK National Institute for Health Research (NIHR), and to identify possible minor improvements to current processes and any major changes or potential innovations to achieve a more efficient peer review process. DESIGN:Qualitative interviews with 30 stakeholders involved in the peer review process. PARTICIPANTS:Participants were drawn from three NIHR coordinating centres and represented four types of stakeholders: board members with responsibility for making funding decisions, applicants, external peer reviewers and NIHR staff. METHODS:All interviews were conducted by telephone apart from three that were face to face with NIHR staff. Data were analysed using a thematic template method. RESULTS:The responses from NIHR staff, board members and reviewers differed from those received from applicants. The first three groups focused on how well the process of peer review did or did not function. The applicants mentioned these points but in addition often reflected on how their personal application was assessed. Process improvements suggested included: developing a more proportionate review process; providing greater guidance, feedback, training, acknowledgement or incentives for peer reviewers; reducing the time commitment and amount of paperwork; and asking reviewers to comment on the importance, strengths and weaknesses of applications and flaws which are potentially 'fixable'. CONCLUSIONS:Overall, participants were supportive of the need for peer review in evaluating applications for research funding. This study revealed which parts of the process are working well and are valued, and barriers, difficulties and potential areas for improvement and development.

SUBMITTER: Turner S

PROVIDER: S-EPMC6303555 | biostudies-literature | 2018 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPeer review of grant applications has been criticized as lacking reliability. Studies showing poor agreement among reviewers supported this possibility but usually focused on reviewers' scores and failed to investigate reasons for disagreement. Here, our goal was to determine how reviewers rate applications, by investigating reviewer practices and grant assessment criteria.Methods and findingsWe first collected and analyzed a convenience sample of French and international calls for proposals and assessment guidelines, from which we created an overall typology of assessment criteria comprising nine domains relevance to the call for proposals, usefulness, originality, innovativeness, methodology, feasibility, funding, ethical aspects, and writing of the grant application. We then performed a qualitative study of reviewer practices, particularly regarding the use of assessment criteria, among reviewers of the French Academic Hospital Research Grant Agencies (Programmes Hospitaliers de Recherche Clinique, PHRCs). Semi-structured interviews and observation sessions were conducted. Both the time spent assessing each grant application and the assessment methods varied across reviewers. The assessment criteria recommended by the PHRCs were listed by all reviewers as frequently evaluated and useful. However, use of the PHRC criteria was subjective and varied across reviewers. Some reviewers gave the same weight to each assessment criterion, whereas others considered originality to be the most important criterion (12/34), followed by methodology (10/34) and feasibility (4/34). Conceivably, this variability might adversely affect the reliability of the review process, and studies evaluating this hypothesis would be of interest.ConclusionsVariability across reviewers may result in mistrust among grant applicants about the review process. Consequently, ensuring transparency is of the utmost importance. Consistency in the review process could also be improved by providing common definitions for each assessment criterion and uniform requirements for grant application submissions. Further research is needed to assess the feasibility and acceptability of these measures.

Project description:BackgroundClinical quality registries (CQRs) can likely improve quality in healthcare and research. However, studies indicate that effective use of CQRs is hindered by lack of engagement and interest among stakeholders, as well as factors related to organisational context, registry design and data quality. To fulfil the potential of CQRs, more knowledge on stakeholders' perceptions of the factors that will facilitate or hamper the development of CQRs is essential to the more appropriate targeting of registry implementation and the subsequent use of the data. The primary aim of this study was to examine factors that can potentially affect the development of a national CQR for interventional radiology in Norway from the perspective of stakeholders. Furthermore, we wanted to identify the intervention functions likely to enable CQR development. Only one such registry, located in Sweden, has been established. To provide a broader context for the Norwegian study, we also sought to investigate experiences with the development of this registry.MethodsA qualitative study of ten Norwegian radiologists and radiographers using focus groups was conducted, and an in-depth interview with the initiator of the Swedish registry was carried out. Questions were based on the Capability, Opportunity and Motivation for Behaviour Model and the Theoretical Domains Framework. The participants' responses were categorised into predefined themes using a deductive process of thematic analysis.ResultsKnowledge of the rationale used in establishing a CQR, beliefs about the beneficial consequences of a registry for quality improvement and research and an opportunity to learn from a well-developed registry were perceived by the participants as factors facilitating CQR development. The study further identified a range of development barriers related to environmental and resource factors (e.g., a lack of organisational support, time) and individuallevel factors (e.g., role boundaries, resistance to change), as well as several intervention functions likely to be appropriate in targeting these barriers.ConclusionThis study provides a deeper understanding of factors that may be involved in the behaviour of stakeholders regarding the development of a CQR. The findings may assist in designing, implementing and evaluating a methodologically rigorous CQR intervention.

Dataset Information

Evaluation of stakeholder views on peer review of NIHR applications for funding: a qualitative study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets