Dataset Information

Assessing health research grant applications: A retrospective comparative review of a one-stage versus a two-stage application assessment process.

ABSTRACT: BACKGROUND:Research funders use a wide variety of application assessment processes yet there is little evidence on their relative advantages and disadvantages. A broad distinction can be made between processes with a single stage assessment of full proposals and those that first invite an outline, with full proposals invited at a second stage only for those which are shortlisted. This paper examines the effects of changing from a one-stage to a two-stage process within the UK's National Institute for Health Research's (NIHR) Research for Patient Benefit (RfPB) Programme which made this change in 2015. METHODS:A retrospective comparative design was used to compare eight one-stage funding competitions (912 applications) with eight two-stage funding competitions (1090 applications). Comparisons were made between the number of applications submitted, number of peer and lay reviews required, the duration of the funding round, average external peer review scores, and the total costs involved. RESULTS:There was a mean number of 114 applications per funding round for the one-stage process and 136 for the two-stage process. The one-stage process took a mean of 274 days and the two-stage process 348 days to complete, although those who were not funded (i.e. the majority) were informed at a mean of 195 days (mean 79 days earlier) under the two-stage process. The mean peer review score for full applications using the one-stage process was 6.46 and for the two-stage process 6.82 (5.6% difference using a 1-10 scale (with 10 being the highest), but there was no significant difference between the lay reviewer scores. The one-stage process required a mean of 423 peer reviews and 102 lay reviewers and the two-stage process required a mean of 208 peer reviews and 50 lay reviews (mean difference of 215 peer reviews and 52 lay reviews) per funding round. Overall cost per funding round changed from £148,908 for the one-stage process to £105,342 for the two-stage process saving approximately £43,566 per round. CONCLUSION:We conclude that a two-stage application process increases the number of applications submitted to a funding round, is less burdensome and more efficient for all those involved with the process, is cost effective and has a small increase in peer reviewer scores. For the addition of fewer than 11 weeks to the process substantial efficiencies are gained which benefit funders, applicants and science. Funding agencies should consider adopting a two-stage application assessment process.

SUBMITTER: Morgan B

PROVIDER: S-EPMC7067561 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing health research grant applications: A retrospective comparative review of a one-stage versus a two-stage application assessment process.

Morgan Ben B Yu Ly-Mee LM Solomon Tom T Ziebland Sue S

PloS one 20200312 3

<h4>Background</h4>Research funders use a wide variety of application assessment processes yet there is little evidence on their relative advantages and disadvantages. A broad distinction can be made between processes with a single stage assessment of full proposals and those that first invite an outline, with full proposals invited at a second stage only for those which are shortlisted. This paper examines the effects of changing from a one-stage to a two-stage process within the UK's National ...[more]

PMID: 32163468

Similar Datasets

Project description:BackgroundA diverse research workforce is essential for catalyzing biomedical advancements, but this workforce goal is hindered by persistent sex and racial/ethnic disparities among investigators receiving research grants from the National Institutes of Health (NIH). In response, the NIH-funded National Research Mentoring Network implemented a Grant Writing Coaching Program (GCP) to provide diverse cohorts of early-career investigators across the United States with intensive coaching throughout the proposal development process. We evaluated the GCP's national reach and short-term impact on participants' proposal submissions and funding outcomes.MethodsThe GCP was delivered as six similar but distinct models. All models began with an in-person group session, followed by a series of coaching sessions over 4 to 12 months. Participants were surveyed at 6-, 12- and 18-months after program completion to assess proposal outcomes (submissions, awards). Self-reported data were verified and supplemented by searches of public repositories of awarded grants when available. Submission and award rates were derived from counts of participants who submitted or were awarded at least one grant proposal in a category (NIH, other federal, non-federal).ResultsFrom June 2015 through March 2019, 545 investigators (67% female, 61% under-represented racial/ethnic minority, URM) from 187 different institutions participated in the GCP. Among them, 324 (59% of participants) submitted at least one grant application and 134 (41% of submitters) received funding. A total of 164 grants were awarded, the majority being from the NIH (93, 56%). Of the 74 R01 (or similar) NIH research proposals submitted by GCP participants, 16 have been funded thus far (56% to URM, 75% to women). This 22% award rate exceeded the 2016-2018 NIH success rates for new R01s.ConclusionInter- and intra-institutional grant writing coaching groups are a feasible and effective approach to supporting the grant acquisition efforts of early-career biomedical investigators, including women and those from URM groups.

Project description:BACKGROUND:Although women at all career stages are more likely to leave academia than men, early-career women are a particularly high-risk group. Research supports that women are less likely than men to receive research funding; however, whether funding success rates vary based on research content is unknown. We addressed gender differences in funding success rates for applications directed to one or more of 13 institutes, representing research communities, over a 15-year period. METHODS AND FINDINGS:We retrospectively reviewed 55,700 grant and 4,087 personnel award applications submitted to the Canadian Institutes of Health Research. We analyzed application success rates according to gender and the primary institute selected by applicants, pooled gender differences in success rates using random effects models, and fitted Poisson regression models to assess the effects of gender, time, and institute. We noted variable success rates among grant applications directed to selected institutes and declining success rates over time. Women submitted 31.1% and 44.7% of grant and personnel award applications, respectively. In the pooled estimate, women had significantly lower grant success (risk ratio [RR] 0.89, 95% confidence interval [CI] 0.84-0.94; p < 0.001; absolute difference 3.2%) compared with men, with substantial heterogeneity (I2 = 58%). Compared with men, women who directed grants to the Institutes of Cancer Research (RR 0.86, 95% CI 0.78-0.96), Circulatory and Respiratory Health (RR 0.74, 95% CI 0.66-0.84), Health Services and Policy Research (RR 0.78, 95% CI 0.68-0.90), and Musculoskeletal Health and Arthritis (RR 0.80, 95% CI 0.69-0.93) were significantly less likely to be funded, and those who directed grants to the Institute of Aboriginal People's Health (RR 1.67, 95% CI 1.0-2.7) were more likely to be funded. Overall, women also had significantly lower personnel award success (RR 0.75, 95% CI 0.65-0.86; p < 0.001; absolute difference 6.6%). Regression modelling identified that the effect of gender on grant success rates differed by institute and not time. Study limitations include use of institutes as a surrogate identifier, variability in designation of primary institute, and lack of access to metrics reflecting applicants, coapplicants, peer reviewers, and the peer-review process. CONCLUSIONS:Gender disparity existed overall in grant and personnel award success rates, especially for grants directed to selected research communities. Funding agencies should monitor for gender differences in grant success rates overall and by research content.

Project description:The One Health concept has inspired a rich vein of applied research and scholarly reflection over the past decade, yet with little influence from qualitative methodologists. With this overview, we describe the underpinning assumptions, purposes, and potential pitfalls of data collection techniques and methods of data analysis in key qualitative research methodologies. Our aim is to enhance One Health collaborations involving qualitative researchers, veterinary epidemiologists, and veterinary economists. There exist several distinct traditions of qualitative research, from which we draw selectively for illustrative purposes. Notwithstanding important distinctions, we emphasize commonalities and the potential for collaborative impact. The most important commonality is a shared focus on contextualizing human behavior and experience-culturally, economically, historically, and socially. We demonstrate that in-depth attention to context can assist veterinary economists and epidemiologists in drawing lessons from the implementation of policies and programs. In other words, qualitative researchers can assist One Health teams in distilling insights from "success stories," but also from adverse events and unintended consequences. As a result, qualitative researchers can contribute to One Health research and policy discussions by formulating more accurate and contextually-relevant parameters for future quantitative studies. When performed well, qualitative methodologies can help veterinary economists and epidemiologists to develop impactful research questions, to create more accurate and contextually-relevant parameters for quantitative studies, and to develop policy recommendations and interventions that are attuned to the political and socio-cultural context of their implementation. In sketching out the properties and features of influential methodologies, we underscore the value of working with seasoned qualitative researchers to incorporate questions about "what," "how," and "why" in mixed-methods research designs.

Project description:BackgroundImplementation of practice change is difficult and large scale implementation is particularly difficult. Among the challenges facing the healthcare system in general and healthcare organizations is the overuse of low value care. Improving medication safety also constitutes an attempt to reduce low value or potentially harmful care. Critical issues of overuse of low value practices and medication safety intersect in overtreatment of diabetes. Specifically, (over)intensive glycemic control increases hypoglycemia risk and morbidity without providing meaningful benefit. Our work indicates that among patients with diabetes who are at high risk for hypoglycemia, potential overtreatment is common. The Choosing Wisely Initiative to reduce low value care led by the American Board of Internal Medicine Foundation recommends not to treat most persons over 65 years of age with medications to reduce the A1c<7.5%. For most physicians this involves a change in practice. We will study the implementation of the Veterans Health Administration's Choosing Wisely Initiative (which includes hypoglycemic safety as a targeted condition) with three specific aims: (1) to assess the overall impact, both intended and unintended, of the Choosing Wisely Initiative to reduce overtreatment of diabetes in especially vulnerable populations; (2) to assess the impact of commitment to quality, teaching intensity, and safety culture on likelihood of overtreatment; and (3) to identify configurations of the implementation strategy, provider characteristics and organizational level factors that are associated with successful reduction of overtreatment rates by comparing high and low performers. Because focus on this initiative could have the unintended consequence of paying less attention to poor glycemic control (A1c>9%), we will also assess undertreatment.Methods/designWe will take advantage of a natural experiment and use a Type III Hybrid Design that focuses on study of implementation while at the same time observing and gathering information on clinical interventions and outcomes. This mixed methods study will use longitudinal data and qualitative methods including Qualitative Comparative Analyses.DiscussionOur multi-paradigm approach to examining potential mechanisms to explain the variation in reduction of rates of overtreatment will contribute to a better understanding of implementation of national dissemination projects and multi-component interventions in complex systems.

Dataset Information

Assessing health research grant applications: A retrospective comparative review of a one-stage versus a two-stage application assessment process.

Publications

Assessing health research grant applications: A retrospective comparative review of a one-stage versus a two-stage application assessment process.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets