Dataset Information

Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel.

ABSTRACT:

Objective

To quantify randomness and cost when choosing health and medical research projects for funding.

Design

Retrospective analysis.

Setting

Grant review panels of the National Health and Medical Research Council of Australia.

Participants

Panel members' scores for grant proposals submitted in 2009.

Main outcome measures

The proportion of grant proposals that were always, sometimes, and never funded after accounting for random variability arising from differences in panel members' scores, and the cost effectiveness of different size assessment panels.

Results

59% of 620 funded grants were sometimes not funded when random variability was taken into account. Only 9% (n = 255) of grant proposals were always funded, 61% (n = 1662) never funded, and 29% (n=788) sometimes funded. The extra cost per grant effectively funded from the most effective system was $A18,541 (£11,848; €13,482; $19,343).

Conclusions

Allocating funding for scientific research in health and medicine is costly and somewhat random. There are many useful research questions to be addressed that could improve current processes.

SUBMITTER: Graves N

PROVIDER: S-EPMC3181233 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:To investigate methods and processes for timely, efficient and good quality peer review of research funding proposals in health.A two-stage evidence synthesis: (1) a systematic map to describe the key characteristics of the evidence base, followed by (2) a systematic review of the studies stakeholders prioritised as relevant from the map on the effectiveness and efficiency of peer review 'innovations'. Standard processes included literature searching, duplicate inclusion criteria screening, study keyword coding, data extraction, critical appraisal and study synthesis.A total of 83 studies from 15 countries were included in the systematic map. The evidence base is diverse, investigating many aspects of the systems for, and processes of, peer review. The systematic review included eight studies from Australia, Canada, and the USA, evaluating a broad range of peer review innovations. These studies showed that simplifying the process by shortening proposal forms, using smaller reviewer panels, or expediting processes can speed up the review process and reduce costs, but this might come at the expense of peer review quality, a key aspect that has not been assessed. Virtual peer review using videoconferencing or teleconferencing appears promising for reducing costs by avoiding the need for reviewers to travel, but again any consequences for quality have not been adequately assessed.There is increasing international research activity into the peer review of health research funding. The studies reviewed had methodological limitations and variable generalisability to research funders. Given these limitations it is not currently possible to recommend immediate implementation of these innovations. However, many appear promising based on existing evidence, and could be adapted as necessary by funders and evaluated. Where feasible, experimental evaluation, including randomised controlled trials, should be conducted, evaluating impact on effectiveness, efficiency and quality.

Project description:BACKGROUND:Although women at all career stages are more likely to leave academia than men, early-career women are a particularly high-risk group. Research supports that women are less likely than men to receive research funding; however, whether funding success rates vary based on research content is unknown. We addressed gender differences in funding success rates for applications directed to one or more of 13 institutes, representing research communities, over a 15-year period. METHODS AND FINDINGS:We retrospectively reviewed 55,700 grant and 4,087 personnel award applications submitted to the Canadian Institutes of Health Research. We analyzed application success rates according to gender and the primary institute selected by applicants, pooled gender differences in success rates using random effects models, and fitted Poisson regression models to assess the effects of gender, time, and institute. We noted variable success rates among grant applications directed to selected institutes and declining success rates over time. Women submitted 31.1% and 44.7% of grant and personnel award applications, respectively. In the pooled estimate, women had significantly lower grant success (risk ratio [RR] 0.89, 95% confidence interval [CI] 0.84-0.94; p < 0.001; absolute difference 3.2%) compared with men, with substantial heterogeneity (I2 = 58%). Compared with men, women who directed grants to the Institutes of Cancer Research (RR 0.86, 95% CI 0.78-0.96), Circulatory and Respiratory Health (RR 0.74, 95% CI 0.66-0.84), Health Services and Policy Research (RR 0.78, 95% CI 0.68-0.90), and Musculoskeletal Health and Arthritis (RR 0.80, 95% CI 0.69-0.93) were significantly less likely to be funded, and those who directed grants to the Institute of Aboriginal People's Health (RR 1.67, 95% CI 1.0-2.7) were more likely to be funded. Overall, women also had significantly lower personnel award success (RR 0.75, 95% CI 0.65-0.86; p < 0.001; absolute difference 6.6%). Regression modelling identified that the effect of gender on grant success rates differed by institute and not time. Study limitations include use of institutes as a surrogate identifier, variability in designation of primary institute, and lack of access to metrics reflecting applicants, coapplicants, peer reviewers, and the peer-review process. CONCLUSIONS:Gender disparity existed overall in grant and personnel award success rates, especially for grants directed to selected research communities. Funding agencies should monitor for gender differences in grant success rates overall and by research content.