Dataset Information

Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?

ABSTRACT:

Background

The peer review system has been traditionally challenged due to its many limitations especially for allocating funding. Bibliometric indicators may well present themselves as a complement.

Objective

We analyze the relationship between peers' ratings and bibliometric indicators for Spanish researchers in the 2007 National R&D Plan for 23 research fields.

Methods and materials

We analyze peers' ratings for 2333 applications. We also gathered principal investigators' research output and impact and studied the differences between accepted and rejected applications. We used the Web of Science database and focused on the 2002-2006 period. First, we analyzed the distribution of granted and rejected proposals considering a given set of bibliometric indicators to test if there are significant differences. Then, we applied a multiple logistic regression analysis to determine if bibliometric indicators can explain by themselves the concession of grant proposals.

Results

63.4% of the applications were funded. Bibliometric indicators for accepted proposals showed a better previous performance than for those rejected; however the correlation between peer review and bibliometric indicators is very heterogeneous among most areas. The logistic regression analysis showed that the main bibliometric indicators that explain the granting of research proposals in most cases are the output (number of published articles) and the number of papers published in journals that belong to the first quartile ranking of the Journal Citations Report.

Discussion

Bibliometric indicators predict the concession of grant proposals at least as well as peer ratings. Social Sciences and Education are the only areas where no relation was found, although this may be due to the limitations of the Web of Science's coverage. These findings encourage the use of bibliometric indicators as a complement to peer review in most of the analyzed areas.

SUBMITTER: Cabezas-Clavijo A

PROVIDER: S-EPMC3695904 | biostudies-literature | 2013

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?

Cabezas-Clavijo Alvaro A Robinson-García Nicolás N Escabias Manuel M Jiménez-Contreras Evaristo E

PloS one 20130628 6

<h4>Background</h4>The peer review system has been traditionally challenged due to its many limitations especially for allocating funding. Bibliometric indicators may well present themselves as a complement.<h4>Objective</h4>We analyze the relationship between peers' ratings and bibliometric indicators for Spanish researchers in the 2007 National R&D Plan for 23 research fields.<h4>Methods and materials</h4>We analyze peers' ratings for 2333 applications. We also gathered principal investigators ...[more]

PMID: 23840840

Similar Datasets

Project description:Background and Objectives: The evaluative strength of available bibliometric tools in the field of clinical embryology has never been examined in the literature. The aim is to bring insight regarding the identity of clinical embryology research, introducing concerns when solely relying on the methodology of bibliometric analysis. Methods: An all-inclusive analysis of the most bibliometrically highlighted scientific contributions regarding the cornerstones of clinical embryology was performed employing the Scopus, Web of Science (WoS) and PubMed databases, between 1978-2018. An analysis of the number of publications, respective citations and h-index, g-index, along with m-quotient is presented. The top 30 contributing authors for each distinctive area of research are listed. An attempt at visualizing the yearly published articles, clusters, and collaborations of authors, along with the geographic origin of publications, is also presented. Results: Combining all searches and keywords yielded 54,522 results. In the Scopus database, employing the keyword "In Vitro Fertilization" yielded 41,292 results. The publications of the top five authors in each research field were analytically presented and compared to the total number of publications for each respective field. The research field of Preimplantation Genetic Diagnosis/Screening/Testing was allocated the highest percentage of publications produced by the top five authors. Regarding journal bibliometrics, based on the year 2017 metrics, there are only 29 journals according to WoS that refer to "Reproductive Biology", ranking it 187th among 235 disciplines. The USA produced the highest number of publications (12,537). Conclusion: Results indicate an explosion of interest published in the literature regarding the field of clinical embryology. Further analysis on collaborations and the trends involved should be of added value as productivity between countries varies significantly. This may guide researchers, in vitro fertilization professionals, and prospective authors during literature search, while proving useful regarding manuscript design and concurring on keywords and abstract content.

Project description:INTRODUCTION:The prevalence of smoking among persons with a mental illness has remained unchanged, being 2-3 times higher than the general population in high-income countries. Assessment of the volume and characteristics of research output over time can assist in identifying research priorities to promote progress within a field. The aim of this study was to undertake such an assessment in the field of smoking and mental illness. METHODS:A descriptive repeat cross-sectional study was conducted of peer-reviewed publications in Medline and PsycINFO for the periods 1993-1995, 2003-2005, and 2013-2015. Publications were classified as data- or non-data-based; data-based publications were further categorized by study type, population, setting, and for intervention-focused publications by level of evidence and research translation phase. RESULTS:Included were 547 articles published in 1993-1995 (n = 65), 2003-2005 (n = 153), and 2013-2015 (n = 329). The number and proportion of data-based publications significantly increased over time, although their focus remained predominantly descriptive (?83%); less than 14% of publications in any period had an intervention focus. The proportion of publications reporting on study populations with multiple diagnostic categories and recruiting from nonmental health settings, significantly increased from 1993-1995 to 2003-2005, however then plateaued by 2013-2015. The level of evidence provided by intervention-focused publications was suggested to increase over time, however there was no evident variation in translation phase. CONCLUSIONS:Research has increased over time to characterize smoking among those with a mental illness; however more is needed to inform the development and implementation of effective cessation interventions for this group. IMPLICATIONS:This is the first study to examine the volume and characteristics of research publications in the field of smoking and mental illness over time. The number of publications increased fivefold between 1993-1995 and 2013-2015. Between 1993-1995 and 2003-2005, progression was also indicated by increased: data-based publications, diagnostic diversity of samples, and variation in study settings; however further increases in such measures were not evident in 2013-2015. Notably, it continues to be the case that few intervention studies are undertaken. To achieve meaningful changes in the smoking prevalence of this group, a greater focus on research that assesses the effectiveness and implementation of tailored cessation interventions is required.

Project description:Action is urgently needed to curb the rising rates of non-communicable diseases (NCDs) in low- and middle-income countries (LMICs) and reduce the resulting social and economic burdens. There is global evidence about the most cost-effective interventions for addressing the main NCD risk factors such as tobacco use, unhealthy diets, physical inactivity, and alcohol misuse. However, it is unknown how much research is focused on informing the local adoption and implementation of these interventions.To assess the degree of research activity on NCD priority interventions in LMICs by using bibliometric analysis to quantify the number of relevant peer-reviewed scientific publications.A multidisciplinary, multi-lingual journal database was searched for articles on NCD priority interventions. The interventions examined emphasise population-wide, policy, regulation, and legislation approaches. The publication timeframe searched was the year 2000-2011. Of the 11,211 articles yielded, 525 met the inclusion criteria.Over the 12-year period, the number of articles published increased overall but differed substantially between regions: Latin America & Caribbean had the highest (127) and Middle East & North Africa had the lowest (11). Of the risk factor groups, 'tobacco control' led in publications, with 'healthy diets and physical activity' and 'reducing harmful alcohol use' in second and third place. Though half the publications had a first author from a high-income country institutional affiliation, developing country authorship had increased in recent years.While rising global attention to NCDs has likely produced an increase in peer-reviewed publications on NCDs in LMICs, publication rates directly related to cost-effective interventions are still very low, suggesting either limited local research activity or limited opportunities for LMIC researchers to publish on these issues. More research is needed on high-priority interventions and research funders should re-examine if intervention research is enough of a funding priority.

Dataset Information

Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?

Background

Objective

Methods and materials

Results

Discussion

Publications

Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure