Dataset Information

Citation gaming induced by bibliometric evaluation: A country-level comparative analysis.

ABSTRACT: It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. For investigating this topic, we propose a new inwardness indicator able to gauge the degree of scientific self-referentiality of a country. Inwardness is defined as the proportion of citations coming from the country over the total number of citations gathered by the country. A comparative analysis of the trends for the G10 countries in the years 2000-2016 reveals a net increase of the Italian inwardness. Italy became, both globally and for a large majority of the research fields, the country with the highest inwardness and the lowest rate of international collaborations. The change in the Italian trend occurs in the years following the introduction in 2011 of national regulations in which key passages of professional careers are governed by bibliometric indicators. A most likely explanation of the peculiar Italian trend is a generalized strategic use of citations in the Italian scientific community, both in the form of strategic author self-citations and of citation clubs. We argue that the Italian case offers crucial insights on the constitutive effects of evaluation systems. As such, it could become a paradigmatic case in the debate about the use of indicators in science-policy contexts.

SUBMITTER: Baccini A

PROVIDER: S-EPMC6739054 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Citation gaming induced by bibliometric evaluation: A country-level comparative analysis.

Baccini Alberto A Baccini Alberto A De Nicolao Giuseppe G Petrovich Eugenio E

PloS one 20190911 9

It is several years since national research evaluation systems around the globe started making use of quantitative indicators to measure the performance of researchers. Nevertheless, the effects on these systems on the behavior of the evaluated researchers are still largely unknown. For investigating this topic, we propose a new inwardness indicator able to gauge the degree of scientific self-referentiality of a country. Inwardness is defined as the proportion of citations coming from the countr ...[more]

PMID: 31509555

Similar Datasets

Project description:BackgroundPublication metrics have been traditionally used to compare research productivity amongst academic faculty. However, traditional bibliometrics lack field-normalization and are often biased towards time-dependent publication factors. The National Institutes of Health (NIH) has developed a new, field-normalized, article-level metric, known as the "relative citation ratio" (RCR), that can be used to make accurate self, departmental, and cross-specialty comparisons of research productivity. This study evaluates the use of the RCR amongst academic orthopedic spine surgery faculty and analyzes physician factors associated with RCR values.MethodsA retrospective data analysis was performed using the iCite database for all fellowship trained orthopedic spine surgery (OSS) faculty associated with Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopedic surgery residency program. Mean RCR, weighted RCR, and total publication count were compared by sex, career duration, academic rank, and presence of additional degrees. A value of 1.0 is the NIH-funded field-normalized standard. Student t-tests were used for two-group analyses whereas the analysis of variance tests (ANOVA) was used for between-group comparisons of three or more subgroups. Statistical significance was achieved at P < 0.05.ResultsA total of 502 academic OSS faculty members from 159 institutions were included in the analysis. Overall, OSS faculty were highly productive, with a median RCR of 1.62 (IQR 1.38-2.32) and a median weighted RCR of 68.98 (IQR 21.06-212.70). Advancing academic rank was associated with weighted RCR, career longevity was associated with mean RCR score, and male sex was associated with having increased mean and weighted RCR scores. All subgroups analyzed had an RCR value above 1.0.ConclusionsAcademic orthopedic spine surgery faculty produce impactful research as evidenced by the high median RCR relative to the standard value set by the NIH of 1.0. Our data can be used to evaluate research productivity in the orthopedic spine community.

Dataset Information

Citation gaming induced by bibliometric evaluation: A country-level comparative analysis.

Publications

Citation gaming induced by bibliometric evaluation: A country-level comparative analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets