Dataset Information

Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level.

ABSTRACT: Despite their recognized limitations, bibliometric assessments of scientific productivity have been widely adopted. We describe here an improved method to quantify the influence of a research article by making novel use of its co-citation network to field-normalize the number of citations it has received. Article citation rates are divided by an expected citation rate that is derived from performance of articles in the same field and benchmarked to a peer comparison group. The resulting Relative Citation Ratio is article level and field independent and provides an alternative to the invalid practice of using journal impact factors to identify influential papers. To illustrate one application of our method, we analyzed 88,835 articles published between 2003 and 2010 and found that the National Institutes of Health awardees who authored those papers occupy relatively stable positions of influence across all disciplines. We demonstrate that the values generated by this method strongly correlate with the opinions of subject matter experts in biomedical research and suggest that the same approach should be generally applicable to articles published in all areas of science. A beta version of iCite, our web tool for calculating Relative Citation Ratios of articles listed in PubMed, is available at https://icite.od.nih.gov.

SUBMITTER: Hutchins BI

PROVIDER: S-EPMC5012559 | biostudies-literature | 2016 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level.

Hutchins B Ian BI Yuan Xin X Anderson James M JM Santangelo George M GM

PLoS biology 20160906 9

Despite their recognized limitations, bibliometric assessments of scientific productivity have been widely adopted. We describe here an improved method to quantify the influence of a research article by making novel use of its co-citation network to field-normalize the number of citations it has received. Article citation rates are divided by an expected citation rate that is derived from performance of articles in the same field and benchmarked to a peer comparison group. The resulting Relative ...[more]

PMID: 27599104

Similar Datasets

Project description:BackgroundPublication metrics have been traditionally used to compare research productivity amongst academic faculty. However, traditional bibliometrics lack field-normalization and are often biased towards time-dependent publication factors. The National Institutes of Health (NIH) has developed a new, field-normalized, article-level metric, known as the "relative citation ratio" (RCR), that can be used to make accurate self, departmental, and cross-specialty comparisons of research productivity. This study evaluates the use of the RCR amongst academic orthopedic spine surgery faculty and analyzes physician factors associated with RCR values.MethodsA retrospective data analysis was performed using the iCite database for all fellowship trained orthopedic spine surgery (OSS) faculty associated with Accreditation Council for Graduate Medical Education (ACGME)-accredited orthopedic surgery residency program. Mean RCR, weighted RCR, and total publication count were compared by sex, career duration, academic rank, and presence of additional degrees. A value of 1.0 is the NIH-funded field-normalized standard. Student t-tests were used for two-group analyses whereas the analysis of variance tests (ANOVA) was used for between-group comparisons of three or more subgroups. Statistical significance was achieved at P < 0.05.ResultsA total of 502 academic OSS faculty members from 159 institutions were included in the analysis. Overall, OSS faculty were highly productive, with a median RCR of 1.62 (IQR 1.38-2.32) and a median weighted RCR of 68.98 (IQR 21.06-212.70). Advancing academic rank was associated with weighted RCR, career longevity was associated with mean RCR score, and male sex was associated with having increased mean and weighted RCR scores. All subgroups analyzed had an RCR value above 1.0.ConclusionsAcademic orthopedic spine surgery faculty produce impactful research as evidenced by the high median RCR relative to the standard value set by the NIH of 1.0. Our data can be used to evaluate research productivity in the orthopedic spine community.

Project description:BackgroundThe development of an author-level complementary metric could play a role in the process of academic promotion through objective evaluation of scholars' influence and impact.ObjectiveThe objective of this study was to evaluate the correlation between the Healthcare Social Graph (HSG) score, a novel social media influence and impact metric, and the h-index, a traditional author-level metric.MethodsThis was a cross-sectional study of health care stakeholders with a social media presence randomly sampled from the Symplur database in May 2020. We performed stratified random sampling to obtain a representative sample with all strata of HSG scores. We manually queried the h-index in two reference-based databases (Scopus and Google Scholar). Continuous features (HSG score and h-index) from the included profiles were summarized as the median and IQR. We calculated the Spearman correlation coefficients (ρ) to evaluate the correlation between the HSG scores and h-indexes obtained from Google Scholar and Scopus.ResultsA total of 286 (31.2%) of the 917 stakeholders had a Google Scholar h-index available. The median HSG score for these profiles was 61.1 (IQR 48.2), and the median h-index was 14.5 (IQR 26.0). For the 286 subjects with the HSG score and Google Scholar h-index available, the Spearman correlation coefficient ρ was 0.1979 (P<.001), indicating a weak positive correlation between these two metrics. A total of 715 (78%) of 917 stakeholders had a Scopus h-index available. The median HSG score for these profiles was 57.6 (IQR 46.4), and the median h-index was 7 (IQR 16). For the 715 subjects with the HSG score and Scopus h-index available, ρ was 0.2173 (P<.001), also indicating a weak positive correlation.ConclusionsWe found a weak positive correlation between a novel author-level complementary metric and the h-index. More than a chiasm between traditional citation metrics and novel social media-based metrics, our findings point toward a bridge between the two domains.

Project description:BackgroundDisparities in health outcomes across countries/areas are a central concern in public health and epidemiology. However, few authors have discussed legends that can be complemental to choropleth maps (CMs) and merely linked differences in outcomes to other factors like density in areas. Thus, whether health outcome rates on CMs showing the geographical distribution can be applied to publication citations in bibliometric analyses requires further study. The legends for visualizing the most influential areas in article citation disparities should have sophisticated designs. This paper illustrates the use of cumulative frequency (CF) map legends along with Lorenz curves and Gini coefficients (GC) to characterize the disparity of article citations in areas on CMs, based on the quantile classification method for classes.MethodsBy searching the PubMed database (pubmed.com), we used the keyword "Medicine" [journal] and downloaded 7042 articles published from 1945 to 2016. A total number of 41,628 articles were cited in Pubmed Central (PMC). The publication outputs based on the author's x-index were applied to plot CM about research contributions. The approach uses two methods (i.e., quantiles and equal total values for each class) with CF legends, in order to highlight the difference in x-indices across geographical areas on CMs. GC was applied to observe the x-index disparities in areas. Microsoft Excel Visual Basic for Application (VBA) was used for creating the CMs.ResultsResults showed that the most productive and cited countries in Medicine (Baltimore) were China and the US. The most-cited states and cities were Maryland (the US) and Beijing (China). Taiwan (x-index?=?24.38) ranked behind Maryland (25.97), but ahead of Beijing (16.9). China earned lower disparity (0.42) than the US (0.49) and the rest of the world (0.53) when the GCs were applied.ConclusionCF legends, particularly using the quantile classification for classes, can be useful to complement CMs. They also contain more information than those in standard CM legends that are commonly used with other classification methods. The steps of creating CM legends are described and introduced. Bibliometric analysts on CM can be replicated in the future.

Dataset Information

Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level.

Publications

Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets