A citation-based, author- and age-normalized, logarithmic index for evaluation of individual researchers independently of publication counts [version 1; referees: 2 approved]
Ontology highlight
ABSTRACT:
SUBMITTER: Belikov A
PROVIDER: S-EPMC4654436 | biostudies-literature | 2015 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Quantifying and comparing the scientific output of researchers has become critical for governments, funding agencies and universities. Comparison by reputation and direct assessment of contributions to the field is no longer possible, as the number of scientists increases and traditional definitions about scientific fields become blurred. The h-index is often used for comparing scientists, but has several well-documented shortcomings. In this paper, we introduce a new index for measuring and comparing the publication records of scientists: the pagerank-index (symbolised as π). The index uses a version of pagerank algorithm and the citation networks of papers in its computation, and is fundamentally different from the existing variants of h-index because it considers not only the number of citations but also the actual impact of each citation. We adapt two approaches to demonstrate the utility of the new index. Firstly, we use a simulation model of a community of authors, whereby we create various 'groups' of authors which are different from each other in inherent publication habits, to show that the pagerank-index is fairer than the existing indices in three distinct scenarios: (i) when authors try to 'massage' their index by publishing papers in low-quality outlets primarily to self-cite other papers (ii) when authors collaborate in large groups in order to obtain more authorships (iii) when authors spend most of their time in producing genuine but low quality publications that would massage their index. Secondly, we undertake two real world case studies: (i) the evolving author community of quantum game theory, as defined by Google Scholar (ii) a snapshot of the high energy physics (HEP) theory research community in arXiv. In both case studies, we find that the list of top authors vary very significantly when h-index and pagerank-index are used for comparison. We show that in both cases, authors who have collaborated in large groups and/or published less impactful papers tend to be comparatively favoured by the h-index, whereas the pagerank-index highlights authors who have made a relatively small number of definitive contributions, or written papers which served to highlight the link between diverse disciplines, or typically worked in smaller groups. Thus, we argue that the pagerank-index is an inherently fairer and more nuanced metric to quantify the publication records of scientists compared to existing measures.
| S-EPMC4545754 | biostudies-literature
Project description:BackgroundPreprints are preliminary reports that have not been peer-reviewed. In December 2019, a novel coronavirus appeared in China, and since then, scientific production, including preprints, has drastically increased. In this study, we intend to evaluate how often preprints about COVID-19 were published in scholarly journals and cited.MethodsWe searched the iSearch COVID-19 portfolio to identify all preprints related to COVID-19 posted on bioRxiv, medRxiv, and Research Square from January 1, 2020, to May 31, 2020. We used a custom-designed program to obtain metadata using the Crossref public API. After that, we determined the publication rate and made comparisons based on citation counts using non-parametric methods. Also, we compared the publication rate, citation counts, and time interval from posting on a preprint server to publication in a scholarly journal among the three different preprint servers.ResultsOur sample included 5,061 preprints, out of which 288 were published in scholarly journals and 4,773 remained unpublished (publication rate of 5.7%). We found that articles published in scholarly journals had a significantly higher total citation count than unpublished preprints within our sample (p < 0.001), and that preprints that were eventually published had a higher citation count as preprints when compared to unpublished preprints (p < 0.001). As well, we found that published preprints had a significantly higher citation count after publication in a scholarly journal compared to as a preprint (p < 0.001). Our results also show that medRxiv had the highest publication rate, while bioRxiv had the highest citation count and shortest time interval from posting on a preprint server to publication in a scholarly journal.ConclusionsWe found a remarkably low publication rate for preprints within our sample, despite accelerated time to publication by multiple scholarly journals. These findings could be partially attributed to the unprecedented surge in scientific production observed during the COVID-19 pandemic, which might saturate reviewing and editing processes in scholarly journals. However, our findings show that preprints had a significantly lower scientific impact, which might suggest that some preprints have lower quality and will not be able to endure peer-reviewing processes to be published in a peer-reviewed journal.
| S-EPMC7936563 | biostudies-literature
Project description:BackgroundAuthor self-citation is the practice of citing one's previous publications in a new publication. Its extent is unknown. We studied author self-citation, choosing the major clinical field of diabetes mellitus to represent the general medical literature.MethodsWe identified every article about diabetes mellitus in 170 hand-searched clinical journals published in 2000. For every article, we recorded the bibliographic citation and publication type (original or review article) and assessed the methodologic rigour. Citation information was obtained from the ISI Web of Knowledge in April 2003.ResultsOf 49,028 articles, 289 were about diabetes mellitus and had citation information. Citation counts ranged from 0 to 347 (median 6, interquartile range [IQR] 2-12). Author self-citation counts ranged from 0 to 16 (median 1, IQR 0-2). Author self-citations accounted for an average of 18% (95% confidence interval [CI] 15%-21%) and a median of 7% (95% CI 5%- 11%) of all citations of each publication that was cited at least once (n = 266). Original articles had double the mean proportion of author self-citations compared with review articles (19% v. 9%; median 7% v. 0%, difference 7%, 95% CI 0- 10%). Methodologic rigour and review type were not significantly associated with subsequent author self-citation.InterpretationNearly one-fifth of all citations to articles about diabetes mellitus in clinical journals in the year 2000 were author self-citations. The frequency of self-citation was not associated with the quality of publications. These findings are likely applicable to the general clinical medicine literature and may have important implications for the assessment of journal or publication importance and the process of scientific discovery.
| S-EPMC421720 | biostudies-literature
Project description:BackgroundThe aims of this bibliometric study were to determine author self-citation trends in high-impact orthodontic literature and to investigate possible association between self-citation and publication characteristics.MethodsSix orthodontic journals with the highest impact factor as ranked by 2017 Journal Citation Reports were screened for a full publication year (2018) for original research articles, reviews, and case reports. Eligible articles were scrutinized for article and author characteristics and citation metrics. Univariable and multivariable negative binomial regression was used to examine associations between self-citation incidence and publication characteristics.ResultsMedians for author self-citation rate of the most self-citing authors and self-citations were 3.03% (range 0-50) and 1 (range 0-19), respectively. In the univariable analysis, there was no association between self-citation counts and study type (P = 0.41), article topic (P = 0.61), number of authors (P = 0.62), and rank of authors (P = 0.56). Author origin (P = 0.001), gender (P = 0.001) and journal (P = 0.05) were associated with self-citation counts and in the multivariable analysis only origin and gender remained strong self-citation predictors. Asian authors and females self-cited significantly less often than all other regions and male authors.ConclusionsAuthors in orthodontics do not self-cite at a frequency that suggests potential citation manipulation. Author origin and gender were the only variables associated with citations counts. More bibliometric research is necessary to draw solid conclusions about author self-citation trends in orthodontic literature.
| S-EPMC7788150 | biostudies-literature
Project description:This Formal Comment presents an update to citation databases of top-cited scientists across all scientific fields, including more granular information on diverse indicators.
| S-EPMC7567353 | biostudies-literature
Project description:Many publications on COVID-19 were released on preprint servers such as medRxiv and bioRxiv. It is unknown how reliable these preprints are, and which ones will eventually be published in scientific journals. In this study, we use crowdsourced human forecasts to predict publication outcomes and future citation counts for a sample of 400 preprints with high Altmetric score. Most of these preprints were published within 1 year of upload on a preprint server (70%), with a considerable fraction (45%) appearing in a high-impact journal with a journal impact factor of at least 10. On average, the preprints received 162 citations within the first year. We found that forecasters can predict if preprints will be published after 1 year and if the publishing journal has high impact. Forecasts are also informative with respect to Google Scholar citations within 1 year of upload on a preprint server. For both types of assessment, we found statistically significant positive correlations between forecasts and observed outcomes. While the forecasts can help to provide a preliminary assessment of preprints at a faster pace than traditional peer-review, it remains to be investigated if such an assessment is suited to identify methodological problems in preprints.
| S-EPMC9515639 | biostudies-literature
Project description:ObjectivesPublication bias, non-publication, and selective reporting of animal studies limit progress toward the 3Rs (Replacement, Reduction, and Refinement) that guide ethical animal testing, waste public resources, and result in redundant research, which collectively undermine the public's trust in scientific reliability. In this study, we aimed to 1) validate findings from a previous follow-up study by our team that examined the publication rates of animal studies from protocol to publication and 2) identify incentives for improving publication rates in animal research.MethodsThe researchers responsible for the animal proposals (n = 210) from our previous study were contacted as participants for a Web-based survey between October 2019 and April 2020. Question types varied between free text questions, answer options based on a 5-point Likert scale and closed yes/no questions.ResultsIn total, 78 researchers responsible for 101 of 210 animal study proposals participated, yielding a response rate of 48.1%. Results showed that the publication rate increased from 67% in our follow-up study to 70%. According to a 5-point Likert scale (from 1 = "not relevant" to 5 = "extremely relevant"), the most widely accepted suggestions for increasing publication rates were "Publication costs for open access journals are fully covered by funders or universities" (mean 4.02, SD 1.01), "Performance-based allocation of intramural funds for results reporting of animal research not supporting the initial hypothesis (including preprints and repositories)" (mean 3.37, SD 1.05), and "Researchers receive more information from scientific journals that also publish non-significant results" (mean 3.30, SD 1.02).ConclusionWhile the extent of publication and publication practices have been thoroughly investigated for clinical trials, less data is available for animal research to date. Therefore, the study contributes in complementing the picture of publication practice in animal research. Suggestions from our survey may help improve the publication rates of animal studies.
| S-EPMC8101964 | biostudies-literature
Project description:BackgroundPublication records and citation indices often are used to evaluate academic performance. For this reason, obtaining or computing them accurately is important. This can be difficult, largely due to a lack of complete knowledge of an individual's publication list and/or lack of time available to manually obtain or construct the publication-citation record. While online publication search engines have somewhat addressed these problems, using raw search results can yield inaccurate estimates of publication-citation records and citation indices.MethodologyIn this paper, we present a new, automated method that produces estimates of an individual's publication-citation record from an individual's name and a set of domain-specific vocabulary that may occur in the individual's publication titles. Because this vocabulary can be harvested directly from a research web page or online (partial) publication list, our method delivers an easy way to obtain estimates of a publication-citation record and the relevant citation indices. Our method works by applying a series of stringent name and content filters to the raw publication search results returned by an online publication search engine. In this paper, our method is run using Google Scholar, but the underlying filters can be easily applied to any existing publication search engine. When compared against a manually constructed data set of individuals and their publication-citation records, our method provides significant improvements over raw search results. The estimated publication-citation records returned by our method have an average sensitivity of 98% and specificity of 72% (in contrast to raw search result specificity of less than 10%). When citation indices are computed using these records, the estimated indices are within of the true value 10%, compared to raw search results which have overestimates of, on average, 75%.ConclusionsThese results confirm that our method provides significantly improved estimates over raw search results, and these can either be used directly for large-scale (departmental or university) analysis or further refined manually to quickly give accurate publication-citation records.
| S-EPMC2924380 | biostudies-literature
Project description:Following scholars in Scientometrics as examples, we develop five author relationship networks, namely, co-authorship, author co-citation (AC), author bibliographic coupling (ABC), author direct citation (ADC), and author keyword coupling (AKC). The time frame of data sets is divided into two periods: before 2011 (i.e., T1) and after 2011 (i.e., T2). Through quadratic assignment procedure analysis, we found that some authors have ABC or AC relationships (i.e., potential communication relationship, PCR) but do not have actual collaborations or direct citations (i.e., actual communication relationship, ACR) among them. In addition, we noticed that PCR and AKC are highly correlated and that the old PCR and the new ACR are correlated and consistent. Such facts indicate that PCR tends to produce academic exchanges based on similar themes, and ABC bears more advantages in predicting potential relations. Based on tripartite citation analysis, including AC, ABC, and ADC, we also present an author-relation mining process. Such process can be used to detect deep and potential author relationships. We analyze the prediction capacity by comparing between the T1 and T2 periods, which demonstrate that relation mining can be complementary in identifying authors based on similar themes and discovering more potential collaborations and academic communities.
| S-EPMC5678701 | biostudies-literature
Project description:Academic departments, research clusters and evaluators analyze author and citation data to measure research impact and to support strategic planning. We created Scholar Metrics Scraper (SMS) to automate the retrieval of bibliometric data for a group of researchers. The project contains Jupyter notebooks that take a list of researchers as an input and exports a CSV file of citation metrics from Google Scholar (GS) to visualize the group's impact and collaboration. A series of graph outputs are also available. SMS is an open solution for automating the retrieval and visualization of citation data.
| S-EPMC10917922 | biostudies-literature