Dataset Information

Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005-2019.

ABSTRACT: Despite growing demand for practicable methods of research evaluation, the use of bibliometric indicators remains controversial. This paper examines performance assessment practice in Europe-first, identifying the most commonly used bibliometric methods and, second, identifying the actors who have defined wide-spread practices. The framework of this investigation is Abbott's theory of professions, and I argue that indicator-based research assessment constitutes a potential jurisdiction for both individual experts and expert organizations. This investigation was conducted using a search methodology that yielded 138 evaluation studies from 21 EU countries, covering the period 2005 to 2019. Structured content analysis revealed the following findings: (1) Bibliometric research assessment is most frequently performed in the Nordic countries, the Netherlands, Italy, and the United Kingdom. (2) The Web of Science (WoS) is the dominant database used for public research assessment in Europe. (3) Expert organizations invest in the improvement of WoS citation data, and set technical standards with regards to data quality. (4) Citation impact is most frequently assessed with reference to international scientific fields. (5) The WoS classification of science fields retained its function as a de facto reference standard for research performance assessment. A detailed comparison of assessment practices between five dedicated organizations and other individual bibliometric experts suggests that corporate ownership and limited access to the most widely used citation databases have had a restraining effect on the development and diffusion of professional bibliometric methods during this period.

SUBMITTER: Jappe A

PROVIDER: S-EPMC7170233 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005-2019.

Jappe Arlette A

PloS one 20200420 4

Despite growing demand for practicable methods of research evaluation, the use of bibliometric indicators remains controversial. This paper examines performance assessment practice in Europe-first, identifying the most commonly used bibliometric methods and, second, identifying the actors who have defined wide-spread practices. The framework of this investigation is Abbott's theory of professions, and I argue that indicator-based research assessment constitutes a potential jurisdiction for both ...[more]

PMID: 32310984

Similar Datasets

Project description:BackgroundHeat stroke (HS) is an acute physical disorder that is associated with a high risk of organ dysfunction and even death. HS patients are usually treated symptomatically and conservatively; however, there remains a lack of specific and effective drugs in clinical practice. An analysis of publication contributions from institutions, journals and authors in different countries/regions was used to study research progress and trends regarding HS.MethodsWe extracted all relevant publications on HS between 1989 and 2019 from Web of Science. Using the Statistical Package for Social Science (SPSS, version 24) and the software GraphPad Prism 8, graphs were generated and statistical analyses were performed, while VOSviewer software was employed to visualize the research trends in HS from the perspectives of co-occurring keywords.ResultsAs of April 14, 2020, we identified 1443 publications with a citation frequency of 5216. The United States accounted for the largest number of publications (36.2%) and the highest number of citations (14,410), as well as the highest H-index at 74. Although the sum of publications from China ranked second, there was a contradiction between the quantity and quality of publications. Furthermore, Medicine & Science in Sports & Exercise published the most papers related to HS, with Lin MT publishing the most papers in this field (112), while the review by Knochel JP received the highest citation frequency at 969. The keyword heat-stress appeared most recently, with an average appearing year of 2015.5. In the clinical research cluster, exertional heat-stroke was determined to be the hotspot, while ambient-temperature and heat waves were the new trends in the epidemiological research cluster.ConclusionsCorresponding to this important field, while the contributions of the publications from the United States were significant, the mismatch between the quantity and quality of publications from China must be examined. Moreover, it is hypothesized that clinical and epidemiological studies may become hotspots in the near future.

Project description:To establish the feasibility of empirically testing crisis standards of care guidelines.DesignRetrospective single-center study.SettingICUs at a large academic medical center in the United States.SubjectsAdult, critically ill patients admitted to ICU, with 27 patients admitted for acute respiratory failure due to coronavirus disease 2019 and 37 patients admitted for diagnoses other than coronavirus disease 2019.InterventionsReview of electronic health record.Measurements and main resultsMany U.S. states released crisis standards of care guidelines with algorithms to allocate scarce healthcare resources during the coronavirus disease 2019 pandemic. We compared state guidelines that represent different approaches to incorporating disease severity and comorbidities: New York, Maryland, Pennsylvania, and Colorado. Following each algorithm, we calculated priority scores at the time of ICU admission for a cohort of patients with primary diagnoses of coronavirus disease 2019 and diseases other than coronavirus disease 2019 (n = 64). We assessed discrimination of 28-day mortality by area under the receiver operating characteristic curve. We simulated real-time decision-making by applying the triage algorithms to groups of two, five, or 10 patients. For prediction of 28-day mortality by priority scores, area under the receiver operating characteristic curve was 0.56, 0.49, 0.53, 0.66, and 0.69 for New York, Maryland, Pennsylvania, Colorado, and raw Sequential Organ Failure Assessment score algorithms, respectively. For groups of five patients, the percentage of decisions made without deferring to a lottery were 1%, 57%, 80%, 88%, and 95% for New York, Maryland, Pennsylvania, Colorado, and raw Sequential Organ Failure Assessment score algorithms, respectively. The percentage of decisions made without lottery was higher in the subcohort without coronavirus disease 2019, compared with the subcohort with coronavirus disease 2019.ConclusionsInclusion of comorbidities does not consistently improve an algorithm's performance in predicting 28-day mortality. Crisis standards of care algorithms result in a substantial percentage of tied priority scores. Crisis standards of care algorithms operate differently in cohorts with and without coronavirus disease 2019. This proof-of-principle study demonstrates the feasibility and importance of empirical testing of crisis standards of care guidelines to understand whether they meet their goals.

Dataset Information

Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005-2019.

Publications

Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005-2019.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets