Automated Research Impact Assessment: A New Bibliometrics Approach.
Ontology highlight
ABSTRACT:
SUBMITTER: Drew CH
PROVIDER: S-EPMC4791943 | biostudies-literature | 2016 Mar
REPOSITORIES: biostudies-literature
Publications
Automated Research Impact Assessment: A New Bibliometrics Approach.
Drew Christina H CH Pettibone Kristianna G KG Finch Fallis Owen FO Giles Douglas D Jordan Paul P
Scientometrics 20160130 3
As federal programs are held more accountable for their research investments, The National Institute of Environmental Health Sciences (NIEHS) has developed a new method to quantify the impact of our funded research on the scientific and broader communities. In this article we review traditional bibliometric analyses, address challenges associated with them, and describe a new bibliometric analysis method, the Automated Research Impact Assessment (ARIA). ARIA taps into a resource that has only ra ...[more]
Similar Datasets
Project description:IntroductionTo enhance the performance evaluation of Clinical and Translational Science Award (CTSA) hubs, we examined the utility of advanced bibliometric measures that go beyond simple publication counts to demonstrate the impact of translational research output.MethodsThe sampled data included North Carolina Translational and Clinical Science Institute (NC TraCS)-supported publications produced between September 2008 and March 2017. We adopted advanced bibliometric measures and a state-of-the-art bibliometric network analysis tool to assess research productivity, citation impact, the scope of research collaboration, and the clusters of research topics.ResultsTotally, 754 NC TraCS-supported publications generated over 24,000 citation counts by April 2017 with an average of 33 cites per article. NC TraCS-supported research papers received more than twice as many cites per year as the average National Institute of Health-funded research publications from the same field and time. We identified the top productive researchers and their networks within the CTSA hub. Findings demonstrated the impact of NC TraCS in facilitating interdisciplinary collaborations within the CTSA hub and across the CTSA consortium and connecting researchers with right peers and organizations.ConclusionBoth improved bibliometrics measures and bibliometric network analysis can bring new perspectives to CTSA evaluation via citation influence and the scope of research collaborations.
| S-EPMC7681148 | biostudies-literature
Project description:Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer-reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a specific researcher, is assessed through the use of various metrics. While researchers may be acquainted with such matrices, many do not know how to use them to enhance their careers. In addition to these metrics, a number of other factors should be taken into consideration to objectively evaluate a scientist's profile as a researcher and academician. Moreover, each metric has its own limitations that need to be considered when selecting an appropriate metric for evaluation. This paper provides a broad overview of the wide array of metrics currently in use in academia and research. Popular metrics are discussed and defined, including traditional metrics and article-level metrics, some of which are applied to researchers for a greater understanding of a particular concept, including varicocele that is the thematic area of this Special Issue of Asian Journal of Andrology. We recommend the combined use of quantitative and qualitative evaluation using judiciously selected metrics for a more objective assessment of scholarly output and research impact.
| S-EPMC4770502 | biostudies-other
Project description:Data acquisition in (pre)clinical studies is often based on a hypothesis. Numerical algorithms, however, may help to find biomarkers from existing data without formulating any hypothesis. By simply assessing whether a statistical relationship exists between two parameters from a (unlimited) database, every (in)conceivable combination of data becomes a hypothesis. The aim was to create an unbiased and highly automated approach for secondary analysis of (pre)clinical research, including the possibility of a non-linear functional relationship. In our example, an almost homogeneous database was formed by overall 45 parameters (vital, blood and plasma parameters) measured in 11 individual experimental studies at 6 different time points using 57 rats without and 63 rats with systemic inflammation following lipopolysaccharide infusion. For each rat, four group classifiers (treatment, survival, study, ID) were used to get valid samples by a later filtering of the statistical base. Any information about the hypothesis leading to the respective studies was suppressed. In order to assess whether a statistical relationship exists, a total of six different functional prototypes (linear and non-linear) were postulated and examined for their regression. Regression quality, correlation and significance were obtained in form of matrices. In our example, ultimately 510 300 regressions were optimized, automatically evaluated and filtered. The developed algorithm is able to reveal statistical relationships from a nearly crude database with low effort by systematic and unbiased analysis. The finding of well-known correlations proves its reliability, whose validity could be increased by clean aggregation of different studies. In addition, new interesting hints for future research could be gained. Thus, unknown markers can be found which are associated with an increased risk of death during systemic inflammation and sepsis. A further development of the program is planned including multiple regressions (more than two parameters could be related to each other) or cluster analysis.
| S-EPMC6859257 | biostudies-literature
Project description:This bibliometric study of a large publication set dealing with research on climate change aims at mapping the relevant literature from a bibliometric perspective and presents a multitude of quantitative data: (1) The growth of the overall publication output as well as (2) of some major subfields, (3) the contributing journals and countries as well as their citation impact, and (4) a title word analysis aiming to illustrate the time evolution and relative importance of specific research topics. The study is based on 222,060 papers (articles and reviews only) published between 1980 and 2014. The total number of papers shows a strong increase with a doubling every 5-6 years. Continental biomass related research is the major subfield, closely followed by climate modeling. Research dealing with adaptation, mitigation, risks, and vulnerability of global warming is comparatively small, but their share of papers increased exponentially since 2005. Research on vulnerability and on adaptation published the largest proportion of very important papers (in terms of citation impact). Climate change research has become an issue also for disciplines beyond the natural sciences. The categories Engineering and Social Sciences show the strongest field-specific relative increase. The Journal of Geophysical Research, the Journal of Climate, the Geophysical Research Letters, and Climatic Change appear at the top positions in terms of the total number of papers published. Research on climate change is quantitatively dominated by the USA, followed by the UK, Germany, and Canada. The citation-based indicators exhibit consistently that the UK has produced the largest proportion of high impact papers compared to the other countries (having published more than 10,000 papers). Also, Switzerland, Denmark and also The Netherlands (with a publication output between around 3,000 and 6,000 papers) perform top-the impact of their contributions is on a high level. The title word analysis shows that the term climate change comes forward with time. Furthermore, the term impact arises and points to research dealing with the various effects of climate change. The discussion of the question of human induced climate change towards a clear fact (for the majority of the scientific community) stimulated research on future pathways for adaptation and mitigation. Finally, the term model and related terms prominently appear independent of time, indicating the high relevance of climate modeling.
| S-EPMC4966958 | biostudies-other
Project description:This paper analyzes the concordance between bibliometrics and peer review. It draws evidence from the data of two experiments of the Italian governmental agency for research evaluation. The experiments were performed by the agency for validating the adoption in the Italian research assessment exercises of a dual system of evaluation, where some outputs were evaluated by bibliometrics and others by peer review. The two experiments were based on stratified random samples of journal articles. Each article was scored by bibliometrics and by peer review. The degree of concordance between the two evaluations is then computed. The correct setting of the experiments is defined by developing the design-based estimation of the Cohen's kappa coefficient and some testing procedures for assessing the homogeneity of missing proportions between strata. The results of both experiments show that for each research areas of science, technology, engineering and mathematics the degree of agreement between bibliometrics and peer review is-at most-weak at an individual article level. Thus, the outcome of the experiments does not validate the use of the dual system of evaluation in the Italian research assessments. More in general, the very weak concordance indicates that metrics should not replace peer review at the level of individual article. Hence, the use of the dual system in a research assessment might worsen the quality of information compared to the adoption of peer review only or bibliometrics only.
| S-EPMC7673579 | biostudies-literature
Project description:The Brassicaceae is a worldwide family that produces ornamental flowers, edible vegetables, and oilseed plants, with high economic value in agriculture, horticulture, and landscaping. This study used the Web of Science core dataset and the CiteSpace bibliometric tool to quantitatively visualize the number of publications, authors, institutions, and countries of 3139 papers related to Brassicaceae plants from 2002 to 2022. The keywords and references were divided into two phases: Phase 1 (2002-2011) and Phase 2 (2012-2022) for quantitative and qualitative analysis. The results showed: An average annual publication volume of 149 articles, with an overall fluctuating upward trend; the research force was mainly led by Professor Ihsan A. Al-shehbaz from Missouri Botanical Garden; and the United States had the highest number of publications. In the first phase, research focused on the phylogeny of Brassicaceae plants, while the second phase delved into diverse research based on previous studies, research in areas such as polyploidy, molecular technique, physiology, and hyperaccumulator has been extended. Based on this research, we propounded some ideas for future studies on Brassicaceae plants and summarized the research gaps.
| S-EPMC10864531 | biostudies-literature
Project description:Mountain areas provide essential resources for a significant proportion of the Earth's population. This study presents the development of mountain research between 1900 and 2019 based on peer-reviewed articles in English listed in Web of ScienceTM (WOS). We analyzed the number of publications over time, journals and scientific categories, frequent topics, and geographical distributions based on 40 mountain ranges and authors' countries as well as institutional contributions. From 1900-2019, 195k ±10% mountain research papers were published; over 50% from 2010-2019. While papers were published in more than 1000 different journals, indicating a wide range of disciplines engaged in mountain research, 94% of the papers were assigned to "Science & Technology", only <5% to "Social Sciences" and "Arts & Humanities". The most papers were written by researchers in the USA, followed by China. The number of papers per area or capita showed high variability across the investigated mountain ranges. Thus, geographically and disciplinarily more balanced research activities and better accessibility of knowledge about mountain regions are recommended.
| S-EPMC9409586 | biostudies-literature
Project description:IntroductionConduct bibliometric analyses documenting the output of National Institutes of Health (NIH) tobacco-related and Food and Drug Administration (FDA) tobacco regulatory science (FDA-TRS) research portfolios.Aims and methodsPubMed identifiers for publications between 2015 and 2020 citing tobacco funding by NIH and/or FDA were imported into NIH iCite generating measures of productivity and influence, including number of citations, journal, relative citation ratios (RCR), and comparison of research influence across Web of Science (WoS) disciplines. Coauthorship and measures of centrality among and between NIH and FDA-supported investigators gauged collaboration.ResultsBetween FY 2015 and 2020, 8160 publications cited funding from NIH tobacco-related grants, 1776 cited FDA-TRS grants and 496 cited Common funding (ie, both NIH and FDA-TRS funding). The proportion of publications citing NIH grants declined while those citing FDA-TRS or Common funding rose significantly. Publications citing Common funding showed the highest influence (mean RCR = 2.52). Publications citing FDA-TRS funding displayed higher median RCRs than publications citing NIH funding in most WoS categories. Higher translational progress was estimated over time for FDA-TRS and Common publications compared to NIH publications. Authors citing Common funding scored highest across all collaboration measures.ConclusionsThis study demonstrates the high bibliometric output of tobacco research overall. The rise in publications citing FDA-TRS and Common likely reflects increased funding for TRS research. Higher RCRs across WoS subject categories and trends towards human translation among FDA-TRS and Common publications indicate focus on research to inform regulation. This analysis suggests that FDA support for TRS has expanded the field of tobacco control resulting in sustained productivity, influence, and collaboration.ImplicationsThis paper is the first effort to better describe the impact of tobacco research resulting from the addition of FDA funding for TRS in the past decade. The analysis provides impetus for further investigation into the publication topics and their focus which would offer insight into the specific evidence generated on tobacco control and regulation.
| S-EPMC10202643 | biostudies-literature
Project description:IntroductionThe Semantic Web community provides a common Resource Description Framework (RDF) that allows representation of resources such that they can be linked. To maximize the potential of linked data - machine-actionable interlinked resources on the Web - a certain level of quality of RDF resources should be established, particularly in the biomedical domain in which concepts are complex and high-quality biomedical ontologies are in high demand. However, it is unclear which quality metrics for RDF resources exist that can be automated, which is required given the multitude of RDF resources. Therefore, we aim to determine these metrics and demonstrate an automated approach to assess such metrics of RDF resources.MethodsAn initial set of metrics are identified through literature, standards, and existing tooling. Of these, metrics are selected that fulfil these criteria: (1) objective; (2) automatable; and (3) foundational. Selected metrics are represented in RDF and semantically aligned to existing standards. These metrics are then implemented in an open-source tool. To demonstrate the tool, eight commonly used RDF resources were assessed, including data models in the healthcare domain (HL7 RIM, HL7 FHIR, CDISC CDASH), ontologies (DCT, SIO, FOAF, ORDO), and a metadata profile (GRDDL).ResultsSix objective metrics are identified in 3 categories: Resolvability (1), Parsability (1), and Consistency (4), and represented in RDF. The tool demonstrates that these metrics can be automated, and application in the healthcare domain shows non-resolvable URIs (ranging from 0.3% to 97%) among all eight resources and undefined URIs in HL7 RIM, and FHIR. In the tested resources no errors were found for parsability and the other three consistency metrics for correct usage of classes and properties.ConclusionWe extracted six objective and automatable metrics from literature, as the foundational quality requirements of RDF resources to maximize the potential of linked data. Automated tooling to assess resources has shown to be effective to identify quality issues that must be avoided. This approach can be expanded to incorporate more automatable metrics so as to reflect additional quality dimensions with the assessment tool implementing more metrics.
| S-EPMC10170671 | biostudies-literature
Project description:The habenula (Hb) is a small structure of the posterior diencephalon that is highly conserved across vertebrates but nonetheless has attracted relatively little research attention until the past two decades. The resurgent interest is motivated by neurobehavioral studies demonstrating critical functions in a broad spectrum of motivational and cognitive processes, including functions relevant to psychiatric diseases. The Hb is widely conceived as an "anti-reward" center that acts by regulating brain monoaminergic systems. However, there is still no general conceptual framework for habenula research, and no study has focused on uncovering potentially significant but overlooked topics that may advance our understanding of physiological functions or suggest potential clinical applications of Hb-targeted interventions. Using science mapping tools, we quantitatively and qualitatively analyzed the relevant publications retrieved from the Web of Science Core Collection (WoSCC) database from 2002 to 2021. Herein we present an overview of habenula-related publications, reveal primary research trends, and prioritize some key research fronts by complementary bibliometric analysis. High-priority research fronts include Ventral Pallidum, Nucleus Accumbens, Nicotine and MHb, GLT-1, Zebrafish, and GCaMP, Ketamine, Deep Brain Stimulation, and GPR139. The high intrinsic heterogeneity of the Hb, extensive connectivity with both hindbrain and forebrain structures, and emerging associations with all three dimensions of mental disorders (internalizing, externalizing, and psychosis) suggest that the Hb may be the neuronal substrate for a common psychopathology factor shared by all mental illnesses termed the p factor. A future challenge is to explore the therapeutic potential of habenular modulation at circuit, cellular, and molecular levels.
| S-EPMC9382245 | biostudies-literature