The scientific publication score – a new tool for summarizing evidence and data quality criteria of biomedical publications
Ontology highlight
ABSTRACT:
SUBMITTER: Bettin D
PROVIDER: S-EPMC9832303 | biostudies-literature | 2022 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Preprints, versions of scientific manuscripts that precede peer review, are growing in popularity. They offer an opportunity to democratize and accelerate research, as they have no publication costs or a lengthy peer review process. Preprints are often later published in peer-reviewed venues, but these publications and the original preprints are frequently not linked in any way. To this end, we developed a tool, PreprintMatch, to find matches between preprints and their corresponding published papers, if they exist. This tool outperforms existing techniques to match preprints and papers, both on matching performance and speed. PreprintMatch was applied to search for matches between preprints (from bioRxiv and medRxiv), and PubMed. The preliminary nature of preprints offers a unique perspective into scientific projects at a relatively early stage, and with better matching between preprint and paper, we explored questions related to research inequity. We found that preprints from low income countries are published as peer-reviewed papers at a lower rate than high income countries (39.6% and 61.1%, respectively), and our data is consistent with previous work that cite a lack of resources, lack of stability, and policy choices to explain this discrepancy. Preprints from low income countries were also found to be published quicker (178 vs 203 days) and with less title, abstract, and author similarity to the published version compared to high income countries. Low income countries add more authors from the preprint to the published version than high income countries (0.42 authors vs 0.32, respectively), a practice that is significantly more frequent in China compared to similar countries. Finally, we find that some publishers publish work with authors from lower income countries more frequently than others.
| S-EPMC9994746 | biostudies-literature
Project description:ObjectiveValid, reliable critical appraisal tools advance quality improvement (QI) intervention impacts by helping stakeholders identify higher quality studies. QI approaches are diverse and differ from clinical interventions. Widely used critical appraisal instruments do not take unique QI features into account and existing QI tools (eg, Standards for QI Reporting Excellence) are intended for publication guidance rather than critical appraisal. This study developed and psychometrically tested a critical appraisal instrument, the QI Minimum Quality Criteria Set (QI-MQCS) for assessing QI-specific features of QI publications.MethodsApproaches to developing the tool and ensuring validity included a literature review, in-person and online survey expert panel input, and application to empirical examples. We investigated psychometric properties in a set of diverse QI publications (N=54) by analysing reliability measures and item endorsement rates and explored sources of disagreement between reviewers.ResultsThe QI-MQCS includes 16 content domains to evaluate QI intervention publications: Organisational Motivation, Intervention Rationale, Intervention Description, Organisational Characteristics, Implementation, Study Design, Comparator Description, Data Sources, Timing, Adherence/Fidelity, Health Outcomes, Organisational Readiness, Penetration/Reach, Sustainability, Spread and Limitations. Median inter-rater agreement for QI-MQCS items was κ 0.57 (83% agreement). Item statistics indicated sufficient ability to differentiate between publications (median quality criteria met 67%). Internal consistency measures indicated coherence without excessive conceptual overlap (absolute mean interitem correlation=0.19). The critical appraisal instrument is accompanied by a user manual detailing What to consider, Where to look and How to rate.ConclusionsWe developed a ready-to-use, valid and reliable critical appraisal instrument applicable to healthcare QI intervention publications, but recognise scope for continuing refinement.
| S-EPMC4680162 | biostudies-literature
Project description:Scientific publications enable results and ideas to be transmitted throughout the scientific community. The number and type of journal publications also have become the primary criteria used in evaluating career advancement. Our analysis suggests that publication practices have changed considerably in the life sciences over the past 30 years. More experimental data are now required for publication, and the average time required for graduate students to publish their first paper has increased and is approaching the desirable duration of PhD training. Because publication is generally a requirement for career progression, schemes to reduce the time of graduate student and postdoctoral training may be difficult to implement without also considering new mechanisms for accelerating communication of their work. The increasing time to publication also delays potential catalytic effects that ensue when many scientists have access to new information. The time has come for life scientists, funding agencies, and publishers to discuss how to communicate new findings in a way that best serves the interests of the public and the scientific community.
| S-EPMC4640799 | biostudies-other
Project description:PNAS is one of world's most cited multidisciplinary scientific journals. The PNAS official classification structure of subjects is reflected in topic labels submitted by the authors of articles, largely related to traditionally established disciplines. These include broad field classifications into physical sciences, biological sciences, social sciences, and further subtopic classifications within the fields. Focusing on biological sciences, we explore an internal soft-classification structure of articles based only on semantic decompositions of abstracts and bibliographies and compare it with the formal discipline classifications. Our model assumes that there is a fixed number of internal categories, each characterized by multinomial distributions over words (in abstracts) and references (in bibliographies). Soft classification for each article is based on proportions of the article's content coming from each category. We discuss the appropriateness of the model for the PNAS database as well as other features of the data relevant to soft classification.
| S-EPMC387299 | biostudies-literature
Project description:We propose a new method to assess the merit of any set of scientific papers in a given field based on the citations they receive. Given a field and a citation impact indicator, such as the mean citation or the [Formula: see text]-index, the merit of a given set of [Formula: see text] articles is identified with the probability that a randomly drawn set of [Formula: see text] articles from a given pool of articles in that field has a lower citation impact according to the indicator in question. The method allows for comparisons between sets of articles of different sizes and fields. Using a dataset acquired from Thomson Scientific that contains the articles published in the periodical literature in the period 1998-2007, we show that the novel approach yields rankings of research units different from those obtained by a direct application of the mean citation or the [Formula: see text]-index.
| S-EPMC3496740 | biostudies-literature
Project description:BACKGROUND:Social media coverage is increasingly used to spread the message of scientific publications. Traditionally, the scientific impact of an article is measured by the number of citations. At a journal level, this conventionally matures over a 2-year period, and it is challenging to gauge impact around the time of publication. OBJECTIVE:We, therefore, aimed to assess whether Web-based attention is associated with citations and to develop a predictive model that assigns relative importance to different elements of social media coverage: the #SoME_Impact score. METHODS:We included all original articles published in 2015 in a selection of the highest impact journals: The New England Journal of Medicine, The Lancet, the Journal of the American Medical Association, Nature, Cell, and Science. We first characterized the change in Altmetric score over time by taking a single month's sample of recently published articles from the same journals and gathered Altmetric data daily from the time of publication to create a mixed effects spline model. We then obtained the overall weighted Altmetric score for all articles from 2015, the unweighted data for each Altmetric component, and the 2-year citation count from Scopus for each of these articles from 2016 to 2017. We created a stepwise multivariable linear regression model to develop a #SoME_Score that was predictive of 2-year citations. The score was validated using a dataset of articles from the same journals published in 2016. RESULTS:In our unselected sample of 145 recently published articles, social media coverage appeared to plateau approximately 14 days after publication. A total of 3150 articles with a median citation count of 16 (IQR 5-33) and Altmetric score of 72 (IQR 28-169) were included for analysis. On multivariable regression, compared with articles in the lowest quantile of #SoME_Score, articles in the second, third, and upper quantiles had 0.81, 15.20, and 87.67 more citations, respectively. On the validation dataset, #SoME_Score model outperformed the Altmetric score (adjusted R2 0.19 vs 0.09; P<.001). Articles in the upper quantile of #SoME_Score were more than 5 times more likely to be among the upper quantile of those cites (odds ratio 5.61, 95% CI 4.70-6.73). CONCLUSIONS:Social media attention predicts citations and could be used as an early surrogate measure of scientific impact. Owing to the cross-sectional study design, we cannot determine whether correlation relates to causation.
| S-EPMC7195668 | biostudies-literature
Project description:Nucleotide sequence reagents are verifiable experimental reagents in biomedical publications, because their sequence identities can be independently verified and compared with associated text descriptors. We have previously reported that incorrectly identified nucleotide sequence reagents are characteristic of highly similar human gene knockdown studies, some of which have been retracted from the literature on account of possible research fraud. Because of the throughput limitations of manual verification of nucleotide sequences, we developed a semi-automated fact checking tool, Seek & Blastn, to verify the targeting or non-targeting status of published nucleotide sequence reagents. From previously described and unknown corpora of 48 and 155 publications, respectively, Seek & Blastn correctly extracted 304/342 (88.9%) and 1066/1522 (70.0%) nucleotide sequences and a predicted targeting/ non-targeting status. Seek & Blastn correctly predicted the targeting/ non-targeting status of 293/304 (96.4%) and 988/1066 (92.7%) of the correctly extracted nucleotide sequences. A total of 38/39 (97.4%) or 31/79 (39.2%) Seek & Blastn predictions of incorrect nucleotide sequence reagent use were correct in the two literature corpora. Combined Seek & Blastn and manual analyses identified a list of 91 misidentified nucleotide sequence reagents, which could be built upon through future studies. In summary, incorrect nucleotide sequence reagents represent an under-recognized source of error within the biomedical literature, and fact checking tools such as Seek & Blastn may help to identify papers and manuscripts affected by these errors.
| S-EPMC6396917 | biostudies-literature
Project description:Publications in scientific journals contain a considerable fraction of our scientific knowledge. Analyzing data from publication databases helps us understand how this knowledge is obtained and how it changes over time. In this study, we present a mathematical model for the temporal dynamics of data on the scientific content of publications. Our data set consists of references to thousands of genes in the >15 million publications listed in PubMed. We show that the observed dynamics may result from a simple process: Researchers predominantly publish on genes that already appear in many publications. This might be a rewarding strategy for researchers, because there is a positive correlation between the frequency of a gene in scientific publications and the journal impact of the publications. By comparing the empirical data with model predictions, we are able to detect unusual publication patterns that often correspond to major achievements in the field. We identify interactions between yeast genes from PubMed and show that the frequency differences of genes in publications lead to a biased picture of the resulting interaction network.
| S-EPMC1924584 | biostudies-literature
Project description:Ranking the significance of scientific publications is a long-standing challenge. The network-based analysis is a natural and common approach for evaluating the scientific credit of papers. Although the number of citations has been widely used as a metric to rank papers, recently some iterative processes such as the well-known PageRank algorithm have been applied to the citation networks to address this problem. In this paper, we introduce nonlinearity to the PageRank algorithm when aggregating resources from different nodes to further enhance the effect of important papers. The validation of our method is performed on the data of American Physical Society (APS) journals. The results indicate that the nonlinearity improves the performance of the PageRank algorithm in terms of ranking effectiveness, as well as robustness against malicious manipulations. Although the nonlinearity analysis is based on the PageRank algorithm, it can be easily extended to other iterative ranking algorithms and similar improvements are expected.
| S-EPMC4200399 | biostudies-literature
Project description: Not available
| S-EPMC6119233 | biostudies-literature