Quantifying the consistency of scientific databases.
Ontology highlight
ABSTRACT:
SUBMITTER: Subelj L
PROVIDER: S-EPMC4436008 | biostudies-literature | 2015
REPOSITORIES: biostudies-literature
Publications
Quantifying the consistency of scientific databases.
Šubelj Lovro L Bajec Marko M Boshkoska Biljana Mileva BM Kastrin Andrej A Levnajić Zoran Z
PloS one 20150518 5
Science is a social process with far-reaching impact on our modern society. In recent years, for the first time we are able to scientifically study the science itself. This is enabled by massive amounts of data on scientific publications that is increasingly becoming available. The data is contained in several databases such as Web of Science or PubMed, maintained by various public and private entities. Unfortunately, these databases are not always consistent, which considerably hinders this stu ...[more]
Similar Datasets
Project description:The Scientific Revolution represents a turning point in the history of humanity. Yet it remains ill-understood, partly because of a lack of quantification. Here, we leverage large datasets of individual biographies (N = 22,943) and present the first estimates of scientific production during the late medieval and early modern period (1300-1850). Our data reveal striking differences across countries, with England and the United Provinces being much more creative than other countries, suggesting that economic development has been key in generating the Scientific Revolution. In line with recent results in behavioural sciences, we show that scientific creativity and economic development are associated with other kinds of creative activities in philosophy, literature, music and the arts, as well as with inclusive institutions and ascetic religiosity, suggesting a common underlying mindset associated with long-term orientation and exploration. Finally, we investigate the interplay between economic development and cultural transmission (the so-called 'Republic of Letters') using partially observed Markov models imported from population biology. Surprisingly, the role of horizontal transmission (from one country to another) seems to have been marginal. Beyond the case of science, our results suggest that economic development is an important factor in the evolution of aspects of human culture.
| S-EPMC10426016 | biostudies-literature
Project description:Qualitative and quantitative methods are being developed to measure the impacts of research on society, but they suffer from serious drawbacks associated with linking a piece of research to its subsequent impacts. We have developed a method to derive impact scores for individual research publications according to their contribution to answering questions of quantified importance to end users of research. To demonstrate the approach, here we evaluate the impacts of research into means of conserving wild bee populations in the UK. For published papers, there is a weak positive correlation between our impact score and the impact factor of the journal. The process identifies publications that provide high quality evidence relating to issues of strong concern. It can also be used to set future research agendas.
| S-EPMC3217965 | biostudies-literature
Project description:Science advances by pushing the boundaries of the adjacent possible. While the global scientific enterprise grows at an exponential pace, at the mesoscopic level the exploration and exploitation of research ideas are reflected through the rise and fall of research fields. The empirical literature has largely studied such dynamics on a case-by-case basis, with a focus on explaining how and why communities of knowledge production evolve. Although fields rise and fall on different temporal and population scales, they are generally argued to pass through a common set of evolutionary stages. To understand the social processes that drive these stages beyond case studies, we need a way to quantify and compare different fields on the same terms. In this paper we develop techniques for identifying common patterns in the evolution of scientific fields and demonstrate their usefulness using 1.5 million preprints from the arXiv repository covering 175 research fields spanning Physics, Mathematics, Computer Science, Quantitative Biology and Quantitative Finance. We show that fields consistently follow a rise and fall pattern captured by a two parameters right-tailed Gumbel temporal distribution. We introduce a field-specific re-scaled time and explore the generic properties shared by articles and authors at the creation, adoption, peak, and decay evolutionary phases. We find that the early phase of a field is characterized by disruptive works mixing of cognitively distant fields written by small teams of interdisciplinary authors, while late phases exhibit the role of specialized, large teams building on the previous works in the field. This method provides foundations to quantitatively explore the generic patterns underlying the evolution of research fields in science, with general implications in innovation studies.
| S-EPMC9223313 | biostudies-literature
Project description:Bioinformatics sequence databases such as Genbank or UniProt contain hundreds of millions of records of genomic data. These records are derived from direct submissions from individual laboratories, as well as from bulk submissions from large-scale sequencing centres; their diversity and scale means that they suffer from a range of data quality issues including errors, discrepancies, redundancies, ambiguities, incompleteness and inconsistencies with the published literature. In this work, we seek to investigate and analyze the data quality of sequence databases from the perspective of a curator, who must detect anomalous and suspicious records. Specifically, we emphasize the detection of inconsistent records with respect to the literature. Focusing on GenBank, we propose a set of 24 quality indicators, which are based on treating a record as a query into the published literature, and then use query quality predictors. We then carry out an analysis that shows that the proposed quality indicators and the quality of the records have a mutual relationship, in which one depends on the other. We propose to represent record-literature consistency as a vector of these quality indicators. By reducing the dimensionality of this representation for visualization purposes using principal component analysis, we show that records which have been reported as inconsistent with the literature fall roughly in the same area, and therefore share similar characteristics. By manually analyzing records not previously known to be erroneous that fall in the same area than records know to be inconsistent, we show that one record out of four is inconsistent with respect to the literature. This high density of inconsistent record opens the way towards the development of automatic methods for the detection of faulty records. We conclude that literature inconsistency is a meaningful strategy for identifying suspicious records. Database URL:https://github.com/rbouadjenek/DQBioinformatics.
| S-EPMC5467556 | biostudies-literature
Project description:The degree of consistency with which groups of animals use the landscape is determined by a variety of ecological processes that influence their movements and patterns of habitat use. We developed a technique termed Distributional Consistency that uses survey data of unmarked individuals to quantify temporal consistency in their spatial distribution, while accounting for changes in population size. Distributional consistency is quantified by comparing the observed distribution patterns to all theoretically possible distribution patterns of observed individuals, leading to a proportional score between 0 and 1, reflecting increasingly consistent use of sites within a region. The technique can be applied to survey data for any taxa across a range of spatial and temporal scales. We suggest ways in which distributional consistency could provide inferences about the dispersal and habitat decisions of individuals, and the scales at which these decisions operate. Distributional consistency integrates spatial and temporal processes to quantify an important characteristic of different habitats and their use by populations, which in turn will be particularly useful in complimenting and interpreting other ecological measures such as population density and stability. The technique can be applied to many existing data sets to investigate and evaluate a range of important ecological questions using simple survey data.
| S-EPMC3434158 | biostudies-literature
Project description:Misspellings of organism scientific names create barriers to optimal storage and organization of biological data, reconciliation of data stored under different spelling variants of the same name, and appropriate responses from user queries to taxonomic data systems. This study presents an analysis of the nature of the problem from first principles, reviews some available algorithmic approaches, and describes Taxamatch, an improved name matching solution for this information domain. Taxamatch employs a custom Modified Damerau-Levenshtein Distance algorithm in tandem with a phonetic algorithm, together with a rule-based approach incorporating a suite of heuristic filters, to produce improved levels of recall, precision and execution time over the existing dynamic programming algorithms n-grams (as bigrams and trigrams) and standard edit distance. Although entirely phonetic methods are faster than Taxamatch, they are inferior in the area of recall since many real-world errors are non-phonetic in nature. Excellent performance of Taxamatch (as recall, precision and execution time) is demonstrated against a reference database of over 465,000 genus names and 1.6 million species names, as well as against a range of error types as present at both genus and species levels in three sets of sample data for species and four for genera alone. An ancillary authority matching component is included which can be used both for misspelled names and for otherwise matching names where the associated cited authorities are not identical.
| S-EPMC4172526 | biostudies-literature
Project description:Tie strength has been examined as an antecedent of creativity. Although it has been discovered that international collaboration affects scientific performance, the effect of tie strength in the international collaboration network has been largely neglected. Based on international publications of 72 countries/regions published from 1993 to 2013, we combine descriptive and panel regression methods to examine how the bonding of strong collaboration ties contributes to countries’ international scientific performance. Strong ties occur at an average rate of 1 in 4 collaborators, whereas countries/regions share on average 84% of articles with their strong-tie collaborators. Our quantitative results provide an explanation for this phenomenon in international collaboration: the establishment of a strong tie relationship contributes to above-average productivity and citation frequency for countries/regions. To further explore which types of strong ties tend to have stronger citation impact, we analyse the relationship between persistent and stable collaboration and publication citation impact. Experimental results show that international collaborations with greater persistence and moderate stability tend to produce high impact publications. It is noteworthy that when the collaboration period is divided into different time intervals, similar findings can be found after the same analysis procedure is carried out. This indicates that our conclusions are robust. Overall, this study provides quantitative insights into the added value of long-term commitment and social trust associated with strong collaborative partnerships in international collaboration.
| S-EPMC9844855 | biostudies-literature
Project description:Screening measures are used in psychology and medicine to identify respondents who are high or low on a construct. Based on the screening, the evaluator assigns respondents to classes corresponding to different courses of action: Make a diagnosis versus reject a diagnosis; provide services versus withhold services; or conduct further assessment versus conclude the assessment process. When measures are used to classify individuals, it is important that the decisions be consistent and equitable across groups. Ideally, if respondents completed the screening measure repeatedly in quick succession, they would be consistently assigned into the same class each time. In addition, the consistency of the classification should be unrelated to the respondents' background characteristics, such as sex, race, or ethnicity (i.e., the measure is free of measurement bias). Reporting estimates of classification consistency is a common practice in educational testing, but there has been limited application of these estimates to screening in psychology and medicine. In this article, we present two procedures based on item response theory that are used (a) to estimate the classification consistency of a screening measure and (b) to evaluate how classification consistency is impacted by measurement bias across respondent groups. We provide R functions to conduct the procedures, illustrate the procedures with real data, and use Monte Carlo simulations to guide their appropriate use. Finally, we discuss how estimates of classification consistency can help assessment specialists make more informed decisions on the use of a screening measure with protected groups (e.g., groups defined by gender, race, or ethnicity). (PsycInfo Database Record (c) 2021 APA, all rights reserved).
| S-EPMC8412438 | biostudies-literature
Project description:Scientists are frequently faced with the important decision to start or terminate a creative partnership. This process can be influenced by strategic motivations, as early career researchers are pursuers, whereas senior researchers are typically attractors, of new collaborative opportunities. Focusing on the longitudinal aspects of scientific collaboration, we analyzed 473 collaboration profiles using an egocentric perspective that accounts for researcher-specific characteristics and provides insight into a range of topics, from career achievement and sustainability to team dynamics and efficiency. From more than 166,000 collaboration records, we quantify the frequency distributions of collaboration duration and tie strength, showing that collaboration networks are dominated by weak ties characterized by high turnover rates. We use analytic extreme value thresholds to identify a new class of indispensable super ties, the strongest of which commonly exhibit >50% publication overlap with the central scientist. The prevalence of super ties suggests that they arise from career strategies based upon cost, risk, and reward sharing and complementary skill matching. We then use a combination of descriptive and panel regression methods to compare the subset of publications coauthored with a super tie to the subset without one, controlling for pertinent features such as career age, prestige, team size, and prior group experience. We find that super ties contribute to above-average productivity and a 17% citation increase per publication, thus identifying these partnerships--the analog of life partners--as a major factor in science career development.
| S-EPMC4553788 | biostudies-literature
Project description:BackgroundClinical genomic testing is dependent on the robust identification and reporting of variant-level information in relation to disease. With the shift to high-throughput sequencing, a major challenge for clinical diagnostics is the cross-identification of variants called on their genomic position to resources that rely on transcript- or protein-based descriptions.MethodsWe evaluated the accuracy of three tools (SnpEff, Variant Effect Predictor, and Variation Reporter) that generate transcript and protein-based variant nomenclature from genomic coordinates according to guidelines by the Human Genome Variation Society (HGVS). Our evaluation was based on transcript-controlled comparisons to a manually curated set of 126 test variants of various types drawn from data sources, each with HGVS-compliant transcript and protein descriptors. We further evaluated the concordance between annotations generated by Snpeff and Variant Effect Predictor and those in major germline and cancer databases: ClinVar and COSMIC, respectively.ResultsWe find that there is substantial discordance between the annotation tools and databases in the description of insertions and/or deletions. Using our ground truth set of variants, constructed specifically to identify challenging events, accuracy was between 80 and 90% for coding and 50 and 70% for protein changes for 114 to 126 variants. Exact concordance for SNV syntax was over 99.5% between ClinVar and Variant Effect Predictor and SnpEff, but less than 90% for non-SNV variants. For COSMIC, exact concordance for coding and protein SNVs was between 65 and 88% and less than 15% for insertions. Across the tools and datasets, there was a wide range of different but equivalent expressions describing protein variants.ConclusionsOur results reveal significant inconsistency in variant representation across tools and databases. While some of these syntax differences may be clear to a clinician, they can confound variant matching, an important step in variant classification. These results highlight the urgent need for the adoption and adherence to uniform standards in variant annotation, with consistent reporting on the genomic reference, to enable accurate and efficient data-driven clinical care.
| S-EPMC5267466 | biostudies-literature