Dataset Information

Irreproducibility in searches of scientific literature: A comparative analysis

ABSTRACT: Abstract Repeatability is the cornerstone of science, and it is particularly important for systematic reviews. However, little is known on how researchers’ choice of database, and search platform influence the repeatability of systematic reviews. Here, we aim to unveil how the computer environment and the location where the search was initiated from influence hit results. We present a comparative analysis of time‐synchronized searches at different institutional locations in the world and evaluate the consistency of hits obtained within each of the search terms using different search platforms. We revealed a large variation among search platforms and showed that PubMed and Scopus returned consistent results to identical search strings from different locations. Google Scholar and Web of Science's Core Collection varied substantially both in the number of returned hits and in the list of individual articles depending on the search location and computing environment. Inconsistency in Web of Science results has most likely emerged from the different licensing packages at different institutions. To maintain scientific integrity and consistency, especially in systematic reviews, action is needed from both the scientific community and scientific search platforms to increase search consistency. Researchers are encouraged to report the search location and the databases used for systematic reviews, and database providers should make search algorithms transparent and revise access rules to titles behind paywalls. Additional options for increasing the repeatability and transparency of systematic reviews are storing both search metadata and hit results in open repositories and using Application Programming Interfaces (APIs) to retrieve standardized, machine‐readable search metadata. The consistency level with which scientific search platforms return hits for the same search initiated at different locations and from different computing environments was investigated. Scopus and PubMed returned the same results regardless of the location or computing environment but Web of Science and Google Scholar varied substantially. Since this is a crucial issue for the repeatability of systematic reviews, action should be taken to improve the within‐database consistency in the returned search hits.

SUBMITTER: Pozsgai G

PROVIDER: S-EPMC8571571 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionPopulation surveillance provides data on the health status of the population through continuous scrutiny of different indicators. Identifying risk factors is essential for the quickly detecting and controlling of epidemic outbreaks and reducing the incidence of cross-infections and non-communicable diseases. The objective of the present study is to analyze research on population surveillance, identifying the main topics of interest for investigators in the area.MethodologyWe included documents indexed in the Web of Science Core Collection in the period from 2000 to 2019 and assigned with the generic Medical Subject Heading (MeSH) "population surveillance" or its related terms ("public health surveillance," "sentinel surveillance" or "biosurveillance"). A co-occurrence analysis was undertaken to identify the document clusters comprising the main research topics. Scientific production, collaboration, and citation patterns in each of the clusters were characterized bibliometrically. We also analyzed research on coronaviruses, relating the results obtained to the management of the COVID-19 pandemic.ResultsWe included 39,184 documents, which reflected a steady growth in scientific output driven by papers on "Public, Environmental & Occupational Health" (21.62% of the documents) and "Infectious Diseases" (10.49%). Research activity was concentrated in North America (36.41%) and Europe (32.09%). The USA led research in the area (40.14% of documents). Ten topic clusters were identified, including "Disease Outbreaks," which is closely related to two other clusters ("Genetics" and "Influenza"). Other clusters of note were "Cross Infections" as well as one that brought together general public health concepts and topics related to non-communicable diseases (cardiovascular and coronary diseases, mental diseases, diabetes, wound and injuries, stroke, and asthma). The rest of the clusters addressed "Neoplasms," "HIV," "Pregnancy," "Substance Abuse/Obesity," and "Tuberculosis." Although research on coronavirus has focused on population surveillance only occasionally, some papers have analyzed and collated guidelines whose relevance to the dissemination and management of the COVID-19 pandemic has become obvious. Topics include tracing the spread of the virus, limiting mass gatherings that would facilitate its propagation, and the imposition of quarantines. There were important differences in the scientific production and citation of different clusters: the documents on mental illnesses, stroke, substance abuse/obesity, and cross-infections had much higher citations than the clusters on disease outbreaks, tuberculosis, and especially coronavirus, where these values are substantially lower.ConclusionsThe role of population surveillance should be strengthened, promoting research and the development of public health surveillance systems in countries whose contribution to the area is limited.

Project description:Background and purposeBreast cancer is the most common malignancy in women and radiation therapy (RT) is crucial in its multimodality management. Since bibliometrics is a powerful tool to reveal the scientific literature, we decided to perform a bibliometric analysis of the literature on breast cancer radiotherapy. We explored emerging trends and common patterns in research, tracking collaboration and networks, and foreseeing future directions in this clinical setting.Material and methodsThe electronic Scopus database was searched using the keywords "breast cancer" and "radiotherapy" to include manuscripts published in English, between 2000 and 2021. Data analysis was performed using R-Studio 0.98.1091 software with a machine-learning bibliometric method, based on the bibliometrix R package. The most relevant authors were quantified per number and fractionalized number of authored documents. Author productivity was analysed through Lotka's law. Bradford's law was applied to identify the nucleus of journals focused on the addressed topic. Mainstream themes area included isolated topics (niche themes), new topics (emerging themes), hot topics (motor themes) and essential topics (basic themes).ResultsA total of 27 184 documents was found, mainly original articles (76 %). The annual growth rate was 6.98 %, with an increase in scientific production from 485 to 2000 documents between 2000 and 2021. Overall, 2 544 journals published ≥ 1 documents. The most relevant authors were affiliated in the United States. Surgical procedures, cancer type and treatment strategies represented basic themes, while primary systemic therapy and sentinel lymph node biopsy were emerging themes. Health-related quality of life was a niche theme, while RT techniques had high centrality.ConclusionThe primary interests of breast cancer radiation oncologists have evolved over time, adding safety, health related quality of life, sustainability of treatments and combination to systemic therapies to radiotherapy efficacy and effectiveness and treatment outcomes.

Project description:ObjectivesThe rapid production of a large volume of literature during the early phase of the COVID-19 outbreak created a substantial burden for clinicians and scientists. Therefore, this manuscript aims to identify and describe the scientific literature addressing COVID-19 from a dental research perspective, in terms of the manuscript origin, research domain, study type, and level of evidence (LoE).Materials and methodsData were retrieved from Web of Science, Scopus, and PubMed. A descriptive analysis of bibliographic data, collaboration network, and keyword co-occurrence analysis were performed. Articles were further classified according to the field of interest, main research question, type of study, and LoE.ResultsThe present study identified 296 dental scientific COVID-19 original papers, published in 89 journals, and co-authored by 1331 individuals affiliated with 429 institutions from 53 countries. Although 81.4% were single-country papers, extensive collaboration among the institutions of single countries (Italian, British, and Brazilian institutions) was observed. The main research areas were as follows: the potential use of saliva and other oral fluids as promising samples for COVID-19 testing, dental education, and guidelines for the prevention of COVID-19 transmission in dental practice. The majority of articles were narrative reviews, cross-sectional studies, and short communications. The overall LoE in the analyzed dental literature was low, with only two systematic reviews with the highest LoE I.ConclusionThe dental literature on the COVID-19 pandemic does not provide data relevant to the evidence-based decision-making process. Future studies with a high LoE are essential to gain precise knowledge on COVID-19 infection within the various fields of Dentistry.Clinical relevanceThe published dental literature on COVID-19 consists principally of articles with a low level of scientific evidence which do not provide sufficient reliable high-quality evidence that is essential for decision making in clinical dental practice.

Dataset Information

Irreproducibility in searches of scientific literature: A comparative analysis

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets