Project description:BackgroundThe COVID-19 pandemic has severely affected health systems and medical research worldwide but its impact on the global publication dynamics and non-COVID-19 research has not been measured. We hypothesized that the COVID-19 pandemic may have impacted the scientific production of non-COVID-19 research.MethodsWe conducted a comprehensive meta-research on studies (original articles, research letters and case reports) published between 01/01/2019 and 01/01/2021 in 10 high-impact medical and infectious disease journals (New England Journal of Medicine, Lancet, Journal of the American Medical Association, Nature Medicine, British Medical Journal, Annals of Internal Medicine, Lancet Global Health, Lancet Public Health, Lancet Infectious Disease and Clinical Infectious Disease). For each publication, we recorded publication date, publication type, number of authors, whether the publication was related to COVID-19, whether the publication was based on a case series, and the number of patients included in the study if the publication was based on a case report or a case series. We estimated the publication dynamics with a locally estimated scatterplot smoothing method. A Natural Language Processing algorithm was designed to calculate the number of authors for each publication. We simulated the number of non-COVID-19 studies that could have been published during the pandemic by extrapolating the publication dynamics of 2019 to 2020, and comparing the expected number to the observed number of studies.ResultsAmong the 22,525 studies assessed, 6319 met the inclusion criteria, of which 1022 (16.2%) were related to COVID-19 research. A dramatic increase in the number of publications in general journals was observed from February to April 2020 from a weekly median number of publications of 4.0 (IQR: 2.8-5.5) to 19.5 (IQR: 15.8-24.8) (p < 0.001), followed afterwards by a pattern of stability with a weekly median number of publications of 10.0 (IQR: 6.0-14.0) until December 2020 (p = 0.045 in comparison with April). Two prototypical editorial strategies were found: 1) journals that maintained the volume of non-COVID-19 publications while integrating COVID-19 research and thus increased their overall scientific production, and 2) journals that decreased the volume of non-COVID-19 publications while integrating COVID-19 publications. We estimated using simulation models that the COVID pandemic was associated with a 18% decrease in the production of non-COVID-19 research. We also found a significant change of the publication type in COVID-19 research as compared with non-COVID-19 research illustrated by a decrease in the number of original articles, (47.9% in COVID-19 publications vs 71.3% in non-COVID-19 publications, p < 0.001). Last, COVID-19 publications showed a higher number of authors, especially for case reports with a median of 9.0 authors (IQR: 6.0-13.0) in COVID-19 publications, compared to a median of 4.0 authors (IQR: 3.0-6.0) in non-COVID-19 publications (p < 0.001).ConclusionIn this meta-research gathering publications from high-impact medical journals, we have shown that the dramatic rise in COVID-19 publications was accompanied by a substantial decrease of non-COVID-19 research. META-RESEARCH REGISTRATION: https://osf.io/9vtzp/ .
Project description:Journal impact factors have become an important criterion to judge the quality of scientific publications over the years, influencing the evaluation of institutions and individual researchers worldwide. However, they are also subject to a number of criticisms. Here we point out that the calculation of a journal's impact factor is mainly based on the date of publication of its articles in print form, despite the fact that most journals now make their articles available online before that date. We analyze 61 neuroscience journals and show that delays between online and print publication of articles increased steadily over the last decade. Importantly, such a practice varies widely among journals, as some of them have no delays, while for others this period is longer than a year. Using a modified impact factor based on online rather than print publication dates, we demonstrate that online-to-print delays can artificially raise a journal's impact factor, and that this inflation is greater for longer publication lags. We also show that correcting the effect of publication delay on impact factors changes journal rankings based on this metric. We thus suggest that indexing of articles in citation databases and calculation of citation metrics should be based on the date of an article's online appearance, rather than on that of its publication in print.
Project description:During crises such as the present coronavirus disease-19 (COVID-19) pandemic, nonprofits play a key role in ensuring support to improve the most vulnerable individuals' health, social, and economic conditions. One year into the COVID-19 pandemic, an extensive automated literature analysis was conducted of 154 academic articles on nonprofit management during the pandemic-all of which were published in 2020. This study sought to identify and systematize academics' contributions to knowledge about the crisis's impact on the nonprofit sector and to ascertain the most urgent directions for future research. The results provide policymakers, nonprofit practitioners, and scholars an overview of the themes addressed and highlight the important assistance academic researchers provide to nonprofits dealing with the COVID-19 pandemic.Supplementary informationThe online version contains supplementary material available at 10.1007/s11266-021-00432-9.
Project description:BACKGROUND:Publication and related biases (including publication bias, time-lag bias, outcome reporting bias and p-hacking) have been well documented in clinical research, but relatively little is known about their presence and extent in health services research (HSR). This paper aims to systematically review evidence concerning publication and related bias in quantitative HSR. METHODS:Databases including MEDLINE, EMBASE, HMIC, CINAHL, Web of Science, Health Systems Evidence, Cochrane EPOC Review Group and several websites were searched to July 2018. Information was obtained from: (1) Methodological studies that set out to investigate publication and related biases in HSR; (2) Systematic reviews of HSR topics which examined such biases as part of the review process. Relevant information was extracted from included studies by one reviewer and checked by another. Studies were appraised according to commonly accepted scientific principles due to lack of suitable checklists. Data were synthesised narratively. RESULTS:After screening 6155 citations, four methodological studies investigating publication bias in HSR and 184 systematic reviews of HSR topics (including three comparing published with unpublished evidence) were examined. Evidence suggestive of publication bias was reported in some of the methodological studies, but evidence presented was very weak, limited in both quality and scope. Reliable data on outcome reporting bias and p-hacking were scant. HSR systematic reviews in which published literature was compared with unpublished evidence found significant differences in the estimated intervention effects or association in some but not all cases. CONCLUSIONS:Methodological research on publication and related biases in HSR is sparse. Evidence from available literature suggests that such biases may exist in HSR but their scale and impact are difficult to estimate for various reasons discussed in this paper. SYSTEMATIC REVIEW REGISTRATION:PROSPERO 2016 CRD42016052333.
Project description:The impact of COVID-19 has underlined the need for reliable information to guide clinical practice and policy. This urgency has to be balanced against disruption to journal handling capacity and the continued need to ensure scientific rigour. We examined the reporting quality of highly disseminated COVID-19 research papers using a bibliometric analysis examining reporting quality and risk of bias (RoB) amongst 250 top scoring Altmetric Attention Score (AAS) COVID-19 research papers between January and April 2020. Method-specific RoB tools were used to assess quality. After exclusions, 84 studies from 44 journals were included. Forty-three (51%) were case series/studies, and only one was an randomized controlled trial. Most authors were from institutions based in China (n = 44, 52%). The median AAS and impact factor was 2015 (interquartile range [IQR] 1,105-4,051.5) and 12.8 (IQR 5-44.2) respectively. Nine studies (11%) utilized a formal reporting framework, 62 (74%) included a funding statement, and 41 (49%) were at high RoB. This review of the most widely disseminated COVID-19 studies highlights a preponderance of low-quality case series with few research papers adhering to good standards of reporting. It emphasizes the need for cautious interpretation of research and the increasingly vital responsibility that journals have in ensuring high-quality publications.
Project description:BackgroundSince the start of the COVID-19 outbreak, a large number of COVID-19-related papers have been published. However, concerns about the risk of expedited science have been raised. We aimed at reviewing and categorizing COVID-19-related medical research and to critically appraise peer-reviewed original articles.MethodsThe data sources were Pubmed, Cochrane COVID-19 register study, arXiv, medRxiv and bioRxiv, from 01/11/2019 to 01/05/2020. Peer-reviewed and preprints publications related to COVID-19 were included, written in English or Chinese. No limitations were placed on study design. Reviewers screened and categorized studies according to i) publication type, ii) country of publication, and iii) topics covered. Original articles were critically appraised using validated quality assessment tools.ResultsAmong the 11,452 publications identified, 10,516 met the inclusion criteria, among which 7468 (71.0%) were peer-reviewed articles. Among these, 4190 publications (56.1%) did not include any data or analytics (comprising expert opinion pieces). Overall, the most represented topics were infectious disease (n = 2326, 22.1%), epidemiology (n = 1802, 17.1%), and global health (n = 1602, 15.2%). The top five publishing countries were China (25.8%), United States (22.3%), United Kingdom (8.8%), Italy (8.1%) and India (3.4%). The dynamic of publication showed that the exponential growth of COVID-19 peer-reviewed articles was mainly driven by publications without original data (mean 261.5 articles ± 51.1 per week) as compared with original articles (mean of 69.3 ± 22.3 articles per week). Original articles including patient data accounted for 713 (9.5%) of peer-reviewed studies. A total of 576 original articles (80.8%) showed intermediate to high risk of bias. Last, except for simulation studies that mainly used large-scale open data, the median number of patients enrolled was of 102 (IQR = 37-337).ConclusionsSince the beginning of the COVID-19 pandemic, the majority of research is composed by publications without original data. Peer-reviewed original articles with data showed a high risk of bias and included a limited number of patients. Together, these findings underscore the urgent need to strike a balance between the velocity and quality of research, and to cautiously consider medical information and clinical applicability in a pressing, pandemic context. SYSTEMATIC REVIEW REGISTRATION: https://osf.io/5zjyx/.
Project description:BackgroundSystematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular.Methods/designThe objectives of this systematic review are as follows: •To investigate the epidemiology of published systematic reviews of animal studies until present. •To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias. •To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies. Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior. In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions. Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews.DiscussionResults are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.
Project description:BackgroundThe COVID-19 pandemic has highlighted the importance of rapid dissemination of scientific and medical discoveries. Current platforms available for the distribution of scientific and clinical research data and information include preprint repositories and traditional peer-reviewed journals. In recent times, social media has emerged as a helpful platform to share scientific and medical discoveries.ObjectiveThis study aimed to comparatively analyze activity on social media (specifically, Twitter) and that related to publications in the form of preprint and peer-reviewed journal articles in the context of COVID-19 and gastroenterology during the early stages of the COVID-19 pandemic.MethodsCOVID-19-related data from Twitter (tweets and user data) and articles published in preprint servers (bioRxiv and medRxiv) as well as in the PubMed database were collected and analyzed during the first 6 months of the pandemic, from December 2019 through May 2020. Global and regional geographic and gastrointestinal organ-specific social media trends were compared to preprint and publication activity. Any relationship between Twitter activity and preprint articles published and that between Twitter activity and PubMed articles published overall, by organ system, and by geographic location were identified using Spearman's rank-order correlation.ResultsOver the 6-month period, 73,079 tweets from 44,609 users, 7164 journal publications, and 4702 preprint publications were retrieved. Twitter activity (ie, number of tweets) peaked in March 2020, whereas preprint and publication activity (ie, number of articles published) peaked in April 2020. Overall, strong correlations were identified between trends in Twitter activity and preprint and publication activity (P<.001 for both). COVID-19 data across the three platforms mainly concentrated on pulmonology or critical care, but when analyzing the field of gastroenterology specifically, most tweets pertained to pancreatology, most publications focused on hepatology, and most preprints covered hepatology and luminal gastroenterology. Furthermore, there were significant positive associations between trends in Twitter and publication activity for all gastroenterology topics (luminal gastroenterology: P=.009; hepatology and inflammatory bowel disease: P=.006; gastrointestinal endoscopy: P=.007), except pancreatology (P=.20), suggesting that Twitter activity did not correlate with publication activity for this topic. Finally, Twitter activity was the highest in the United States (7331 tweets), whereas PubMed activity was the highest in China (1768 publications).ConclusionsThe COVID-19 pandemic has highlighted the potential of social media as a vehicle for disseminating scientific information during a public health crisis. Sharing and spreading information on COVID-19 in a timely manner during the pandemic has been paramount; this was achieved at a much faster pace on social media, particularly on Twitter. Future investigation could demonstrate how social media can be used to augment and promote scholarly activity, especially as the world begins to increasingly rely on digital or virtual platforms. Scientists and clinicians should consider the use of social media in augmenting public awareness regarding their scholarly pursuits.
Project description:The COVID-19 pandemic has highlighted the importance of rapid dissemination of scientific and medical discovery. Social media (SoMe) has become an invaluable platform in science and medicine. This study analyzed activity of SoMe (Twitter), preprints, and publications related to COVID-19 and gastroenterology (GI) during the COVID-19 pandemic. Data from Twitter, preprint servers and PubMed was collected and analyzed from December 2019 through May 2020. Global and regional geographic and gastrointestinal organ specific social media trends were compared to preprint and publication activity; any associations were identified. Over the 6-month period, there were 73,079 tweets from 44,609 users, 7,164 publications, and 4,702 preprints. Twitter activity peaked during March while preprints and publications peaked in April 2020. Strong correlations were identified between Twitter and both preprints and publications activity (p<0.001 for both). While COVID-19 data across the 3 platforms concentrated on pulmonology/critical care, the majority of GI tweets pertained to pancreatology, most publications focused on hepatology, and most preprints covered hepatology and luminal GI (LGI). There were significant associations between Twitter activity and research for all GI subfields (p=0.009 for LGI, p=0.006 for hepatology and IBD, p=0.007 for endoscopy), except pancreatology (p=0.2). Twitter activity was highest in the US (7,331 tweets) whereas PubMed activity was highest in China (1,768 publications). The COVID-19 pandemic has highlighted the utility of SoMe as a vehicle for disseminating scientific information during a public health crisis. Scientists and clinicians should consider the use of SoMe in augmenting public awareness of their scholarly pursuits.
Project description:ObjectiveTo assess the time to publication of primary research and evidence syntheses funded by the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme published as a monograph in Health Technology Assessment and as a journal article in the wider biomedical literature.Study designRetrospective cohort study.SettingPrimary research and evidence synthesis projects funded by the HTA Programme were included in the cohort if they were registered in the NIHR research programmes database and was planned to submit the draft final report for publication in Health Technology Assessment on or before 9 December 2011.Main outcome measuresThe median time to publication and publication at 30 months in Health Technology Assessment and in an external journal were determined by searching the NIHR research programmes database and HTA Programme website.ResultsOf 458 included projects, 184 (40.2%) were primary research projects and 274 (59.8%) were evidence syntheses. A total of 155 primary research projects had a completion date; the median time to publication was 23 months (26.5 and 35.5 months to publish a monograph and to publish in an external journal, respectively) and 69% were published within 30 months. The median time to publication of HTA-funded trials (n=126) was 24 months and 67.5% were published within 30 months. Among the evidence syntheses with a protocol online date (n=223), the median time to publication was 25.5 months (28 months to publication as a monograph), but only 44.4% of evidence synthesis projects were published in an external journal. 65% of evidence synthesis studies had been published within 30.0 months.ConclusionsResearch funded by the HTA Programme publishes promptly. The importance of Health Technology Assessment was highlighted as the median time to publication was 9 months shorter for a monograph than an external journal article.