Dataset Information

Scientific quality of COVID-19 and SARS CoV-2 publications in the highest impact medical journals during the early phase of the pandemic: A case control study.

ABSTRACT:

Background

A debate about the scientific quality of COVID-19 themed research has emerged. We explored whether the quality of evidence of COVID-19 publications is lower when compared to nonCOVID-19 publications in the three highest ranked scientific medical journals.

Methods

We searched the PubMed Database from March 12 to April 12, 2020 and identified 559 publications in the New England Journal of Medicine, the Journal of the American Medical Association, and The Lancet which were divided into COVID-19 (cases, n = 204) and nonCOVID-19 (controls, n = 355) associated content. After exclusion of secondary, unauthored, response letters and non-matching article types, 155 COVID-19 publications (including 13 original articles) and 130 nonCOVID-19 publications (including 52 original articles) were included in the comparative analysis. The hierarchical level of evidence was determined for each publication included and compared between cases and controls as the main outcome. A quantitative scoring of quality was carried out for the subgroup of original articles. The numbers of authors and citation rates were also compared between groups.

Results

The 130 nonCOVID-19 publications were associated with higher levels of evidence on the level of evidence pyramid, with a strong association measure (Cramer's V: 0.452, P <0.001). The 155 COVID-19 publications were 186-fold more likely to be of lower evidence (95% confidence interval [CI] for odds ratio, 7.0-47; P <0.001). The quantitative quality score (maximum possible score, 28) was significantly different in favor of nonCOVID-19 (mean difference, 11.1; 95% CI, 8.5-13.7; P <0.001). There was a significant difference in the early citation rate of the original articles that favored the COVID-19 original articles (median [interquartile range], 45 [30-244] vs. 2 [1-4] citations; P <0.001).

Conclusions

We conclude that the quality of COVID-19 publications in the three highest ranked scientific medical journals is below the quality average of these journals. These findings need to be verified at a later stage of the pandemic.

SUBMITTER: Zdravkovic M

PROVIDER: S-EPMC7643945 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe COVID-19 pandemic has spread at an alarming speed, and effective treatment for the disease is still lacking. The body of evidence on COVID-19 has been increasing at an impressive pace, creating the need for a method to rapidly assess the current knowledge and identify key information. Gold standard methods such as systematic reviews and meta-analyses are regarded unsuitable because they have a narrow scope and are very time consuming.ObjectiveThis study aimed to explore the published scientific literature on COVID-19 and map the research evolution during the early phase of the COVID-19 pandemic.MethodsWe performed a PubMed search to analyze the titles, keywords, and abstracts of published papers on COVID-19. We used latent Dirichlet allocation modeling to extract topics and conducted a trend analysis to understand the temporal changes in research for each topic, journal impact factor (JIF), and geographic origin.ResultsBased on our search, we identified 16,670 relevant articles dated between February 14, 2020, and June 1, 2020. Of these, 6 articles were reports from peer-reviewed randomized trials on patients with COVID-19. We identified 14 main research topics, of which the most common topics were health care responses (2812/16,670, 16.86%) and clinical manifestations (1828/16,670, 10.91%). We found an increasing trend for research on clinical manifestations and protective measures and a decreasing trend for research on disease transmission, epidemiology, health care response, and radiology. Publications on protective measures, immunology, and clinical manifestations were associated with the highest JIF. The overall median JIF was 3.7 (IQR 2.6-5.9), and we found that the JIF for these publications declined over time. The top countries producing research were the United States, China, Italy, and the United Kingdom.ConclusionsIn less than 6 months since the novel coronavirus was first detected, a remarkably high number of research articles on COVID-19 have been published. Here, we discuss and present the temporal changes in the available COVID-19 research during the early phase of the pandemic. Our findings may aid researchers and policy makers to form a structured view of the current COVID-19 evidence base and provide further research directions.

Project description:The author apply concepts and tools from evolutionary medicine to understanding the SARS-CoV-2 pandemic. The pandemic represents a mismatched conflict, with dynamics and pathology apparently driven by three main factors: (i) bat immune systems that rely on low inflammation but high efficacy of interferon-based defenses; (ii) viral tactics that differentially target the human interferon system, leading to substantial asymptomatic and pre-symptomatic transmission; and (ii) high mortality caused by hyper-inflammatory and hyper-coagulatory phenotypes, that represent dysregulated tradeoffs whereby collateral immune-induced damage becomes systemic and severe. This framework can explain the association of mortality with age (which involves immune life-history shifts towards higher inflammation and coagulation and reduced adaptive immunity), and sex (since males senesce faster than females). Genetic-risk factors for COVID-19 mortality can be shown, from a phenome-wide association analysis of the relevant SNPs, to be associated with inflammation and coagulation; the phenome-wide association study also provides evidence, consistent with several previous studies, that the calcium channel blocking drug amlodipine mediates risk of mortality. Lay Summary: SARS-CoV-2 is a bat virus that jumped into humans. The virus is adapted to bat immune systems, where it evolved to suppress the immune defenses (interferons) that mammals use to tell that they are infected. In humans, the virus can apparently spread effectively in the body with a delay in the production of symptoms and the initiation of immune responses. This delay may then promote overactive immune responses, when the virus is detected, that damage the body as a side effect. Older people are more vulnerable to the virus because they are less adapted to novel infectious agents, and invest less in immune defense, compared to younger people. Genes that increase risk of mortality from SARS-CoV-2 are functionally associated with a drug called amlodipine, which may represent a useful treatment.

Project description:ObjectivesRheumatology-related diseases remain a significant burden worldwide. However, little is known about the comparative status of rheumatology research between Mainland China (MC) and the world's leading countries. The aim of this study is to compare the quantity and quality of research output in the field of rheumatology that were written by researchers from MC, the USA, the UK, the Netherlands and France.MethodsBetween 2007 and 2017, all articles published in 30 rheumatology journals were identified via Science Citation Index Expanded database. The number of total and annual articles, article types (randomized controlled trials (RCTs), reviews, case reports, clinical trials and meta-analysis), impact factor (IF), citations, h-index and articles in the high-impact journals were collected for quantity and quality comparisons. The correlation of socioeconomic factors and annual publications was also analyzed.ResultsFrom 2007 to 2017, there were 53,439 articles published in rheumatology journals, of which researchers from the USA published 13,391 articles, followed by the UK, the Netherlands, France and MC with 6,179, 4,310, 4,066 and 2,898 articles, respectively. Publications from MC represented the ninth, but the number is growing rapidly. For total and average citations, MC still lags behind the other four countries in the study. Similar trends were observed in average IF, h-index and articles in the high-impact journals. In terms of article types, the USA occupies the dominant place, except for meta-analysis. The annual numbers of articles from MC and the USA were positively correlated with gross domestic product (p < 0.05).ConclusionsThe USA has played predominant role in rheumatology research for the last 11 years. The annual number of published articles from MC has increased notably from 2007 to 2017. Although MC has made progress in the number of published articles over the past decade, it still lags far behind the highly developed countries in most bibliometric indicators. Thus, the general quality of publications from MC needs further improvement.

Project description:BackgroundNoninferiority trials are increasingly being performed. However, little is known about their methodological quality. We sought to characterize noninferiority cardiovascular trials published in the highest-impact journals, features that may bias results toward noninferiority, features related to reporting of noninferiority trials, and the time trends.MethodsWe identified cardiovascular noninferiority trials published in JAMA, Lancet, or New England Journal of Medicine from 1990 to 2016. Two independent reviewers extracted the data. Data elements included the noninferiority margin and the success of studies in achieving noninferiority. The proportion of trials showing major or minor features that may have affected the noninferiority inference was determined. Major factors included the lack of presenting the results in both intention-to-treat and per-protocol/as-treated cohorts, α>0.05, the new intervention not being compared with the best alternative, not justifying the noninferiority margin, and exclusion or loss of ≥10% of the cohort. Minor factors included suboptimal blinding, allocation concealment, and others.ResultsFrom 2544 screened studies, we identified 111 noninferiority cardiovascular trials. Noninferiority margins varied widely: risk differences of 0.4% to 25%, hazard ratios of 1.05 to 2.85, odds ratios of 1.1 to 2.0, and relative risks of 1.1 to 1.8. Eighty-six trials claimed noninferiority, of which 20 showed superiority, whereas 23 (21.1%) did not show noninferiority, of which 8 also demonstrated inferiority. Only 7 (6.3%) trials were considered low risk for all the major and minor biasing factors. Among common major factors for bias, 41 (37%) did not confirm the findings in both intention-to-treat and per-protocol/as-treated cohorts and 4 (3.6%) reported discrepant results between intention-to-treat and per-protocol analyses. Forty-three (38.7%) did not justify the noninferiority margin. Overall, 27 (24.3%) underenrolled or had >10% exclusions. Sixty trials (54.0%) were open label. Allocation concealment was not maintained or unclear in 11 (9.9%). Publication of noninferiority trials increased over time (P<0.001). Fifty-two (46.8%) were published after 2010 and had a lower risk of methodological or reporting limitations for major (P=0.03) and minor factors (P=0.002).ConclusionsNoninferiority trials in highest-impact journals commonly conclude noninferiority of the tested intervention, but vary markedly in the selected noninferiority margin, and frequently have limitations that may impact the inference related to noninferiority.