Dataset Information

A multivariate model for successful publication of intensive care medicine randomized controlled trials in the highest impact factor journals: the SCOTI score.

ABSTRACT:

Background

Critical care randomized controlled trials (RCTs) are often published in high-impact journals, whether general journals [the New England Journal of Medicine (NEJM), The Lancet, the Journal of the American Medical Association (JAMA)] or critical care journals [Intensive Care Medicine (ICM), the American Journal of Respiratory and Critical Care Medicine (AJRCCM), Critical Care Medicine (CCM)]. As rejection occurs in up to 97% of cases, it might be appropriate to assess pre-submission probability of being published. The objective of this study was to develop and internally validate a simplified score predicting whether an ongoing trial stands a chance of being published in high-impact general journals.

Methods

A cohort of critical care RCTs published between 1999 and 2018 in the three highest impact medical journals (NEJM, The Lancet, JAMA) or the three highest impact critical care journals (ICM, AJRCCM, CCM) was split into two samples (derivation cohort, validation cohort) to develop and internally validate the simplified score. Primary outcome was journal of publication assessed as high-impact general journal (NEJM, The Lancet, JAMA) or critical care journal (ICM, AJRCCM, CCM).

Results

A total of 968 critical care RCTs were included in the predictive cohort and split into a derivation cohort (n = 510) and a validation cohort (n = 458). In the derivation cohort, the sample size (P value < 0.001), the number of centers involved (P value = 0.01), mortality as primary outcome (P value = 0.002) or a composite item including mortality as primary outcome (P value = 0.004), and topic [ventilation (P value < 0.001) or miscellaneous (P value < 0.001)] were independent factors predictive of publication in high-impact general journals, compared to high-impact critical care journals. The SCOTI score (Sample size, Centers, Outcome, Topic, and International score) was developed with an area under the ROC curve of 0.84 (95% Confidence Interval, 0.80-0.88) in validation by split sample.

Conclusions

The SCOTI score, developed and validated by split sample, accurately predicts the chances of a critical care RCT being published in high-impact general journals, compared to high-impact critical care journals.

SUBMITTER: Pensier J

PROVIDER: S-EPMC8626742 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundNoninferiority trials are increasingly being performed. However, little is known about their methodological quality. We sought to characterize noninferiority cardiovascular trials published in the highest-impact journals, features that may bias results toward noninferiority, features related to reporting of noninferiority trials, and the time trends.MethodsWe identified cardiovascular noninferiority trials published in JAMA, Lancet, or New England Journal of Medicine from 1990 to 2016. Two independent reviewers extracted the data. Data elements included the noninferiority margin and the success of studies in achieving noninferiority. The proportion of trials showing major or minor features that may have affected the noninferiority inference was determined. Major factors included the lack of presenting the results in both intention-to-treat and per-protocol/as-treated cohorts, α>0.05, the new intervention not being compared with the best alternative, not justifying the noninferiority margin, and exclusion or loss of ≥10% of the cohort. Minor factors included suboptimal blinding, allocation concealment, and others.ResultsFrom 2544 screened studies, we identified 111 noninferiority cardiovascular trials. Noninferiority margins varied widely: risk differences of 0.4% to 25%, hazard ratios of 1.05 to 2.85, odds ratios of 1.1 to 2.0, and relative risks of 1.1 to 1.8. Eighty-six trials claimed noninferiority, of which 20 showed superiority, whereas 23 (21.1%) did not show noninferiority, of which 8 also demonstrated inferiority. Only 7 (6.3%) trials were considered low risk for all the major and minor biasing factors. Among common major factors for bias, 41 (37%) did not confirm the findings in both intention-to-treat and per-protocol/as-treated cohorts and 4 (3.6%) reported discrepant results between intention-to-treat and per-protocol analyses. Forty-three (38.7%) did not justify the noninferiority margin. Overall, 27 (24.3%) underenrolled or had >10% exclusions. Sixty trials (54.0%) were open label. Allocation concealment was not maintained or unclear in 11 (9.9%). Publication of noninferiority trials increased over time (P<0.001). Fifty-two (46.8%) were published after 2010 and had a lower risk of methodological or reporting limitations for major (P=0.03) and minor factors (P=0.002).ConclusionsNoninferiority trials in highest-impact journals commonly conclude noninferiority of the tested intervention, but vary markedly in the selected noninferiority margin, and frequently have limitations that may impact the inference related to noninferiority.

Project description:Background Well-conducted meta-analyses are considered to be at the top of the evidence-based hierarchy pyramid, with an expansion of these publications within the cardiovascular research arena. There are limited data evaluating the trends and quality of such publications. The objective of this study was to evaluate the methodological rigor and temporal trends of cardiovascular medicine-related meta-analyses published in the highest impact journals. Methods and Results Using the Medline database, we retrieved cardiovascular medicine-related systematic reviews and meta-analyses published in The New England Journal of Medicine, The Lancet, Journal of the American Medical Association, The British Medical Journal, Annals of Internal Medicine, Circulation, European Heart Journal, and Journal of American College of Cardiology between January 1, 2012 and December 31, 2018. Among 6406 original investigations published during the study period, meta-analyses represented 422 (6.6%) articles, with an annual decline in the proportion of published meta-analyses (8.7% in 2012 versus 4.6% in 2018, Ptrend=0.002). A substantial number of studies failed to incorporate elements of Preferred Reporting Items for Systematic Reviews and Meta-Analyses or Meta-Analysis of Observational Studies in Epidemiology guidelines (51.9%) and only a minority of studies (10.4%) were registered in PROSPERO (International Prospective Register of Systematic Reviews). Fewer manuscripts failed to incorporate the Preferred Reporting Items for Systematic Reviews and Meta-Analyses or Meta-Analysis of Observational Studies in Epidemiology elements over time (60.2% in 2012 versus 40.0% in 2018, Ptrend<0.001) whereas the number of meta-analyses registered at PROSPERO has increased (2.4% in 2013 versus 17.5% in 2018, Ptrend<0.001). Conclusions The proportion of cardiovascular medicine-related meta-analyses published in the highest impact journals has declined over time. Although there is an increasing trend in compliance with quality-based guidelines, the overall compliance remains low.

Project description:ContextOver the past three decades, industry sponsored research expanded in the United States. Financial incentives can lead to potential conflicts of interest (COI) resulting in underreporting of negative study results.ObjectiveWe hypothesized that over the three decades, there would be an increase in: a) reporting of conflict of interest and source of funding; b) percentage of randomized control trials c) number of patients per study and d) industry funding.Data sources and study selectionOriginal articles published in three calendar years (1988, 1998, and 2008) in The Lancet, New England Journal of Medicine and Journal of American Medical Association were collected.Data extractionStudies were reviewed and investigational design categorized as prospective and retrospective clinical trials. Prospective trials were categorized into randomized or non-randomized and single-center or multi-center trials. Retrospective trials were categorized as registries, meta-analyses and other studies, mostly comprising of case reports or series. Study outcomes were categorized as positive or negative depending on whether the pre-specified hypothesis was met. Financial disclosures were researched for financial relationships and profit status, and accordingly categorized as government, non-profit or industry sponsored. Studies were assessed for reporting COI.Results1,671 original articles were included in this analysis. Total number of published studies decreased by 17% from 1988 to 2008. Over 20 year period, the proportion of prospective randomized trials increased from 22 to 46% (p < 0.0001); whereas the proportion of prospective non-randomized trials decreased from 59% to 27% (p < 0.001). There was an increase in the percentage of prospective randomized multi-center trials from 11% to 41% (p < 0.001). Conversely, there was a reduction in non-randomized single-center trials from 47% to 10% (p < 0.001). Proportion of government funded studies remained constant, whereas industry funded studies more than doubled (17% to 40%; p < 0.0001). The number of studies with negative results more than doubled (10% to 22%; p<0.0001). While lack of funding disclosure decreased from 35% to 7%, COI reporting increased from 2% to 84% (p < 0.0001).ConclusionImproved reporting of COI, clarity in financial sponsorship, increased publication of negative results in the setting of larger and better designed clinical trials represents a positive step forward in the scientific publications, despite the higher percentage of industry funded studies.

Project description:The United States (US) Food and Drug Administration (FDA) approves new drugs based on sponsor-submitted clinical trials. The publication status of these trials in the medical literature and factors associated with publication have not been evaluated. We sought to determine the proportion of trials submitted to the FDA in support of newly approved drugs that are published in biomedical journals that a typical clinician, consumer, or policy maker living in the US would reasonably search.We conducted a cohort study of trials supporting new drugs approved between 1998 and 2000, as described in FDA medical and statistical review documents and the FDA approved drug label. We determined publication status and time from approval to full publication in the medical literature at 2 and 5 y by searching PubMed and other databases through 01 August 2006. We then evaluated trial characteristics associated with publication. We identified 909 trials supporting 90 approved drugs in the FDA reviews, of which 43% (394/909) were published. Among the subset of trials described in the FDA-approved drug label and classified as "pivotal trials" for our analysis, 76% (257/340) were published. In multivariable logistic regression for all trials 5 y postapproval, likelihood of publication correlated with statistically significant results (odds ratio [OR] 3.03, 95% confidence interval [CI] 1.78-5.17); larger sample sizes (OR 1.33 per 2-fold increase in sample size, 95% CI 1.17-1.52); and pivotal status (OR 5.31, 95% CI 3.30-8.55). In multivariable logistic regression for only the pivotal trials 5 y postapproval, likelihood of publication correlated with statistically significant results (OR 2.96, 95% CI 1.24-7.06) and larger sample sizes (OR 1.47 per 2-fold increase in sample size, 95% CI 1.15-1.88). Statistically significant results and larger sample sizes were also predictive of publication at 2 y postapproval and in multivariable Cox proportional models for all trials and the subset of pivotal trials.Over half of all supporting trials for FDA-approved drugs remained unpublished >/= 5 y after approval. Pivotal trials and trials with statistically significant results and larger sample sizes are more likely to be published. Selective reporting of trial results exists for commonly marketed drugs. Our data provide a baseline for evaluating publication bias as the new FDA Amendments Act comes into force mandating basic results reporting of clinical trials.

Project description:BackgroundScholarly publishing system relies on external peer review. However, the duration of publication process is a major concern for authors and funding bodies.ObjectiveTo evaluate the duration of the publication process in pharmacy practice journals compared with other biomedical journals indexed in PubMed.MethodsAll the articles published from 2009 to 2018 by the 33 pharmacy practice journals identified in Mendes et al. study and indexed in PubMed were gathered as study group. A comparison group was created through a random selection of 3000 PubMed PMIDs for each year of study period. Articles with publication dates outside the study period were excluded. Metadata of both groups of articles were imported from PubMed. The duration of editorial process was calculated with three periods: acceptance lag (days between 'submission date' and 'acceptance date'), lead lag (days between 'acceptance date' and 'online publication date'), and indexing lag (days between 'online publication date' and 'Entry date'). Null hypothesis significance tests and effect size measures were used to compare these periods between both groups.ResultsThe 33 pharmacy practice journals published 26,256 articles between 2009 and 2018. Comparison group random selection process resulted in a pool of 23,803 articles published in 5,622 different journals. Acceptance lag was 105 days (IQR 57-173) for pharmacy practice journals and 97 days (IQR 56-155) for the comparison group with a null effect difference (Cohen's d 0.081). Lead lag was 13 (IQR 6-35) and 23 days (IQR 9-45) for pharmacy practice and comparison journals, respectively, which resulted in a small effect. Indexing lag was 5 days (IQR 2-46) and 4 days (IQR 2-12) for pharmacy practice and control journals, which also resulted in a small effect. Slight positive time trend was found in pharmacy practice acceptance lag, while slight negative trends were found for lead and indexing lags for both groups.ConclusionsPublication process duration of pharmacy practice journals is similar to a general random sample of articles from all disciplines.