Dataset Information

Ratio measures in leading medical journals: structured review of accessibility of underlying absolute risks.

ABSTRACT:

Objective

To examine the accessibility of absolute risk in articles reporting ratio measures in leading medical journals.

Design

Structured review of abstracts presenting ratio measures.

Setting

Articles published between 1 June 2003 and 1 May 2004 in Annals of Internal Medicine, BMJ, Journal of the American Medical Association, Journal of the National Cancer Institute, Lancet, and New England Journal of Medicine.

Participants

222 articles based on study designs in which absolute risks were directly calculable (61 randomised trials, 161 cohort studies).

Main outcome measure

Accessibility of the absolute risks underlying the first ratio measure in the abstract.

Results

68% of articles (150/222) failed to report the underlying absolute risks for the first ratio measure in the abstract (range 55-81% across the journals). Among these articles, about half did report the underlying absolute risks elsewhere in the article (text, table, or figure) but half did not report them anywhere. Absolute risks were more likely to be reported in the abstract for randomised trials compared with cohort studies (62% v 21%; relative risk 3.0, 95% confidence interval 2.1 to 4.2) and for studies reporting crude compared with adjusted ratio measures (62% v 21%; relative risk 3.0, 2.1 to 4.3).

Conclusion

Absolute risks are often not easily accessible in articles reporting ratio measures and sometimes are missing altogether-this lack of accessibility can easily exaggerate readers' perceptions of benefit or harm.

SUBMITTER: Schwartz LM

PROVIDER: S-EPMC1702463 | biostudies-literature | 2006 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Ratio measures in leading medical journals: structured review of accessibility of underlying absolute risks.

Schwartz Lisa M LM Woloshin Steven S Dvorin Evan L EL Welch H Gilbert HG

BMJ (Clinical research ed.) 20061023 7581

<h4>Objective</h4>To examine the accessibility of absolute risk in articles reporting ratio measures in leading medical journals.<h4>Design</h4>Structured review of abstracts presenting ratio measures.<h4>Setting</h4>Articles published between 1 June 2003 and 1 May 2004 in Annals of Internal Medicine, BMJ, Journal of the American Medical Association, Journal of the National Cancer Institute, Lancet, and New England Journal of Medicine.<h4>Participants</h4>222 articles based on study designs in w ...[more]

PMID: 17060338

Similar Datasets

Project description:ImportanceHigh-quality peer reviews are often thought to be essential to ensuring the integrity of the scientific publication process, but measuring peer review quality is challenging. Although imperfect, review word count could potentially serve as a simple, objective metric of review quality.ObjectiveTo determine the prevalence of very short peer reviews and how often they inform editorial decisions on research articles in 3 leading general medical journals.Design, setting, and participantsThis cross-sectional study compiled a data set of peer reviews from published, full-length original research articles from 3 general medical journals (The BMJ, PLOS Medicine, and BMC Medicine) between 2003 and 2022. Eligible articles were those with peer review data; all peer reviews used to make the first editorial decision (ie, accept vs revise and resubmit) were included.Main outcomes and measuresPrevalence of very short reviews was the primary outcome, which was defined as a review of fewer than 200 words. In secondary analyses, thresholds of fewer than 100 words and fewer than 300 words were used. Results were disaggregated by journal and year. The proportion of articles for which the first editorial decision was made based on a set of peer reviews in which very short reviews constituted 100%, 50% or more, 33% or more, and 20% or more of the reviews was calculated.ResultsIn this sample of 11 466 reviews (including 6086 in BMC Medicine, 3816 in The BMJ, and 1564 in PLOS Medicine) corresponding to 4038 published articles, the median (IQR) word count per review was 425 (253-575) words, and the mean (SD) word count was 520.0 (401.0) words. The overall prevalence of very short (<200 words) peer reviews was 1958 of 11 466 reviews (17.1%). Across the 3 journals, 843 of 4038 initial editorial decisions (20.9%) were based on review sets containing 50% or more very short reviews. The prevalence of very short reviews and share of editorial decisions based on review sets containing 50% or more very short reviews was highest for BMC Medicine (693 of 2585 editorial decisions [26.8%]) and lowest for The BMJ (76 of 1040 editorial decisions [7.3%]).Conclusion and relevanceIn this study of 3 leading general medical journals, one-fifth of initial editorial decisions for published articles were likely based at least partially on reviews of such short length that they were unlikely to be of high quality. Future research could determine whether monitoring peer review length improves the quality of peer reviews and which interventions, such as incentives and norm-based interventions, may elicit more detailed reviews.

Project description:ObjectivesTo assess how often stratified randomisation is used, whether analysis adjusted for all balancing variables, and whether the method of randomisation was adequately reported, and to reanalyse a previously reported trial to assess the impact of ignoring balancing factors in the analysis.DesignReview of published trials and reanalysis of a previously reported trial.SettingFour leading general medical journals (BMJ, Journal of the American Medical Association, Lancet, and New England Journal of Medicine) and the second Multicenter Intrapleural Sepsis Trial (MIST2).Participants258 trials published in 2010 in the four journals. Cluster randomised, crossover, non-randomised, single arm, and phase I or II trials were excluded, as were trials reporting secondary analyses, interim analyses, or results that had been previously published in 2010.Main outcome measuresWhether the method of randomisation was adequately reported, how often balanced randomisation was used, and whether balancing factors were adjusted for in the analysis.ResultsReanalysis of MIST2 showed that an unadjusted analysis led to larger P values and a loss of power. The review of published trials showed that balanced randomisation was common, with 163 trials (63%) using at least one balancing variable. The most common methods of balancing were stratified permuted blocks (n=85) and minimisation (n=27). The method of randomisation was unclear in 37% of trials. Most trials that balanced on centre or prognostic factors were not adequately analysed; only 26% of trials adjusted for all balancing factors in their primary analysis. Trials that did not adjust for balancing factors in their analysis were less likely to show a statistically significant result (unadjusted 57% v adjusted 78%, P=0.02).ConclusionBalancing on centre or prognostic factors is common in trials but often poorly described, and the implications of balancing are poorly understood. Trialists should adjust their primary analysis for balancing factors to obtain correct P values and confidence intervals and to avoid an unnecessary loss in power.

Dataset Information

Ratio measures in leading medical journals: structured review of accessibility of underlying absolute risks.

Objective

Design

Setting

Participants

Main outcome measure

Results

Conclusion

Publications

Ratio measures in leading medical journals: structured review of accessibility of underlying absolute risks.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets