Dataset Information

Examination of CIs in health and medical journals from 1976 to 2019: an observational study.

ABSTRACT: OBJECTIVES:Previous research has shown clear biases in the distribution of published p values, with an excess below the 0.05 threshold due to a combination of p-hacking and publication bias. We aimed to examine the bias for statistical significance using published confidence intervals. DESIGN:Observational study. SETTING:Papers published in Medline since 1976. PARTICIPANTS:Over 968?000 confidence intervals extracted from abstracts and over 350?000 intervals extracted from the full-text. OUTCOME MEASURES:Cumulative distributions of lower and upper confidence interval limits for ratio estimates. RESULTS:We found an excess of statistically significant results with a glut of lower intervals just above one and upper intervals just below 1. These excesses have not improved in recent years. The excesses did not appear in a set of over 100?000 confidence intervals that were not subject to p-hacking or publication bias. CONCLUSIONS:The huge excesses of published confidence intervals that are just below the statistically significant threshold are not statistically plausible. Large improvements in research practice are needed to provide more results that better reflect the truth.

SUBMITTER: Barnett AG

PROVIDER: S-EPMC6887056 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Examination of CIs in health and medical journals from 1976 to 2019: an observational study.

Barnett Adrian Gerard AG Wren Jonathan D JD

BMJ open 20191121 11

<h4>Objectives</h4>Previous research has shown clear biases in the distribution of published p values, with an excess below the 0.05 threshold due to a combination of p-hacking and publication bias. We aimed to examine the bias for statistical significance using published confidence intervals.<h4>Design</h4>Observational study.<h4>Setting</h4>Papers published in <i>Medline</i> since 1976.<h4>Participants</h4>Over 968 000 confidence intervals extracted from abstracts and over 350 000 intervals ex ...[more]

PMID: 31753893

Similar Datasets

Project description:ImportanceThe language of medical research appears to be intrinsically tied to the culture of medical research and provides a unique window into broader trends in the culture of medicine.ObjectiveTo analyze medical language from 5 premier medical journals and investigate broader changes in the culture of clinical investigation during the last 40 years.Design, setting, and participantsIn this qualitative study using a data-driven analysis, 302 293 PubMed records were extracted from JAMA, The Lancet, Annals of Internal Medicine, the BMJ, and New England Journal of Medicine from January 1, 1976, through December 31, 2015, to identify key trends in medical language. A frequency analysis was applied across the 40-year time frame in JAMA to assess the major trends in all publication types. Patient-centered language was analyzed in clinical trials in the flanking time periods (1976-1980 and 2011-2015) across the 5 journals. Data were analyzed from November 16, 2016, through November 9, 2018.Main outcomes and measuresIncreasing or decreasing frequency of words (monograms) and word pairs (bigrams) and the proportion of patient-centric words in journal article titles.ResultsIn JAMA, 50 277 articles of all publication types were included. In the frequency analysis, the most increased terms were reflective of the language of epidemiological research. The bigram analysis revealed a decline in causal language (-2.42/100 000 words to -2.03/100 000 words; false discovery rate [FDR], <0.01) and an increased description of patients in the plural form (6.92/100 000 words to 11.4/100 000 words; FDR, <0.01). A trend to separate patient from disease was observed; for example, there was a decrease in describing a patient as a diabetic (-2.21/100 000 words; FDR, <0.01) compared with a patient with diabetes. In the analysis of clinical trials in all 5 journals, 3125 titles were identified (range, 193-932 per journal). In 4 of the 5 journals, use of patient-centric keywords increased significantly (absolute increase, 18.9%-34.3%; P < .001 for 3 journals; P = .01 for 1 journal), with the New England Journal of Medicine as the exception. This finding reflects a change from shorter disease-centric titles to longer titles that describe patients with a disease.Conclusions and relevanceTrends in medical language reflect the rise of evidence-based medicine, a shift in focus from individuals to populations, and a separation of patient and disease. Data-driven analysis of medical language provides a unique window into the changing landscape of medical culture.

Project description:BackgroundQualitative research appears to be gaining acceptability in medical journals. Yet, little is actually known about the proportion of qualitative research and factors affecting its publication. This study describes the proportion of qualitative research over a 10 year period and correlates associated with its publication.DesignA quantitative longitudinal examination of the proportion of original qualitative research in 67 journals of general medicine during a 10 year period (1998-2007). The proportion of qualitative research was determined by dividing original qualitative studies published (numerator) by all original research articles published (denominator). We used a generalized estimating equations approach to assess the longitudinal association between the proportion of qualitative studies and independent variables (i.e. journals' country of publication and impact factor; editorial/methodological papers discussing qualitative research; and specific journal guidelines pertaining to qualitative research).FindingsA 2.9% absolute increase and 3.4-fold relative increase in qualitative research publications occurred over a 10 year period (1.2% in 1998 vs. 4.1% in 2007). The proportion of original qualitative research was independently and significantly associated with the publication of editorial/methodological papers in the journal (b?=?3.688, P?=?0.012); and with qualitative research specifically mentioned in guidelines for authors (b?=?6.847, P<0.001). Additionally, a higher proportion of qualitative research was associated only with journals published in the UK in comparison to other countries, yet with borderline statistical significance (b?=?1.776, P?=?0.075). The journals' impact factor was not associated with the publication of qualitative research.ConclusionsDespite an increase in the proportion of qualitative research in medical journals over a 10 year period, the proportion remains low. Journals' policies pertaining to qualitative research, as expressed by the appearance of specific guidelines and editorials/methodological papers on the subject, are independently associated with the publication of original qualitative research; irrespective of the journals' impact factor.

Project description:Letters to the Editor offer ways for readers to engage with authors' publications. Letters are the shortest manuscript for medical students to publish and medical-education journals are best suited. The UK Foundation Programme rewards medical students achieving PubMed ID publications and we hypothesise that this is a main motivation for medical students to submit Letters to the Editor. A review of 15 medical-education journals with an impact factor was conducted to identify numbers and percentages of Letters to the Editor by medical students between July 2018 and June 2020. Affiliation of medical students was collected. Our results show over two years, 299 letters were published by medical students equating to 45.9% of total letters. There was a 60% overall increase in letters by medical students published in the first 12 months compared to second 12 months. During this period overall numbers of letters published increased by 27%. 86% of the letters published by medical students over the two-year period were from UK medical schools. Five schools accounted for 60.5% of these letters. The three medical schools with highest numbers of letters published were King's College London, Imperial College London and University of Oxford for both 2018/19 and 2019/20. The increase in letters published overall with greater numbers published by students, may indicate greater awareness of Letters to the Editor as means of dissemination amongst medical students. UK medical schools published large numbers of letters, perhaps reflecting increasing importance to students of publications due to impacting on subsequent jobs. Results from our quantitative research revealing: large numbers of letters by medical students, increase in letters published from 2018/19 to 2019/20 and overrepresentation of UK medical students supports the hypothesis that medical students are publishing letters to achieve PubMed IDs. Further qualitative research is required to test our hypothesis.

Dataset Information

Examination of CIs in health and medical journals from 1976 to 2019: an observational study.

Publications

Examination of CIs in health and medical journals from 1976 to 2019: an observational study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets