Dataset Information

How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey.

ABSTRACT: Scientific claims in biomedical research are typically derived from statistical analyses. However, misuse or misunderstanding of statistical procedures and results permeate the biomedical literature, affecting the validity of those claims. One approach journals have taken to address this issue is to enlist expert statistical reviewers. How many journals do this, how statistical review is incorporated, and how its value is perceived by editors is of interest. Here we report an expanded version of a survey conducted more than 20 years ago by Goodman and colleagues (1998) with the intention of characterizing contemporary statistical review policies at leading biomedical journals. We received eligible responses from 107 of 364 (28%) journals surveyed, across 57 fields, mostly from editors in chief. 34% (36/107) rarely or never use specialized statistical review, 34% (36/107) used it for 10-50% of their articles and 23% used it for all articles. These numbers have changed little since 1998 in spite of dramatically increased concern about research validity. The vast majority of editors regarded statistical review as having substantial incremental value beyond regular peer review and expressed comparatively little concern about the potential increase in reviewing time, cost, and difficulty identifying suitable statistical reviewers. Improved statistical education of researchers and different ways of employing statistical expertise are needed. Several proposals are discussed.

SUBMITTER: Hardwicke TE

PROVIDER: S-EPMC7529205 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey.

Hardwicke Tom E TE Goodman Steven N SN

PloS one 20201001 10

Scientific claims in biomedical research are typically derived from statistical analyses. However, misuse or misunderstanding of statistical procedures and results permeate the biomedical literature, affecting the validity of those claims. One approach journals have taken to address this issue is to enlist expert statistical reviewers. How many journals do this, how statistical review is incorporated, and how its value is perceived by editors is of interest. Here we report an expanded version of ...[more]

PMID: 33002031

Similar Datasets

Project description:Good quality medical research generally requires not only an expertise in the chosen medical field of interest but also a sound knowledge of statistical methodology. The number of medical research articles which have been published in Indian medical journals has increased quite substantially in the past decade. The aim of this study was to collate all evidence on study design quality and statistical analyses used in selected leading Indian medical journals. Ten (10) leading Indian medical journals were selected based on impact factors and all original research articles published in 2003 (N = 588) and 2013 (N = 774) were categorized and reviewed. A validated checklist on study design, statistical analyses, results presentation, and interpretation was used for review and evaluation of the articles. Main outcomes considered in the present study were - study design types and their frequencies, error/defects proportion in study design, statistical analyses, and implementation of CONSORT checklist in RCT (randomized clinical trials). From 2003 to 2013: The proportion of erroneous statistical analyses did not decrease (χ2=0.592, Φ=0.027, p=0.4418), 25% (80/320) in 2003 compared to 22.6% (111/490) in 2013. Compared with 2003, significant improvement was seen in 2013; the proportion of papers using statistical tests increased significantly (χ2=26.96, Φ=0.16, p<0.0001) from 42.5% (250/588) to 56.7 % (439/774). The overall proportion of errors in study design decreased significantly (χ2=16.783, Φ=0.12 p<0.0001), 41.3% (243/588) compared to 30.6% (237/774). In 2013, randomized clinical trials designs has remained very low (7.3%, 43/588) with majority showing some errors (41 papers, 95.3%). Majority of the published studies were retrospective in nature both in 2003 [79.1% (465/588)] and in 2013 [78.2% (605/774)]. Major decreases in error proportions were observed in both results presentation (χ2=24.477, Φ=0.17, p<0.0001), 82.2% (263/320) compared to 66.3% (325/490) and interpretation (χ2=25.616, Φ=0.173, p<0.0001), 32.5% (104/320) compared to 17.1% (84/490), though some serious ones were still present. Indian medical research seems to have made no major progress regarding using correct statistical analyses, but error/defects in study designs have decreased significantly. Randomized clinical trials are quite rarely published and have high proportion of methodological problems.

Project description:BackgroundRobust statistical designing, sound statistical analysis, and standardized presentation are important to enhance the quality and transparency of biomedical research. This systematic review was conducted to summarize the statistical reporting requirements introduced by biomedical research journals with an impact factor of 10 or above so that researchers are able to give statistical issues' serious considerations not only at the stage of data analysis but also at the stage of methodological design.MethodsDetailed statistical instructions for authors were downloaded from the homepage of each of the included journals or obtained from the editors directly via email. Then, we described the types and numbers of statistical guidelines introduced by different press groups. Items of statistical reporting guideline as well as particular requirements were summarized in frequency, which were grouped into design, method of analysis, and presentation, respectively. Finally, updated statistical guidelines and particular requirements for improvement were summed up.ResultsTotally, 21 of 23 press groups introduced at least one statistical guideline. More than half of press groups can update their statistical instruction for authors gradually relative to issues of new statistical reporting guidelines. In addition, 16 press groups, covering 44 journals, address particular statistical requirements. The most of the particular requirements focused on the performance of statistical analysis and transparency in statistical reporting, including "address issues relevant to research design, including participant flow diagram, eligibility criteria, and sample size estimation," and "statistical methods and the reasons."ConclusionsStatistical requirements for authors are becoming increasingly perfected. Statistical requirements for authors remind researchers that they should make sufficient consideration not only in regards to statistical methods during the research design, but also standardized statistical reporting, which would be beneficial in providing stronger evidence and making a greater critical appraisal of evidence more accessible.

Project description:ObjectivesTo assess the prevalence of honorary and ghost authors in six leading general medical journals in 2008 and compare this with the prevalence reported by authors of articles published in 1996.DesignCross sectional survey using a web based questionnaire.SettingInternational survey of journal authors.ParticipantsSample of corresponding authors of 896 research articles, review articles, and editorial/opinion articles published in six general medical journals with high impact factors in 2008: Annals of Internal Medicine, JAMA, Lancet, Nature Medicine, New England Journal of Medicine, and PLoS Medicine.Main outcome measuresSelf reported compliance with International Committee of Medical Journal Editors (ICMJE) criteria for authorship for all authors on the selected articles.ResultsA total of 630/896 (70.3%) corresponding authors responded to the survey. The prevalence of articles with honorary authorship or ghost authorship, or both, was 21.0% (95% CI 18.0% to 24.3%), a decrease from 29.2% reported in 1996 (P = 0.004). Based on 545 responses on honorary authorship, 96 articles (17.6% (95% CI 14.6% to 21.0%)) had honorary authors (range by journal 12.2% to 29.3%), a non-significant change from 1996 (19.3%; P = 0.439). Based on 622 responses on ghost authorship, 49 articles (7.9% (6.0% to 10.3%)) had ghost authors (range by journal 2.1% to 11.0%), a significant decline from 1996 (11.5%; P = 0.023). The prevalence of honorary authorship was 25.0% in original research reports, 15.0% in reviews, and 11.2% in editorials, whereas the prevalence of ghost authorship was 11.9% in research articles, 6.0% in reviews, and 5.3% in editorials.ConclusionsEvidence of honorary and ghost authorship in 21% of articles published in major medical journals in 2008 suggests that increased efforts by scientific journals, individual authors, and academic institutions are essential to promote responsibility, accountability, and transparency in authorship, and to maintain integrity in scientific publication.

Project description:BackgroundPredatory journals fail to fulfill the tenets of biomedical publication: peer review, circulation, and access in perpetuity. Despite increasing attention in the lay and scientific press, no studies have directly assessed the perceptions of the authors or editors involved.ObjectiveOur objective was to understand the motivation of authors in sending their work to potentially predatory journals. Moreover, we aimed to understand the perspective of journal editors at journals cited as potentially predatory.MethodsPotential online predatory journals were randomly selected among 350 publishers and their 2204 biomedical journals. Author and editor email information was valid for 2227 total potential participants. A survey for authors and editors was created in an iterative fashion and distributed. Surveys assessed attitudes and knowledge about predatory publishing. Narrative comments were invited.ResultsA total of 249 complete survey responses were analyzed. A total of 40% of editors (17/43) surveyed were not aware that they were listed as an editor for the particular journal in question. A total of 21.8% of authors (45/206) confirmed a lack of peer review. Whereas 77% (33/43) of all surveyed editors were at least somewhat familiar with predatory journals, only 33.0% of authors (68/206) were somewhat familiar with them (P<.001). Only 26.2% of authors (54/206) were aware of Beall's list of predatory journals versus 49% (21/43) of editors (P<.001). A total of 30.1% of authors (62/206) believed their publication was published in a predatory journal. After defining predatory publishing, 87.9% of authors (181/206) surveyed would not publish in the same journal in the future.ConclusionsAuthors publishing in suspected predatory journals are alarmingly uninformed in terms of predatory journal quality and practices. Editors' increased familiarity with predatory publishing did little to prevent their unwitting listing as editors. Some suspected predatory journals did provide services akin to open access publication. Education, research mentorship, and a realignment of research incentives may decrease the impact of predatory publishing.

Dataset Information

How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey.

Publications

How often do leading biomedical journals use statistical experts to evaluate statistical methods? The results of a survey.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets