Misuse of statistical methods in 10 leading Chinese medical journals in 1998 and 2008.
Ontology highlight
ABSTRACT:
SUBMITTER: Wu S
PROVIDER: S-EPMC3217588 | biostudies-other | 2011
REPOSITORIES: biostudies-other
Similar Datasets
Project description:BackgroundHigh quality clinical research not only requires advanced professional knowledge, but also needs sound study design and correct statistical analyses. The number of clinical research articles published in Chinese medical journals has increased immensely in the past decade, but study design quality and statistical analyses have remained suboptimal. The aim of this investigation was to gather evidence on the quality of study design and statistical analyses in clinical researches conducted in China for the first decade of the new millennium.Methodology/principal findingsTen (10) leading Chinese medical journals were selected and all original articles published in 1998 (N = 1,335) and 2008 (N = 1,578) were thoroughly categorized and reviewed. A well-defined and validated checklist on study design, statistical analyses, results presentation, and interpretation was used for review and evaluation. Main outcomes were the frequencies of different types of study design, error/defect proportion in design and statistical analyses, and implementation of CONSORT in randomized clinical trials. From 1998 to 2008: The error/defect proportion in statistical analyses decreased significantly ( = 12.03, p<0.001), 59.8% (545/1,335) in 1998 compared to 52.2% (664/1,578) in 2008. The overall error/defect proportion of study design also decreased ( = 21.22, p<0.001), 50.9% (680/1,335) compared to 42.40% (669/1,578). In 2008, design with randomized clinical trials remained low in single digit (3.8%, 60/1,578) with two-third showed poor results reporting (defects in 44 papers, 73.3%). Nearly half of the published studies were retrospective in nature, 49.3% (658/1,335) in 1998 compared to 48.2% (761/1,578) in 2008. Decreases in defect proportions were observed in both results presentation ( = 93.26, p<0.001), 92.7% (945/1,019) compared to 78.2% (1023/1,309) and interpretation ( = 27.26, p<0.001), 9.7% (99/1,019) compared to 4.3% (56/1,309), some serious ones persisted.Conclusions/significanceChinese medical research seems to have made significant progress regarding statistical analyses, but there remains ample room for improvement regarding study designs. Retrospective clinical studies are the most often used design, whereas randomized clinical trials are rare and often show methodological weaknesses. Urgent implementation of the CONSORT statement is imperative.
| S-EPMC2876024 | biostudies-literature
Project description:Good quality medical research generally requires not only an expertise in the chosen medical field of interest but also a sound knowledge of statistical methodology. The number of medical research articles which have been published in Indian medical journals has increased quite substantially in the past decade. The aim of this study was to collate all evidence on study design quality and statistical analyses used in selected leading Indian medical journals. Ten (10) leading Indian medical journals were selected based on impact factors and all original research articles published in 2003 (N = 588) and 2013 (N = 774) were categorized and reviewed. A validated checklist on study design, statistical analyses, results presentation, and interpretation was used for review and evaluation of the articles. Main outcomes considered in the present study were - study design types and their frequencies, error/defects proportion in study design, statistical analyses, and implementation of CONSORT checklist in RCT (randomized clinical trials). From 2003 to 2013: The proportion of erroneous statistical analyses did not decrease (χ2=0.592, Φ=0.027, p=0.4418), 25% (80/320) in 2003 compared to 22.6% (111/490) in 2013. Compared with 2003, significant improvement was seen in 2013; the proportion of papers using statistical tests increased significantly (χ2=26.96, Φ=0.16, p<0.0001) from 42.5% (250/588) to 56.7 % (439/774). The overall proportion of errors in study design decreased significantly (χ2=16.783, Φ=0.12 p<0.0001), 41.3% (243/588) compared to 30.6% (237/774). In 2013, randomized clinical trials designs has remained very low (7.3%, 43/588) with majority showing some errors (41 papers, 95.3%). Majority of the published studies were retrospective in nature both in 2003 [79.1% (465/588)] and in 2013 [78.2% (605/774)]. Major decreases in error proportions were observed in both results presentation (χ2=24.477, Φ=0.17, p<0.0001), 82.2% (263/320) compared to 66.3% (325/490) and interpretation (χ2=25.616, Φ=0.173, p<0.0001), 32.5% (104/320) compared to 17.1% (84/490), though some serious ones were still present. Indian medical research seems to have made no major progress regarding using correct statistical analyses, but error/defects in study designs have decreased significantly. Randomized clinical trials are quite rarely published and have high proportion of methodological problems.
| S-EPMC4391869 | biostudies-literature
Project description:Scientific claims in biomedical research are typically derived from statistical analyses. However, misuse or misunderstanding of statistical procedures and results permeate the biomedical literature, affecting the validity of those claims. One approach journals have taken to address this issue is to enlist expert statistical reviewers. How many journals do this, how statistical review is incorporated, and how its value is perceived by editors is of interest. Here we report an expanded version of a survey conducted more than 20 years ago by Goodman and colleagues (1998) with the intention of characterizing contemporary statistical review policies at leading biomedical journals. We received eligible responses from 107 of 364 (28%) journals surveyed, across 57 fields, mostly from editors in chief. 34% (36/107) rarely or never use specialized statistical review, 34% (36/107) used it for 10-50% of their articles and 23% used it for all articles. These numbers have changed little since 1998 in spite of dramatically increased concern about research validity. The vast majority of editors regarded statistical review as having substantial incremental value beyond regular peer review and expressed comparatively little concern about the potential increase in reviewing time, cost, and difficulty identifying suitable statistical reviewers. Improved statistical education of researchers and different ways of employing statistical expertise are needed. Several proposals are discussed.
| S-EPMC7529205 | biostudies-literature
Project description:ImportanceWomen remain underrepresented among editors of scientific journals, particularly in senior positions. However, to what extent this applies to medical journals of different specialties remains unclear.ObjectiveTo investigate the gender distribution of the editors in chief at leading medical journals.Design, setting, and participantsCross-sectional study of the editors in chief at the top 10 international medical journals of 41 categories related to the medical specialties of the Clarivate Analytics Web of Science Journal Citation Reports in 2019.Main outcomes and measuresProportion of women as editors in chief.ResultsThis study found that, overall, women represented 21% (94 of 44) of the editors in chief, with wide variation across medical specialties from 0% to 82%. There were 5 categories for which none of the editors in chief were women (dentistry, oral surgery and medicine; allergy; psychiatry; anesthesiology; and ophthalmology) and only 3 categories for which women outnumbered men as editors in chief (primary health care, microbiology, and genetics and heredity). In 27 of the 41 categories, women represented less than a third of the editors in chief (eg, 1 of 10 for critical care medicine, 2 of 10 for gastroenterology and hepatology, and 3 of 10 for endocrinology and metabolism).Conclusions and relevanceThis study found that women are underrepresented among editors in chief of leading medical journals. For the benefit of medical research, a joint effort from editorial boards, publishers, authors, and academic institutions is required to address this gender gap.
| S-EPMC8427369 | biostudies-literature
Project description:ObjectivesAcademical and not-for-profit research funders are increasingly requiring that the research they fund must be published open access, with some insisting on publishing with a Creative Commons Attribution (CC BY) licence to allow the broadest possible use. We aimed to clarify the open access variants provided by leading medical journals and record the availability of the CC BY licence for commercially funded research.MethodsWe identified medical journals with a 2015 impact factor of ≥15.0 on 24 May 2017, then excluded from the analysis journals that only publish review articles. Between 29 June 2017 and 26 July 2017, we collected information about each journal's open access policies from their websites and/or by email contact. We contacted the journals by email again between 6 December 2017 and 2 January 2018 to confirm our findings.ResultsThirty-five medical journals publishing original research from 13 publishers were included in the analysis. All 35 journals offered some form of open access allowing articles to be free-to-read, either immediately on publication or after a delay of up to 12 months. Of these journals, 21 (60%) provided immediate open access with a CC BY licence under certain circumstances (eg, to specific research funders). Of these 21, 20 only offered a CC BY licence to authors funded by non-commercial organisations and one offered this option to any funder who required it.ConclusionsMost leading medical journals do not offer to authors reporting commercially funded research an open access licence that allows unrestricted sharing and adaptation of the published material. The journals' policies are therefore not aligned with open access declarations and guidelines. Commercial research funders lag behind academical funders in the development of mandatory open access policies, and it is time for them to work with publishers to advance the dissemination of the research they fund.
| S-EPMC6596940 | biostudies-literature
Project description:ImportanceHigh-quality peer reviews are often thought to be essential to ensuring the integrity of the scientific publication process, but measuring peer review quality is challenging. Although imperfect, review word count could potentially serve as a simple, objective metric of review quality.ObjectiveTo determine the prevalence of very short peer reviews and how often they inform editorial decisions on research articles in 3 leading general medical journals.Design, setting, and participantsThis cross-sectional study compiled a data set of peer reviews from published, full-length original research articles from 3 general medical journals (The BMJ, PLOS Medicine, and BMC Medicine) between 2003 and 2022. Eligible articles were those with peer review data; all peer reviews used to make the first editorial decision (ie, accept vs revise and resubmit) were included.Main outcomes and measuresPrevalence of very short reviews was the primary outcome, which was defined as a review of fewer than 200 words. In secondary analyses, thresholds of fewer than 100 words and fewer than 300 words were used. Results were disaggregated by journal and year. The proportion of articles for which the first editorial decision was made based on a set of peer reviews in which very short reviews constituted 100%, 50% or more, 33% or more, and 20% or more of the reviews was calculated.ResultsIn this sample of 11 466 reviews (including 6086 in BMC Medicine, 3816 in The BMJ, and 1564 in PLOS Medicine) corresponding to 4038 published articles, the median (IQR) word count per review was 425 (253-575) words, and the mean (SD) word count was 520.0 (401.0) words. The overall prevalence of very short (<200 words) peer reviews was 1958 of 11 466 reviews (17.1%). Across the 3 journals, 843 of 4038 initial editorial decisions (20.9%) were based on review sets containing 50% or more very short reviews. The prevalence of very short reviews and share of editorial decisions based on review sets containing 50% or more very short reviews was highest for BMC Medicine (693 of 2585 editorial decisions [26.8%]) and lowest for The BMJ (76 of 1040 editorial decisions [7.3%]).Conclusion and relevanceIn this study of 3 leading general medical journals, one-fifth of initial editorial decisions for published articles were likely based at least partially on reviews of such short length that they were unlikely to be of high quality. Future research could determine whether monitoring peer review length improves the quality of peer reviews and which interventions, such as incentives and norm-based interventions, may elicit more detailed reviews.
| S-EPMC10722334 | biostudies-literature
Project description:BackgroundRandomized controlled trials (RCTs) are not always well reported, especially in terms of their methodological descriptions. This study aimed to investigate the adherence of methodological reporting complying with CONSORT and explore associated trial level variables in the Chinese nursing care field.MethodsIn June 2012, we identified RCTs published in five leading Chinese nursing journals and included trials with details of randomized methods. The quality of methodological reporting was measured through the methods section of the CONSORT checklist and the overall CONSORT methodological items score was calculated and expressed as a percentage. Meanwhile, we hypothesized that some general and methodological characteristics were associated with reporting quality and conducted a regression with these data to explore the correlation. The descriptive and regression statistics were calculated via SPSS 13.0.ResultsIn total, 680 RCTs were included. The overall CONSORT methodological items score was 6.34 ± 0.97 (Mean ± SD). No RCT reported descriptions and changes in "trial design," changes in "outcomes" and "implementation," or descriptions of the similarity of interventions for "blinding." Poor reporting was found in detailing the "settings of participants" (13.1%), "type of randomization sequence generation" (1.8%), calculation methods of "sample size" (0.4%), explanation of any interim analyses and stopping guidelines for "sample size" (0.3%), "allocation concealment mechanism" (0.3%), additional analyses in "statistical methods" (2.1%), and targeted subjects and methods of "blinding" (5.9%). More than 50% of trials described randomization sequence generation, the eligibility criteria of "participants," "interventions," and definitions of the "outcomes" and "statistical methods." The regression analysis found that publication year and ITT analysis were weakly associated with CONSORT score.ConclusionsThe completeness of methodological reporting of RCTs in the Chinese nursing care field is poor, especially with regard to the reporting of trial design, changes in outcomes, sample size calculation, allocation concealment, blinding, and statistical methods.
| S-EPMC4240555 | biostudies-literature
Project description:BackgroundAlthough medical publications are frequently used as the source of information, the prevalence of errata remains unclear. The objective of this study was to examine peer-review and publication processes of medical journals as well as to determine the occurrence of reported errata in medical journals and timeliness in identifying and correcting errata.MethodsFive medical journals, New England Journal of Medicine, Annals of Internal Medicine, British Medical Journal, Journal of American Medical Association, and Lancet, were evaluated. The characteristics of these journals were obtained from editors' survey. All these journals report errata noted in their prior publications. We retrospectively analyzed all errata reported from January 1, 2012, to December 31, 2012. The mean number of reported errata per issue, the most common errata, and the mean time to report errata were calculated.ResultsThe journals had high impact factors (14-51), received 3,200 to more than 15,000 submissions in 2012, and utilized two or more external reviewers and usually two or more editors for any accepted articles. All the journals edited the accepted articles, including references, figures, and tables for style. A mean of 1.3 articles with ≥1 errata was reported per issue (a total of 306 articles with errata in 226 issues). Errata in author's information, numeric errata, and errata in the figures and tables were the most common errata. The mean time to report the errata was 122 days.ConclusionThe high-impact journals, with extensive pre-publication review, reported relatively few errata per issue. The delay in reporting errata needs further exploration.
| S-EPMC4246137 | biostudies-literature
Project description: Not available
| S-EPMC7154950 | biostudies-literature
Project description:Although regression models play a central role in the analysis of medical research projects, there still exist many misconceptions on various aspects of modeling leading to faulty analyses. Indeed, the rapidly developing statistical methodology and its recent advances in regression modeling do not seem to be adequately reflected in many medical publications. This problem of knowledge transfer from statistical research to application was identified by some medical journals, which have published series of statistical tutorials and (shorter) papers mainly addressing medical researchers. The aim of this review was to assess the current level of knowledge with regard to regression modeling contained in such statistical papers. We searched for target series by a request to international statistical experts. We identified 23 series including 57 topic-relevant articles. Within each article, two independent raters analyzed the content by investigating 44 predefined aspects on regression modeling. We assessed to what extent the aspects were explained and if examples, software advices, and recommendations for or against specific methods were given. Most series (21/23) included at least one article on multivariable regression. Logistic regression was the most frequently described regression type (19/23), followed by linear regression (18/23), Cox regression and survival models (12/23) and Poisson regression (3/23). Most general aspects on regression modeling, e.g. model assumptions, reporting and interpretation of regression results, were covered. We did not find many misconceptions or misleading recommendations, but we identified relevant gaps, in particular with respect to addressing nonlinear effects of continuous predictors, model specification and variable selection. Specific recommendations on software were rarely given. Statistical guidance should be developed for nonlinear effects, model specification and variable selection to better support medical researchers who perform or interpret regression analyses.
| S-EPMC8786189 | biostudies-literature