Dataset Information

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer.

ABSTRACT:

Objective

To compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.

Design

Methodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.

Data sources

Cancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.

Eligibility criteria for selecting studies

We included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.

Results

We included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95%?CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95%?CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95%?CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95%?CI 0.02 to 0.09), use individual patient data (RR 0.03; 95%?CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95%?CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0-143); high-impact journal SRs: 74.45 (0-652)).

Conclusions

When comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.

SUBMITTER: Goldkuhle M

PROVIDER: S-EPMC5875625 | biostudies-literature | 2018 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer.

Goldkuhle Marius M Narayan Vikram M VM Weigl Aaron A Dahm Philipp P Skoetz Nicole N

BMJ open 20180325 3

<h4>Objective</h4>To compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.<h4>Design</h4>Methodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted conte ...[more]

PMID: 29581210

Similar Datasets

Project description:Objective: Systematic reviews are increasingly used as sources of evidence in clinical cardiology guidelines. In the present study, we aimed to assess the quality of published systematic reviews in high impact cardiology journals. Methods: We searched PubMed for systematic reviews published between 2010 and 2019 in five general cardiology journals with the highest impact factor (according to Clarivate Analytics 2019). We extracted data on eligibility criteria, methodological characteristics, bias assessments, and sources of funding. Further, we assessed the quality of retrieved reviews using the AMSTAR tool. Results: A total of 352 systematic reviews were assessed. The AMSTAR quality score was low or critically low in 71% (95% CI: 65.7-75.4) of the assessed reviews. Sixty-four reviews (18.2%, 95% CI: 14.5-22.6) registered/published their protocol. Only 221 reviews (62.8%, 95% CI: 57.6-67.7) reported adherence to the EQUATOR checklists, 208 reviews (58.4%, 95% CI: 53.9-64.1) assessed the risk of bias in the included studies, and 177 reviews (52.3%, 95% CI: 45.1-55.5) assessed the risk of publication bias in their primary outcome analysis. The primary outcome was statistically significant in 274 (79.6%, 95% CI: 75.1-83.6) and had statistical heterogeneity in 167 (48.5%, 95% CI: 43.3-53.8) reviews. The use and sources of external funding was not disclosed in 87 reviews (24.7%, 95% CI: 20.5-29.5). Data analysis showed that the existence of publication bias was significantly associated with statistical heterogeneity of the primary outcome and that complex design, larger sample size, and higher AMSTAR quality score were associated with higher citation metrics. Conclusion: Our analysis uncovered widespread gaps in conducting and reporting systematic reviews in cardiology. These findings highlight the importance of rigorous editorial and peer review policies in systematic review publishing, as well as education of the investigators and clinicians on the synthesis and interpretation of evidence.

Project description:BACKGROUND:Randomised controlled trials (RCTs) provide the most reliable information to inform clinical practice and patient care. We aimed to map global clinical research publication activity through RCT-related articles in high-impact-factor medical journals over the past five decades. METHODS:We conducted a cross-sectional analysis of articles published in the highest ranked medical journals with an impact factor?>?10 (according to Journal Citation Reports published in 2017). We searched PubMed/MEDLINE (from inception to December 31, 2017) for all RCT-related articles (e.g. primary RCTs, secondary analyses and methodology papers) published in high-impact-factor medical journals. For each included article, raw metadata were abstracted from the Web of Science. A process of standardization was conducted to unify the different terms and grammatical variants and to remove typographical, transcription and/or indexing errors. Descriptive analyses were conducted (including the number of articles, citations, most prolific authors, countries, journals, funding sources and keywords). Network analyses of collaborations between countries and co-words are presented. RESULTS:We included 39,305 articles (for the period 1965-2017) published in forty journals. The Lancet (n?=?3593; 9.1%), the Journal of Clinical Oncology (n?=?3343; 8.5%) and The New England Journal of Medicine (n?=?3275 articles; 8.3%) published the largest number of RCTs. A total of 154 countries were involved in the production of articles. The global productivity ranking was led by the United States (n?=?18,393 articles), followed by the United Kingdom (n?=?8028 articles), Canada (n?=?4548 articles) and Germany (n?=?4415 articles). Seventeen authors who had published 100 or more articles were identified; the most prolific authors were affiliated with Duke University (United States), Harvard University (United States) and McMaster University (Canada). The main funding institutions were the National Institutes of Health (United States), Hoffmann-La Roche (Switzerland), Pfizer (United States), Merck Sharp & Dohme (United States) and Novartis (Switzerland). The 100 most cited RCTs were published in nine journals, led by The New England Journal of Medicine (n?=?78 articles), The Lancet (n?=?9 articles) and JAMA (n?=?7 articles). These landmark contributions focused on novel methodological approaches (e.g. the "Bland-Altman method") and trials on the management of chronic conditions (e.g. diabetes control, hormone replacement therapy in postmenopausal women, multiple therapies for diverse cancers, cardiovascular therapies such as lipid-lowering statins, antihypertensive medications, and antiplatelet and antithrombotic therapy). CONCLUSIONS:Our analysis identified authors, countries, funding institutions, landmark contributions and high-impact-factor medical journals publishing RCTs. Over the last 50?years, publication production in leading medical journals has increased, with Western countries leading in research but with low- and middle-income countries showing very limited representation.

Project description:A high-quality search strategy is considered an essential component of systematic reviews but many do not contain reproducible search strategies. It is unclear if low reproducibility spans medical disciplines, is affected by librarian/search specialist involvement or has improved with increased awareness of reporting guidelines.To examine the reporting of search strategies in systematic reviews published in Pediatrics, Surgery or Cardiology journals in 2012 and determine rates and predictors of including a reproducible search strategy.We identified all systematic reviews published in 2012 in the ten highest impact factor journals in Pediatrics, Surgery and Cardiology. Each search strategy was coded to indicate what elements were reported and whether the overall search was reproducible. Reporting and reproducibility rates were compared across disciplines and we measured the influence of librarian/search specialist involvement, discipline or endorsement of a reporting guideline on search reproducibility.272 articles from 25 journals were included. Reporting of search elements ranged widely from 91% of articles naming search terms to 33% providing a full search strategy and 22% indicating the date the search was executed. Only 22% of articles provided at least one reproducible search strategy and 13% provided a reproducible strategy for all databases searched in the article. Librarians or search specialists were reported as involved in 17% of articles. There were strong disciplinary differences on the reporting of search elements. In the multivariable analysis, only discipline (Pediatrics) was a significant predictor of the inclusion of a reproducible search strategy.Despite recommendations to report full, reproducible search strategies, many articles still do not. In addition, authors often report a single strategy as covering all databases searched, further decreasing reproducibility. Further research is needed to determine how disciplinary culture may encourage reproducibility and the role that journal editors and peer reviewers could play.

Project description:BACKGROUND:Consumers, clinicians, policymakers and researchers require high quality evidence to guide decision-making in child health. Though Cochrane systematic reviews (SRs) are a well-established source of evidence, little is known about the characteristics of non-Cochrane child-relevant SRs. To complement published descriptions of Cochrane SRs, we aimed to characterize the epidemiologic, methodological, and reporting qualities of non-Cochrane child-relevant SRs published in 2014. METHODS:English-language child-relevant SRs of quantitative primary research published outside the Cochrane Library in 2014 were eligible for this descriptive analysis. A research librarian searched MEDLINE, CINAHL, Web of Science, and PubMed in August 2015. A single reviewer screened articles for inclusion; a second verified the excluded studies. Reviewers extracted: general characteristics of the review; included study characteristics; methodological approaches. We performed univariate analyses and presented the findings narratively. RESULTS:We identified 1598 child-relevant SRs containing a median (IQR) 19 (11, 33) studies. These originated primarily from high-income countries (n?=?1247, 78.0%) and spanned 47 of the 53 Cochrane Review Groups. Most synthesized therapeutic (n?=?753, 47.1%) or epidemiologic (n?=?701, 43.8%) evidence. Though 39.3% (n?=?628) of SRs included evidence related to children only, few were published in pediatric-specific journals (n?=?283, 17.7%). Reporting quality seemed poor based on the items we assessed; few reviews mentioned an a-priori protocol (n?=?246, 15.4%) or registration (n?=?111, 6.9%), and only 23.4% (n?=?374) specified a primary outcome. Many SRs relied solely on evidence from non-RCTs (n?=?796, 49.8%). Less than two-thirds (n?=?953, 59.6%) appraised the quality of included studies and assessments of the certainty of the body of evidence were rare (n?=?102, 6.4%). CONCLUSIONS:Child-relevant Cochrane SRs are a known source of high quality evidence in pediatrics. There exists, however, an abundance of evidence from non-Cochrane SRs that may be complementary. Our findings show that high-quality non-Cochrane SRs may not be practical nor easy for knowledge users to find. Improvements are needed to ensure that evidence syntheses published outside of the Cochrane Library adhere to the high standard of conduct and reporting characteristic of Cochrane SRs.

Project description:BACKGROUND:A crucial element in the systematic review (SR) methodology is the appraisal of included primary studies, using tools for assessment of methodological quality or risk of bias (RoB). SR authors can conduct sensitivity analyses to explore whether their results are sensitive to exclusion of low quality studies or a high RoB. However, it is unknown which tools do SR authors use for assessing quality/RoB, and how they set threshold for quality/RoB in sensitivity analyses. The aim of this study was to assess quality/RoB assessment tools, the types of sensitivity analyses and quality/RoB thresholds for sensitivity analyses used within SRs published in high-impact pain/anesthesiology journals. METHODS:This was a methodological study. We analyzed SRs published from January 2005 to June 2018 in the 25% highest-ranking journals within the Journal Citation Reports (JCR) "Anesthesiology" category. We retrieved the SRs from PubMed. Two authors independently screened records, full texts, and extracted data on quality/RoB tools and sensitivity analyses. We extracted data about quality/RoB tools, types of sensitivity analyses and the thresholds for quality/RoB used in them. RESULTS:Out of 678 analyzed SRs, 513 (76%) reported the use of quality/RoB assessments. The most commonly reported tools for assessing quality/RoB in the studies were the Cochrane tool for risk of bias assessment (N?=?251; 37%) and Jadad scale (N?=?99; 15%). Meta-analysis was conducted in 451 (66%) of SRs and sensitivity analysis in 219/451 (49%). Most commonly, sensitivity analysis was conducted to explore the influence of study quality/RoB (90/219; 41%) on the results. Quality/RoB thresholds used for sensitivity analysis for those studies were clearly reported in 47 (52%) articles that used them. The quality/RoB thresholds used for sensitivity analyses were highly heterogeneous and inconsistent, even when the same tool was used. CONCLUSIONS:A quarter of SRs reported using quality/RoB assessments, and some of them cited tools that are not meant for assessing quality/RoB. Authors who use quality/RoB to explore the robustness of their results in meta-analyses use highly heterogeneous quality/RoB thresholds in sensitivity analyses. Better methodological consistency for quality/RoB sensitivity analyses is needed.

Dataset Information

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer.

Objective

Design

Data sources

Eligibility criteria for selecting studies

Results

Conclusions

Publications

A systematic assessment of Cochrane reviews and systematic reviews published in high-impact medical journals related to cancer.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure