Dataset Information

Quality Assessment of Published Systematic Reviews in High Impact Cardiology Journals: Revisiting the Evidence Pyramid.

ABSTRACT: Objective: Systematic reviews are increasingly used as sources of evidence in clinical cardiology guidelines. In the present study, we aimed to assess the quality of published systematic reviews in high impact cardiology journals. Methods: We searched PubMed for systematic reviews published between 2010 and 2019 in five general cardiology journals with the highest impact factor (according to Clarivate Analytics 2019). We extracted data on eligibility criteria, methodological characteristics, bias assessments, and sources of funding. Further, we assessed the quality of retrieved reviews using the AMSTAR tool. Results: A total of 352 systematic reviews were assessed. The AMSTAR quality score was low or critically low in 71% (95% CI: 65.7-75.4) of the assessed reviews. Sixty-four reviews (18.2%, 95% CI: 14.5-22.6) registered/published their protocol. Only 221 reviews (62.8%, 95% CI: 57.6-67.7) reported adherence to the EQUATOR checklists, 208 reviews (58.4%, 95% CI: 53.9-64.1) assessed the risk of bias in the included studies, and 177 reviews (52.3%, 95% CI: 45.1-55.5) assessed the risk of publication bias in their primary outcome analysis. The primary outcome was statistically significant in 274 (79.6%, 95% CI: 75.1-83.6) and had statistical heterogeneity in 167 (48.5%, 95% CI: 43.3-53.8) reviews. The use and sources of external funding was not disclosed in 87 reviews (24.7%, 95% CI: 20.5-29.5). Data analysis showed that the existence of publication bias was significantly associated with statistical heterogeneity of the primary outcome and that complex design, larger sample size, and higher AMSTAR quality score were associated with higher citation metrics. Conclusion: Our analysis uncovered widespread gaps in conducting and reporting systematic reviews in cardiology. These findings highlight the importance of rigorous editorial and peer review policies in systematic review publishing, as well as education of the investigators and clinicians on the synthesis and interpretation of evidence.

SUBMITTER: Abushouk AI

PROVIDER: S-EPMC8220077 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo compare cancer-related systematic reviews (SRs) published in the Cochrane Database of SRs (CDSR) and high-impact journals, with respect to type, content, quality and citation rates.DesignMethodological SR with assessment and comparison of SRs and meta-analyses. Two authors independently assessed methodological quality using an Assessment of Multiple Systematic Reviews (AMSTAR)-based extraction form. Both authors independently screened search results, extracted content-relevant characteristics and retrieved citation numbers of the included reviews using the Clarivate Analytics Web of Science database.Data sourcesCancer-related SRs were retrieved from the CDSR, as well as from the 10 journals which publish oncological SRs and had the highest impact factors, using a comprehensive search in both the CDSR and MEDLINE.Eligibility criteria for selecting studiesWe included all cancer-related SRs and meta-analyses published from January 2011 to May 2016. Methodological SRs were excluded.ResultsWe included 346 applicable Cochrane reviews and 215 SRs from high-impact journals. Cochrane reviews consistently met more individual AMSTAR criteria, notably with regard to an a priori design (risk ratio (RR) 3.89; 95%?CI 3.10 to 4.88), inclusion of the grey literature and trial registries (RR 3.52; 95%?CI 2.84 to 4.37) in their searches, and the reporting of excluded studies (RR 8.80; 95%?CI 6.06 to 12.78). Cochrane reviews were less likely to address questions of prognosis (RR 0.04; 95%?CI 0.02 to 0.09), use individual patient data (RR 0.03; 95%?CI 0.01 to 0.09) or be based on non-randomised controlled trials (RR 0.04; 95%?CI 0.02 to 0.09). Citation rates of Cochrane reviews were notably lower than those for high-impact journals (Cochrane reviews: mean number of citations 6.52 (range 0-143); high-impact journal SRs: 74.45 (0-652)).ConclusionsWhen comparing cancer-related SRs published in the CDSR versus those published in high-impact medical journals, Cochrane reviews were consistently of higher methodological quality, but cited less frequently.

Project description:BackgroundA high-quality search strategy is considered an essential component of systematic reviews but many do not contain reproducible search strategies. It is unclear if low reproducibility spans medical disciplines, is affected by librarian/search specialist involvement or has improved with increased awareness of reporting guidelines.ObjectivesTo examine the reporting of search strategies in systematic reviews published in Pediatrics, Surgery or Cardiology journals in 2012 and determine rates and predictors of including a reproducible search strategy.MethodsWe identified all systematic reviews published in 2012 in the ten highest impact factor journals in Pediatrics, Surgery and Cardiology. Each search strategy was coded to indicate what elements were reported and whether the overall search was reproducible. Reporting and reproducibility rates were compared across disciplines and we measured the influence of librarian/search specialist involvement, discipline or endorsement of a reporting guideline on search reproducibility.Results272 articles from 25 journals were included. Reporting of search elements ranged widely from 91% of articles naming search terms to 33% providing a full search strategy and 22% indicating the date the search was executed. Only 22% of articles provided at least one reproducible search strategy and 13% provided a reproducible strategy for all databases searched in the article. Librarians or search specialists were reported as involved in 17% of articles. There were strong disciplinary differences on the reporting of search elements. In the multivariable analysis, only discipline (Pediatrics) was a significant predictor of the inclusion of a reproducible search strategy.ConclusionsDespite recommendations to report full, reproducible search strategies, many articles still do not. In addition, authors often report a single strategy as covering all databases searched, further decreasing reproducibility. Further research is needed to determine how disciplinary culture may encourage reproducibility and the role that journal editors and peer reviewers could play.

Project description:BACKGROUND:A crucial element in the systematic review (SR) methodology is the appraisal of included primary studies, using tools for assessment of methodological quality or risk of bias (RoB). SR authors can conduct sensitivity analyses to explore whether their results are sensitive to exclusion of low quality studies or a high RoB. However, it is unknown which tools do SR authors use for assessing quality/RoB, and how they set threshold for quality/RoB in sensitivity analyses. The aim of this study was to assess quality/RoB assessment tools, the types of sensitivity analyses and quality/RoB thresholds for sensitivity analyses used within SRs published in high-impact pain/anesthesiology journals. METHODS:This was a methodological study. We analyzed SRs published from January 2005 to June 2018 in the 25% highest-ranking journals within the Journal Citation Reports (JCR) "Anesthesiology" category. We retrieved the SRs from PubMed. Two authors independently screened records, full texts, and extracted data on quality/RoB tools and sensitivity analyses. We extracted data about quality/RoB tools, types of sensitivity analyses and the thresholds for quality/RoB used in them. RESULTS:Out of 678 analyzed SRs, 513 (76%) reported the use of quality/RoB assessments. The most commonly reported tools for assessing quality/RoB in the studies were the Cochrane tool for risk of bias assessment (N?=?251; 37%) and Jadad scale (N?=?99; 15%). Meta-analysis was conducted in 451 (66%) of SRs and sensitivity analysis in 219/451 (49%). Most commonly, sensitivity analysis was conducted to explore the influence of study quality/RoB (90/219; 41%) on the results. Quality/RoB thresholds used for sensitivity analysis for those studies were clearly reported in 47 (52%) articles that used them. The quality/RoB thresholds used for sensitivity analyses were highly heterogeneous and inconsistent, even when the same tool was used. CONCLUSIONS:A quarter of SRs reported using quality/RoB assessments, and some of them cited tools that are not meant for assessing quality/RoB. Authors who use quality/RoB to explore the robustness of their results in meta-analyses use highly heterogeneous quality/RoB thresholds in sensitivity analyses. Better methodological consistency for quality/RoB sensitivity analyses is needed.

Project description:Few Orthopaedics and Traumatology journals from Latin America and Spain are indexed in major databases; controlled clinical trials published in these journals cannot be exhaustively retrieved using electronic literature searches. We aimed to identify, describe and assess the quality of controlled clinical trials published in Orthopaedics and Traumatology journals from Latin America and Spain through handsearching and evidence mapping methods. We identified controlled clinical trials published in eligible Orthopaedics/Traumatology journals in Spanish until July 2017 by handsearching. Data were extracted for controlled clinical trials main characteristics and the Cochrane risk of bias tool was used to assess the controlled clinical trials methodological quality. In addition, we mapped the main findings of these trials. As a result, we assessed 5631 references in 29 eligible journals of which 57 were controlled clinical trials (1.0%). Controlled clinical trials were published between 1995 and 2017 at a rate of 2.5 per year. Journals from Spain and Mexico published around 63% of the controlled clinical trials identified. The median sample size of patients enrolled was 60 (range = 30-300 participants). About conditions assessed, 38.5% of controlled clinical trials assessed issues related to knee conditions, 15.7% about hip and 10.5% about trauma or spine. The risk of bias domains most affected was selective reporting bias and random sequence generation. In addition, only two and seven trials had low risk of bias in all items related to participant/personnel and outcome assessment blindings, respectively. More than 40% of studies did not report differences on benefits/harms between the interventions assessed. As a conclusion, the number of controlled clinical trials published in Orthopaedics/Traumatology journals from Latin America and Spain is low. These controlled clinical trials had important methodological shortcomings and were judged as unclear or high risk of bias. These trials are now available in CENTRAL for their potential inclusion in systematic reviews and other documents of synthesis.

Project description:IntroductionAlong with the proliferation of Open Access (OA) publishing, the interest for comparing the scientific quality of studies published in OA journals versus subscription journals has also increased. With our study we aimed to compare the methodological quality and the quality of reporting of primary epidemiological studies and systematic reviews and meta-analyses published in OA and non-OA journals.MethodsIn order to identify the studies to appraise, we listed all OA and non-OA journals which published in 2013 at least one primary epidemiologic study (case-control or cohort study design), and at least one systematic review or meta-analysis in the field of oncology. For the appraisal, we picked up the first studies published in 2013 with case-control or cohort study design from OA journals (Group A; n = 12), and in the same time period from non-OA journals (Group B; n = 26); the first systematic reviews and meta-analyses published in 2013 from OA journals (Group C; n = 15), and in the same time period from non-OA journals (Group D; n = 32). We evaluated the methodological quality of studies by assessing the compliance of case-control and cohort studies to Newcastle and Ottawa Scale (NOS) scale, and the compliance of systematic reviews and meta-analyses to Assessment of Multiple Systematic Reviews (AMSTAR) scale. The quality of reporting was assessed considering the adherence of case-control and cohort studies to STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) checklist, and the adherence of systematic reviews and meta-analyses to Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) checklist.ResultsAmong case-control and cohort studies published in OA and non-OA journals, we did not observe significant differences in the median value of NOS score (Group A: 7 (IQR 7-8) versus Group B: 8 (7-9); p = 0.5) and in the adherence to STROBE checklist (Group A, 75% versus Group B, 80%; p = 0.1). The results did not change after adjustment for impact factor. The compliance with AMSTAR and adherence to PRISMA checklist were comparable between systematic reviews and meta-analyses published in OA and non-OA journals (Group C, 46.0% versus Group D, 55.0%; p = 0.06), (Group C, 72.0% versus Group D, 76.0%; p = 0.1), respectively).ConclusionThe epidemiological studies published in OA journals in the field of oncology approach the same methodological quality and quality of reporting as studies published in non-OA journals.

Dataset Information

Quality Assessment of Published Systematic Reviews in High Impact Cardiology Journals: Revisiting the Evidence Pyramid.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets