Methods of conduct and reporting of living systematic reviews: a protocol for a living methodological survey.
Ontology highlight
ABSTRACT:
SUBMITTER: Khamis AM
PROVIDER: S-EPMC6556985 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveThe objective of this study was to examine methodological and reporting characteristics of systematic reviews and meta-analyses which compare diagnostic test accuracy (DTA) of multiple index tests, identify good practice, and develop guidance for better reporting.Study design and settingMethodological survey of 127 comparative or multiple tests reviews published in 74 different general medical and specialist journals. We summarized methods and reporting characteristics that are likely to differ between reviews of a single test and comparative reviews. We then developed guidance to enhance reporting of test comparisons in DTA reviews.ResultsOf 127 reviews, 16 (13%) reviews restricted study selection and test comparisons to comparative accuracy studies while the remaining 111 (87%) reviews included any study type. Fifty-three reviews (42%) statistically compared test accuracy with only 18 (34%) of these using recommended methods. Reporting of several items-in particular the role of the index tests, test comparison strategy, and limitations of indirect comparisons (i.e., comparisons involving any study type)-was deficient in many reviews. Five reviews with exemplary methods and reporting were identified.ConclusionReporting quality of reviews which evaluate and compare multiple tests is poor. The guidance developed, complemented with the exemplars, can assist review authors in producing better quality comparative reviews.
| S-EPMC7203546 | biostudies-literature
Project description:Background: Systematic reviews (SR) can be classified by type depending on the research question they are based on. This work identifies and describes the most relevant methodological resources to conduct high-quality reviews that answer health care questions regarding prevalence, prognosis, diagnostic accuracy and effects of interventions. Methods: Methodological resources have been identified from literature searches and consulting guidelines from institutions that develop SRs. The selected resources are organized by type of SR, and stage of development of the review (formulation of the research question, development of the protocol, literature search, risk of bias assessment, synthesis of findings, assessment of the quality of evidence, and report of SR results and conclusions). Results: Although the different types of SRs are developed following the same steps, each SR type requires specific methods, differing in characteristics and complexity. The extent of methodological development varies by type of SR, with more solid guidelines available for diagnostic accuracy and effects of interventions SRs. This methodological toolkit describes the most up-to-date risk of bias instruments: Quality in Prognostic Studies (QUIPS) tool and Prediction model study Risk Of Bias Assessment Tool (PROBAST) for prognostic SRs, Quality assessment of diagnostic accuracy studies tool (QUADAS-2) for diagnostic accuracy SRs, Cochrane risk of bias tool (ROB-2) and Risk of bias in non-randomised studies of interventions studies tool (ROBINS-I) for effects of interventions SRs, as well as the latest developments on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Conclusions: This structured compilation of the best methodological resources for each type of SR may prove to be a very useful tool for those researchers that wish to develop SRs or conduct methodological research works on SRs.
| S-EPMC7542253 | biostudies-literature
Project description:BACKGROUND: Systematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions. METHODS: We identified updated Cochrane SRs published in issue 4, 2002 of the Cochrane Library. We assessed the updated and original versions of the SRs using two instruments: the 10 item enhanced Overview Quality Assessment Questionnaire (OQAQ), and an 18-item reporting quality checklist and flow chart based upon the Quality of Reporting of Meta-analyses (QUOROM) statement. At least two reviewers extracted data and assessed quality. We calculated the percentage (with a 95% confidence interval) of 'yes' answers to each question. We calculated mean differences in percentage, 95% confidence intervals and p-values for each of the individual items and the overall methodological quality score of the updated and pre-updated versions using OQAQ. RESULTS: We assessed 53 SRs. There was no significant improvement in the global quality score of the OQAQ (mean difference 0.11 (-0.28; 0.70 p = 0.52)). Updated reviews showed a significant improvement of 18.9 (7.2; 30.6 p < .01) on the OQAQ item assessing whether the conclusions drawn by the author(s) were supported by the data and/or analysis presented in the SR. The QUOROM statement showed that the quality of reporting of Cochrane reviews improved in some areas with updating. Improvements were seen on the items relating to data sources reported in the abstract, with a significant difference of 17.0 (9.8; 28.7 p = 0.01), review methods, reported in the abstract 35 (24.1; 49.1 p = 0.00), searching methods 18.9 (9.7; 31.6 p = 0.01), and data abstraction 18.9 (11.7; 30.9 p = 0.00). CONCLUSION: The overall quality of Cochrane SRs is fair-to-good. Although reporting quality improved on certain individual items there was no overall improvement seen with updating and methodological quality remained unchanged. Further improvement of quality of reporting is possible. There is room for improvement of methodological quality as well. Authors updating reviews should address identified methodological or reporting weaknesses. We recommend to give full attention to both quality domains when updating SRs.
| S-EPMC1569863 | biostudies-other
Project description: Not available
| S-EPMC8356304 | biostudies-literature
Project description:Background and objectiveThe living systematic review (LSR) approach is based on ongoing surveillance of the literature and continual updating. Most currently available guidance documents address the conduct, reporting, publishing, and appraisal of systematic reviews (SRs), but are not suitable for LSRs per se and miss additional LSR-specific considerations. In this scoping review, we aim to systematically collate methodological guidance literature on how to conduct, report, publish, and appraise the quality of LSRs and identify current gaps in guidance.MethodsA standard scoping review methodology was used. We searched MEDLINE (Ovid), EMBASE (Ovid), and The Cochrane Library on August 28, 2021. As for searching gray literature, we looked for existing guidelines and handbooks on LSRs from organizations that conduct evidence syntheses. The screening was conducted by two authors independently in Rayyan, and data extraction was done in duplicate using a pilot-tested data extraction form in Excel. Data was extracted according to four pre-defined categories for (i) conducting, (ii) reporting, (iii) publishing, and (iv) appraising LSRs. We mapped the findings by visualizing overview tables created in Microsoft Word.ResultsOf the 21 included papers, methodological guidance was found in 17 papers for conducting, in six papers for reporting, in 15 papers for publishing, and in two papers for appraising LSRs. Some of the identified key items for (i) conducting LSRs were identifying the rationale, screening tools, or re-revaluating inclusion criteria. Identified items of (ii) the original PRISMA checklist included reporting the registration and protocol, title, or synthesis methods. For (iii) publishing, there was guidance available on publication type and frequency or update trigger, and for (iv) appraising, guidance on the appropriate use of bias assessment or reporting funding of included studies was found. Our search revealed major evidence gaps, particularly for guidance on certain PRISMA items such as reporting results, discussion, support and funding, and availability of data and material of a LSR.ConclusionImportant evidence gaps were identified for guidance on how to report in LSRs and appraise their quality. Our findings were applied to inform and prepare a PRISMA 2020 extension for LSR.
| S-EPMC10722674 | biostudies-literature
Project description:BACKGROUND:A growing body of primary study and systematic review literature evaluates interventions and phenomena in applied and health psychology. Reviews of reviews (i.e., meta-reviews) systematically synthesise and utilise this vast and often overwhelming literature; yet, currently there are few practical guidelines for meta-review authors to follow. OBJECTIVE:The aim of this article is to provide an overview of the best practice guidelines for all research synthesis and to detail additional specific considerations and methodological details for the best practice of conducting a rigorous meta-review. METHODS:This article provides readers with six systematic and practical steps along with accompanying examples to address with rigor the unique challenges that arise when authors familiar with systematic review methods begin a meta-review: (a) detailing a clear scope, (b) identifying synthesis literature through strategic searches, (c) considering datedness of the literature, (d) addressing overlap among included reviews, (e) choosing and applying review quality tools, and (f) appropriate options for handling the synthesis and reporting of the vast amount of data collected in a meta-review. CONCLUSIONS:We have curated best practice recommendations and practical tips for conducting a meta-review. We anticipate that assessments of meta-review quality will ultimately formalise best-method guidelines.
| S-EPMC6819213 | biostudies-literature
Project description:BackgroundBefore considering whether to use a multivariable (diagnostic or prognostic) prediction model, it is essential that its performance be evaluated in data that were not used to develop the model (referred to as external validation). We critically appraised the methodological conduct and reporting of external validation studies of multivariable prediction models.MethodsWe conducted a systematic review of articles describing some form of external validation of one or more multivariable prediction models indexed in PubMed core clinical journals published in 2010. Study data were extracted in duplicate on design, sample size, handling of missing data, reference to the original study developing the prediction models and predictive performance measures.Results11,826 articles were identified and 78 were included for full review, which described the evaluation of 120 prediction models. in participant data that were not used to develop the model. Thirty-three articles described both the development of a prediction model and an evaluation of its performance on a separate dataset, and 45 articles described only the evaluation of an existing published prediction model on another dataset. Fifty-seven percent of the prediction models were presented and evaluated as simplified scoring systems. Sixteen percent of articles failed to report the number of outcome events in the validation datasets. Fifty-four percent of studies made no explicit mention of missing data. Sixty-seven percent did not report evaluating model calibration whilst most studies evaluated model discrimination. It was often unclear whether the reported performance measures were for the full regression model or for the simplified models.ConclusionsThe vast majority of studies describing some form of external validation of a multivariable prediction model were poorly reported with key details frequently not presented. The validation studies were characterised by poor design, inappropriate handling and acknowledgement of missing data and one of the most key performance measures of prediction models i.e. calibration often omitted from the publication. It may therefore not be surprising that an overwhelming majority of developed prediction models are not used in practice, when there is a dearth of well-conducted and clearly reported (external validation) studies describing their performance on independent participant data.
| S-EPMC3999945 | biostudies-literature
Project description:Poor reporting can lead to inadequate presentation of data, confusion regarding research methodology used, selective reporting of results, and other misinformation regarding health research. One of the most recent attempts to improve quality of reporting comes from the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Group, which makes recommendations for the reporting of protocols. In this report, we present a protocol for a systematic review of published surgical randomised controlled trial (RCT) protocols, with the purpose of assessing the reporting quality and completeness of the statistical aspects.We will include all published protocols of randomised trials that investigate surgical interventions. We will search MEDLINE, EMBASE, and CENTRAL for relevant studies. Author pairs will independently review all titles, abstracts, and full texts identified by the literature search, and extract data using a structured data extraction form. We will extract the following: year of publication, country, sample size, description of study population, description of intervention and control, primary outcome, important methodological qualities, and quality of reporting of planned statistical methods based on the SPIRIT guidelines.The results of this review will demonstrate the quality of statistical reporting of published surgical RCT protocols. This knowledge will inform recommendations to surgeons, researchers, journal editors and peer reviewers, and other knowledge users that focus on common deficiencies in reporting and how to rectify them. Ethics approval for this study is not required. We will disseminate the results of this review in peer-reviewed publications and conference presentations, and at a doctoral independent study of oral defence.
| S-EPMC4893854 | biostudies-other
Project description:The methodological and reporting quality of burn-specific systematic reviews has not been established. The aim of this study was to evaluate the methodological quality of systematic reviews in burn care management. Computerised searches were performed in Ovid MEDLINE, Ovid EMBASE and The Cochrane Library through to February 2016 for systematic reviews relevant to burn care using medical subject and free-text terms such as 'burn', 'systematic review' or 'meta-analysis'. Additional studies were identified by hand-searching five discipline-specific journals. Two authors independently screened papers, extracted and evaluated methodological quality using the 11-item A Measurement Tool to Assess Systematic Reviews (AMSTAR) tool and reporting quality using the 27-item Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist. Characteristics of systematic reviews associated with methodological and reporting quality were identified. Descriptive statistics and linear regression identified features associated with improved methodological quality. A total of 60 systematic reviews met the inclusion criteria. Six of the 11 AMSTAR items reporting on 'a priori' design, duplicate study selection, grey literature, included/excluded studies, publication bias and conflict of interest were reported in less than 50% of the systematic reviews. Of the 27 items listed for PRISMA, 13 items reporting on introduction, methods, results and the discussion were addressed in less than 50% of systematic reviews. Multivariable analyses showed that systematic reviews associated with higher methodological or reporting quality incorporated a meta-analysis (AMSTAR regression coefficient 2.1; 95% CI: 1.1, 3.1; PRISMA regression coefficient 6·3; 95% CI: 3·8, 8·7) were published in the Cochrane library (AMSTAR regression coefficient 2·9; 95% CI: 1·6, 4·2; PRISMA regression coefficient 6·1; 95% CI: 3·1, 9·2) and included a randomised control trial (AMSTAR regression coefficient 1·4; 95%CI: 0·4, 2·4; PRISMA regression coefficient 3·4; 95% CI: 0·9, 5·8). The methodological and reporting quality of systematic reviews in burn care requires further improvement with stricter adherence by authors to the PRISMA checklist and AMSTAR tool.
| S-EPMC7949759 | biostudies-literature
Project description:BackgroundSystematic reviews can serve as a tool in translation of basic life sciences research from laboratory to human research and healthcare. The extent to which reviews of animal research are systematic and unbiased is not known.MethodsWe searched, without language restrictions, Medline, Embase, bibliographies of known reviews (1996-2004) and contacted experts to identify citations of reviews of basic science literature which, as a minimum, performed search of a publicly available resource. From these we identified reviews of animal studies where laboratory variables were measured or where treatments were administered to live animals to examine their effects, and compared them with reviews of bench studies in which human or animal tissues, cell systems or organ preparations were examined in laboratories to better understand mechanisms of diseases.ResultsSystematic reviews of animal studies often lacked methodological features such as specification of a testable hypothesis (9/30, 30%); literature search without language restriction (8/30, 26.6%); assessment of publication bias (5/30, 16.6%), study validity (15/30, 50%) and heterogeneity (10/30, 33.3%); and meta-analysis for quantitative synthesis (12/30, 40%). Compared to reviews of bench studies, they were less prone to bias as they specified the question (96.6% vs. 80%, p = 0.04), searched multiple databases (60% vs. 26.6%, p = 0.01), assessed study quality (50% vs. 20%, p = 0.01), and explored heterogeneity (33.3% vs. 2.2%, p = 0.001) more often.ConclusionThere seems to be a gradient of frequency of methodological weaknesses among reviews: Attempted systematic reviews of whole animal research tend to be better than those of bench studies, though compared to systematic reviews of human clinical trials they are apparently poorer. There is a need for rigour when reviewing animal research.
| S-EPMC1435907 | biostudies-literature