Dataset Information

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.

ABSTRACT: BACKGROUND:Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological systematic review was to analyse the evidence available on whether single screening is equivalent to double screening in the screening process conducted in systematic reviews. METHODS:We searched Medline, PubMed and the Cochrane Methodology Register (last search 10/2018). We also used supplementary search techniques and sources ("similar articles" function in PubMed, conference abstracts and reference lists). We included all evaluations comparing single with double screening. Data were summarized in a structured, narrative way. RESULTS:The 4 evaluations included investigated a total of 23 single screenings (12 sets for screening involving 9 reviewers). The median proportion of missed studies was 5% (range 0 to 58%). The median proportion of missed studies was 3% for the 6 experienced reviewers (range: 0 to 21%) and 13% for the 3 reviewers with less experience (range: 0 to 58%). The impact of missing studies on the findings of meta-analyses had been reported in 2 evaluations for 7 single screenings including a total of 18,148 references. In 3 of these 7 single screenings - all conducted by the same reviewer (with less experience) - the findings would have changed substantially. The remaining 4 of these 7 screenings were conducted by experienced reviewers and the missing studies had no impact or a negligible on the findings of the meta-analyses. CONCLUSIONS:Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed.

SUBMITTER: Waffenschmidt S

PROVIDER: S-EPMC6599339 | biostudies-literature | 2019 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.

Waffenschmidt Siw S Knelangen Marco M Sieben Wiebke W Bühn Stefanie S Pieper Dawid D

BMC medical research methodology 20190628 1

<h4>Background</h4>Stringent requirements exist regarding the transparency of the study selection process and the reliability of results. A 2-step selection process is generally recommended; this is conducted by 2 reviewers independently of each other (conventional double-screening). However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget. The aim of the following methodological system ...[more]

PMID: 31253092

Similar Datasets

Project description:BackgroundThe screening process for systematic reviews and meta-analyses in medical research is a labor-intensive and time-consuming task. While machine learning and deep learning have been applied to facilitate this process, these methods often require training data and user annotation. This study aims to assess the efficacy of ChatGPT, a large language model based on the Generative Pretrained Transformers (GPT) architecture, in automating the screening process for systematic reviews in radiology without the need for training data.MethodsA prospective simulation study was conducted between May 2nd and 24th, 2023, comparing ChatGPT's performance in screening abstracts against that of general physicians (GPs). A total of 1198 abstracts across three subfields of radiology were evaluated. Metrics such as sensitivity, specificity, positive and negative predictive values (PPV and NPV), workload saving, and others were employed. Statistical analyses included the Kappa coefficient for inter-rater agreement, ROC curve plotting, AUC calculation, and bootstrapping for p-values and confidence intervals.ResultsChatGPT completed the screening process within an hour, while GPs took an average of 7-10 days. The AI model achieved a sensitivity of 95% and an NPV of 99%, slightly outperforming the GPs' sensitive consensus (i.e., including records if at least one person includes them). It also exhibited remarkably low false negative counts and high workload savings, ranging from 40 to 83%. However, ChatGPT had lower specificity and PPV compared to human raters. The average Kappa agreement between ChatGPT and other raters was 0.27.ConclusionsChatGPT shows promise in automating the article screening phase of systematic reviews, achieving high sensitivity and workload savings. While not entirely replacing human expertise, it could serve as an efficient first-line screening tool, particularly in reducing the burden on human resources. Further studies are needed to fine-tune its capabilities and validate its utility across different medical subfields.

Project description:Qualitative systematic reviews or qualitative evidence syntheses (QES) are increasingly recognised as a way to enhance the value of systematic reviews (SRs) of clinical trials. They can explain the mechanisms by which interventions, evaluated within trials, might achieve their effect. They can investigate differences in effects between different population groups. They can identify which outcomes are most important to patients, carers, health professionals and other stakeholders. QES can explore the impact of acceptance, feasibility, meaningfulness and implementation-related factors within a real world setting and thus contribute to the design and further refinement of future interventions. To produce valid, reliable and meaningful QES requires systematic identification of relevant qualitative evidence. Although the methodologies of QES, including methods for information retrieval, are well-documented, little empirical evidence exists to inform their conduct and reporting.This structured methodological overview examines papers on searching for qualitative research identified from the Cochrane Qualitative and Implementation Methods Group Methodology Register and from citation searches of 15 key papers.A single reviewer reviewed 1299 references. Papers reporting methodological guidance, use of innovative methodologies or empirical studies of retrieval methods were categorised under eight topical headings: overviews and methodological guidance, sampling, sources, structured questions, search procedures, search strategies and filters, supplementary strategies and standards.This structured overview presents a contemporaneous view of information retrieval for qualitative research and identifies a future research agenda. This review concludes that poor empirical evidence underpins current information practice in information retrieval of qualitative research. A trend towards improved transparency of search methods and further evaluation of key search procedures offers the prospect of rapid development of search methods.

Project description:ObjectiveTo examine whether network meta-analyses, increasingly used to assess comparative effectiveness of healthcare interventions, follow the key methodological recommendations for reporting and conduct of systematic reviews.DesignMethodological systematic review of reports of network meta-analyses.Data sourcesCochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Medline, and Embase, searched from inception to 12 July 2012.Review methodsAll network meta-analyses comparing clinical efficacy of three or more interventions based on randomised controlled trials, excluding meta-analyses with an open loop network of three interventions. We assessed the reporting of general characteristics and key methodological components of the systematic review process using two composite outcomes. For some components, if reporting was adequate, we assessed their conduct quality.ResultsOf 121 network meta-analyses covering a wide range of medical areas, 100 (83%) assessed pharmacological interventions and 11 (9%) non-pharmacological interventions; 56 (46%) were published in journals with a high impact factor. The electronic search strategy for each database was not reported in 88 (73%) network meta-analyses; for 36 (30%), the primary outcome was not clearly identified. Overall, 61 (50%) network meta-analyses did not report any information regarding the assessment of risk of bias of individual studies, and 103 (85%) did not report any methods to assess the likelihood of publication bias. Overall, 87 (72%) network meta-analyses did not report the literature search, searched only one database, did not search other sources, or did not report an assessment of risk of bias of individual studies. These methodological components did not differ by publication in a general or specialty journal or by public or private funding.ConclusionsEssential methodological components of the systematic review process-conducting a literature search and assessing risk of bias of individual studies-are frequently lacking in reports of network meta-analyses, even when published in journals with high impact factors.

Project description:BACKGROUND:Single-incision laparoscopic right hemicolectomy (SILS) has long used in surgery for a long time. However, there is barely a systemic review related to the comparison between the SILS and the conventional laparoscopic right hemicolectomy (CLS) for the right colon cancer in the long term follow-up. Herein, we used the most recent articles to compare these two techniques by meta-analysis. METHODS:We searched PubMed, Web of Science, Cochrane Library and Wanfang databases to compare SILS with CLS for right colon cancer up to May 2019. The operative, postoperative, pathological and mid-term follow-up outcomes of nine studies were extracted and compared. RESULTS:A total of 1356 patients participated in 9 studies, while 653 patients were assigned to the SILS group and 703 patients were assigned to the CLS group. The patients' baselines in the SILS group were consistent with those in the CLS group. Compared to the CLS group, the SILS group had a shorter operation duration (SMD - 23.49, 95%CI - 36.71 to - 10.27, P < 0.001, chi-square = 24.11), shorter hospital stay (SMD - 0.76, 95% `CI - 1.07 to - 0.45, P < 0.001, chi-square = 9.85), less blood loss (SMD - 8.46, 95% CI - 14.59 to - 2.34; P < 0.05; chi-square = 2.26), smaller incision length (SMD - 1.60, 95% CI - 2.66 to - 0.55, P < 0.001; chi-square = 280.44), more lymph node harvested (SMD - 0.98, 95% CI - 1.79 to - 0.16, P < 0.05; chi-square = 4.61), and a longer proximal surgical edge (SMD - 0.51, 95% CI - 0.93 to - 0.09, P < 0.05; chi-square = 2.42). No significant difference was found in other indexes. After we removed a single large study, we performed another meta-analysis again. The operation duration in the SILS group was still better than that in the CLS group. CONCLUSION:SILS could be a faster and more reliable approach than CLS for the right colon cancer and could accelerate patient recovery, especially for patients with a low BMI.

Dataset Information

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.

Publications

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets