Dataset Information

Searching Embase and MEDLINE by using only major descriptors or title and abstract fields: a prospective exploratory study.

ABSTRACT:

Background

Researchers performing systematic reviews (SRs) must carefully consider the relevance of thousands of citations retrieved from bibliographic database searches, the majority of which will be excluded later on close inspection. Well-developed bibliographic searches are generally created with thesaurus or index terms in combination with keywords found in the title and/or abstract fields of citation records. Records in the bibliographic database Embase contain many more thesaurus terms than MEDLINE. Here, we aim to examine how limiting searches to major thesaurus terms (in MEDLINE called focus terms) in Embase and MEDLINE as well as limiting to words in the title and abstract fields of those databases affects the overall recall of SR searches.

Methods

To examine the impact of using search techniques aimed at higher precision, we analyzed previously completed SRs and focused our original searches to major thesaurus terms or terms in title and/or abstract only in Embase.com or in Embase.com and MEDLINE (Ovid) combined. We examined the total number of search results in both Embase and MEDLINE and checked whether included references were retrieved by these more focused approaches.

Results

For 73 SRs, we limited Embase searches to major terms only while keeping the search in MEDLINE and other databases such as Web of Science as they were. The overall search yield (or total number of search results) was reduced by 8%. Six reviews (9%) lost more than 5% of the relevant references. Limiting Embase and MEDLINE to major thesaurus terms, the number of references was 13% lower. For 15% of the reviews, the loss of relevant references was more than 5%. Searching Embase for title and abstract caused a loss of more than 5% in 16 reviews (22%), while limiting Embase and MEDLINE that way this happened in 24 reviews (33%).

Conclusions

Of the four search options, two options substantially reduced the overall search yield. However, this also resulted in a greater chance of losing relevant references, even though many references were still found in other databases such as Web of Science.

SUBMITTER: Bramer WM

PROVIDER: S-EPMC6247690 | biostudies-literature | 2018 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Searching Embase and MEDLINE by using only major descriptors or title and abstract fields: a prospective exploratory study.

Bramer Wichor M WM Giustini Dean D Kleijnen Jos J Franco Oscar H OH

Systematic reviews 20181120 1

<h4>Background</h4>Researchers performing systematic reviews (SRs) must carefully consider the relevance of thousands of citations retrieved from bibliographic database searches, the majority of which will be excluded later on close inspection. Well-developed bibliographic searches are generally created with thesaurus or index terms in combination with keywords found in the title and/or abstract fields of citation records. Records in the bibliographic database Embase contain many more thesaurus ...[more]

PMID: 30458825

Similar Datasets

Project description:BackgroundCluster randomized trials (CRTs) are becoming an increasingly important design. However, authors of CRTs do not always adhere to requirements to explicitly identify the design as cluster randomized in titles and abstracts, making retrieval from bibliographic databases difficult. Machine learning algorithms may improve their identification and retrieval. Therefore, we aimed to develop machine learning algorithms that accurately determine whether a bibliographic citation is a CRT report.MethodsWe trained, internally validated, and externally validated two convolutional neural networks and one support vector machine (SVM) algorithm to predict whether a citation is a CRT report or not. We exclusively used the information in an article citation, including the title, abstract, keywords, and subject headings. The algorithms' output was a probability from 0 to 1. We assessed algorithm performance using the area under the receiver operating characteristic (AUC) curves. Each algorithm's performance was evaluated individually and together as an ensemble. We randomly selected 5000 from 87,633 citations to train and internally validate our algorithms. Of the 5000 selected citations, 589 (12%) were confirmed CRT reports. We then externally validated our algorithms on an independent set of 1916 randomized trial citations, with 665 (35%) confirmed CRT reports.ResultsIn internal validation, the ensemble algorithm discriminated best for identifying CRT reports with an AUC of 98.6% (95% confidence interval: 97.8%, 99.4%), sensitivity of 97.7% (94.3%, 100%), and specificity of 85.0% (81.8%, 88.1%). In external validation, the ensemble algorithm had an AUC of 97.8% (97.0%, 98.5%), sensitivity of 97.6% (96.4%, 98.6%), and specificity of 78.2% (75.9%, 80.4%)). All three individual algorithms performed well, but less so than the ensemble.ConclusionsWe successfully developed high-performance algorithms that identified whether a citation was a CRT report with high sensitivity and moderately high specificity. We provide open-source software to facilitate the use of our algorithms in practice.

Project description:BACKGROUND:Systematic reviews often investigate the effectiveness of interventions for one sex. However, identifying interventions with data presented according to the sex of study participants can be challenging due to suboptimal indexing in bibliographic databases and poor reporting in titles and abstracts. The purposes of this study were to develop a highly sensitive search filter to identify literature relevant to men's health and to assess the performance of a range of sex-specific search terms used individually and in various combinations. METHODS:Comprehensive electronic searches were undertaken across a range of databases to inform a series of systematic reviews investigating obesity management for men. The included studies formed a reference standard set. A set of sex-specific search terms, identified from database-specific controlled vocabularies and from natural language used in the titles and abstracts of relevant papers, was investigated in MEDLINE and Embase. Sensitivity, precision, number needed to read (NNR) and percent reduction in results compared to searching without sex-specific terms were calculated. RESULTS:The reference standard set comprised 57 papers in MEDLINE and 63 in Embase. Seven sex-specific search terms were identified. Searching without sex-specific terms returned 31,897 results in MEDLINE and 37,351 in Embase and identified 84% (MEDLINE) and 83% (Embase) of the reference standard sets. The best performing individual sex-specific term achieved 100%/98% sensitivity (MEDLINE/Embase), NNR 544/609 (MEDLINE/Embase) and reduced the number of results by 18%/17% (MEDLINE/Embase), relative to searching without sex-specific terms. The best performing filter, compromising different combinations of controlled vocabulary terms and natural language, achieved higher sensitivity (MEDLINE and Embase 100%), greater reduction in number of results (MEDLINE/Embase 24%/20%) and greater reduction in NNR (MEDLINE/Embase 506/578) than the best performing individual sex-specific term. CONCLUSIONS:The proposed MEDLINE and Embase filters achieved high sensitivity and a reduction in the number of search results and NNR, indicating that they are useful tools for efficient, comprehensive literature searching but their performance is partially dependent on the appropriate use of database controlled vocabularies and index terms.

Project description:BACKGROUND: The production of high quality systematic reviews requires rigorous methods that are time-consuming and resource intensive. Citation screening is a key step in the systematic review process. An opportunity to improve the efficiency of systematic review production involves the use of non-expert groups and new technologies for citation screening. We performed a pilot study of citation screening by medical students using four screening methods and compared students' performance to experienced review authors. METHODS: The aims of this pilot randomised controlled trial were to provide preliminary data on the accuracy of title and abstract screening by medical students, and on the effect of screening modality on screening accuracy and efficiency. Medical students were randomly allocated to title and abstract screening using one of the four modalities and required to screen 650 citations from a single systematic review update. The four screening modalities were a reference management software program (EndNote), Paper, a web-based systematic review workflow platform (ReGroup) and a mobile screening application (Screen2Go). Screening sensitivity and specificity were analysed in a complete case analysis using a chi-squared test and Kruskal-Wallis rank sum test according to screening modality and compared to a final set of included citations selected by expert review authors. RESULTS: Sensitivity of medical students' screening decisions ranged from 46.7% to 66.7%, with students using the web-based platform performing significantly better than the paper-based group. Specificity ranged from 93.2% to 97.4% with the lowest specificity seen with the web-based platform. There was no significant difference in performance between the other three modalities. CONCLUSIONS: Medical students are a feasible population to engage in citation screening. Future studies should investigate the effect of incentive systems, training and support and analytical methods on screening performance. SYSTEMATIC REVIEW REGISTRATION: Cochrane Database CD001048.

Project description:BackgroundWe evaluated the benefits and risks of using the Abstrackr machine learning (ML) tool to semi-automate title-abstract screening and explored whether Abstrackr's predictions varied by review or study-level characteristics.MethodsFor a convenience sample of 16 reviews for which adequate data were available to address our objectives (11 systematic reviews and 5 rapid reviews), we screened a 200-record training set in Abstrackr and downloaded the relevance (relevant or irrelevant) of the remaining records, as predicted by the tool. We retrospectively simulated the liberal-accelerated screening approach. We estimated the time savings and proportion missed compared with dual independent screening. For reviews with pairwise meta-analyses, we evaluated changes to the pooled effects after removing the missed studies. We explored whether the tool's predictions varied by review and study-level characteristics.ResultsUsing the ML-assisted liberal-accelerated approach, we wrongly excluded 0 to 3 (0 to 14%) records that were included in the final reports, but saved a median (IQR) 26 (9, 42) h of screening time. One missed study was included in eight pairwise meta-analyses in one systematic review. The pooled effect for just one of those meta-analyses changed considerably (from MD (95% CI) - 1.53 (- 2.92, - 0.15) to - 1.17 (- 2.70, 0.36)). Of 802 records in the final reports, 87% were correctly predicted as relevant. The correctness of the predictions did not differ by review (systematic or rapid, P = 0.37) or intervention type (simple or complex, P = 0.47). The predictions were more often correct in reviews with multiple (89%) vs. single (83%) research questions (P = 0.01), or that included only trials (95%) vs. multiple designs (86%) (P = 0.003). At the study level, trials (91%), mixed methods (100%), and qualitative (93%) studies were more often correctly predicted as relevant compared with observational studies (79%) or reviews (83%) (P = 0.0006). Studies at high or unclear (88%) vs. low risk of bias (80%) (P = 0.039), and those published more recently (mean (SD) 2008 (7) vs. 2006 (10), P = 0.02) were more often correctly predicted as relevant.ConclusionOur screening approach saved time and may be suitable in conditions where the limited risk of missing relevant records is acceptable. Several of our findings are paradoxical and require further study to fully understand the tasks to which ML-assisted screening is best suited. The findings should be interpreted in light of the fact that the protocol was prepared for the funder, but not published a priori. Because we used a convenience sample, the findings may be prone to selection bias. The results may not be generalizable to other samples of reviews, ML tools, or screening approaches. The small number of missed studies across reviews with pairwise meta-analyses hindered strong conclusions about the effect of missed studies on the results and conclusions of systematic reviews.

Project description:BackgroundSystematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening.MethodsWe retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow.ResultsWe developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction).ConclusionArtificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML.

Project description:BackgroundSystematic reviews are vital to the pursuit of evidence-based medicine within healthcare. Screening titles and abstracts (T&Ab) for inclusion in a systematic review is an intensive, and often collaborative, step. The use of appropriate tools is therefore important. In this study, we identified and evaluated the usability of software tools that support T&Ab screening for systematic reviews within healthcare research.MethodsWe identified software tools using three search methods: a web-based search; a search of the online "systematic review toolbox"; and screening of references in existing literature. We included tools that were accessible and available for testing at the time of the study (December 2018), do not require specific computing infrastructure and provide basic screening functionality for systematic reviews. Key properties of each software tool were identified using a feature analysis adapted for this purpose. This analysis included a weighting developed by a group of medical researchers, therefore prioritising the most relevant features. The highest scoring tools from the feature analysis were then included in a user survey, in which we further investigated the suitability of the tools for supporting T&Ab screening amongst systematic reviewers working in medical research.ResultsFifteen tools met our inclusion criteria. They vary significantly in relation to cost, scope and intended user community. Six of the identified tools (Abstrackr, Colandr, Covidence, DRAGON, EPPI-Reviewer and Rayyan) scored higher than 75% in the feature analysis and were included in the user survey. Of these, Covidence and Rayyan were the most popular with the survey respondents. Their usability scored highly across a range of metrics, with all surveyed researchers (n =?6) stating that they would be likely (or very likely) to use these tools in the future.ConclusionsBased on this study, we would recommend Covidence and Rayyan to systematic reviewers looking for suitable and easy to use tools to support T&Ab screening within healthcare research. These two tools consistently demonstrated good alignment with user requirements. We acknowledge, however, the role of some of the other tools we considered in providing more specialist features that may be of great importance to many researchers.

Dataset Information

Searching Embase and MEDLINE by using only major descriptors or title and abstract fields: a prospective exploratory study.

Background

Methods

Results

Conclusions

Publications

Searching Embase and MEDLINE by using only major descriptors or title and abstract fields: a prospective exploratory study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets