Errors in search strategies used in systematic reviews and their effects on information retrieval.
Ontology highlight
ABSTRACT:
SUBMITTER: Salvador-Olivan JA
PROVIDER: S-EPMC6466507 | biostudies-literature | 2019 Apr
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundHealthcare information professionals play a key role in closing the knowledge gap between medical research and clinical practice. Their work involves meticulous searching of literature databases using complex search strategies that can consist of hundreds of keywords, operators, and ontology terms. This process is prone to error and can lead to inefficiency and bias if performed incorrectly.ObjectiveThe aim of this study was to investigate the search behavior of healthcare information professionals, uncovering their needs, goals, and requirements for information retrieval systems.MethodsA survey was distributed to healthcare information professionals via professional association email discussion lists. It investigated the search tasks they undertake, their techniques for search strategy formulation, their approaches to evaluating search results, and their preferred functionality for searching library-style databases. The popular literature search system PubMed was then evaluated to determine the extent to which their needs were met.ResultsThe 107 respondents indicated that their information retrieval process relied on the use of complex, repeatable, and transparent search strategies. On average it took 60 minutes to formulate a search strategy, with a search task taking 4 hours and consisting of 15 strategy lines. Respondents reviewed a median of 175 results per search task, far more than they would ideally like (100). The most desired features of a search system were merging search queries and combining search results.ConclusionsHealthcare information professionals routinely address some of the most challenging information retrieval problems of any profession. However, their needs are not fully supported by current literature search systems and there is demand for improved functionality, in particular regarding the development and management of search strategies.
| S-EPMC5643841 | biostudies-literature
Project description:ObjectivesDeveloping a search strategy for use in a systematic review is a time-consuming process requiring construction of detailed search strings using complicated syntax, followed by iterative fine-tuning and trial-and-error testing of these strings in online biomedical search engines.MethodsBuilding upon limitations of existing online-only search builders, a user-friendly computer-based tool was created to expedite search strategy development as part of production of a systematic review.ResultsSearch Builder 1.0 is a Microsoft Excel®-based tool that automatically assembles search strategy text strings for PubMed (www.pubmed.com) and Embase (www.embase.com), based on a list of user-defined search terms and preferences. With the click of a button, Search Builder 1.0 automatically populates the syntax needed for functional search strings, and copies the string to the clipboard for pasting into Pubmed or Embase. The offline file-based interface of Search Builder 1.0 also allows for searches to be easily shared and saved for future reference.ConclusionsThis novel, user-friendly tool can save considerable time and streamline a cumbersome step in the systematic review process.
| S-EPMC4848107 | biostudies-literature
Project description:ObjectivesThe review evaluated studies of electronic database search strategies designed to retrieve adverse effects data for systematic reviews.MethodsStudies of adverse effects were located in ten databases as well as by checking references, hand-searching, searching citations, and contacting experts. Two reviewers screened the retrieved records for potentially relevant papers.ResultsFive thousand three hundred thirteen citations were retrieved, yielding 19 studies designed to develop or evaluate adverse effect filters, of which 3 met the inclusion criteria. All 3 studies identified highly sensitive search strategies capable of retrieving over 95% of relevant records. However, 1 study did not evaluate precision, while the level of precision in the other 2 studies ranged from 0.8% to 2.8%. Methodological issues in these papers included the relatively small number of records, absence of a validation set of records for testing, and limited evaluation of precision.ConclusionsThe results indicate the difficulty of achieving highly sensitive searches for information on adverse effects with a reasonable level of precision. Researchers who intend to locate studies on adverse effects should allow for the amount of resources and time required to conduct a highly sensitive search.
| S-EPMC2670220 | biostudies-literature
Project description:To develop optimal search strategies in Medline for retrieving systematic reviews.Analytical survey.161 journals published in 2000 indexed in Medline.The sensitivity, specificity, and precision of retrieval of systematic reviews of 4862 unique terms in 782,485 combinations of one to five terms were determined by comparison with a hand search of all articles (the criterion standard) in 161 journals published during 2000 (49,028 articles).Only 753 (1.5%) of the 49,028 articles were systematic reviews. The most sensitive strategy included five terms and had a sensitivity of 99.9% (95% confidence interval 99.6% to 100%) and a specificity of 52% (51.6% to 52.5%). The strategy that best minimised the difference between sensitivity and specificity had a sensitivity of 98% (97% to 99%) and specificity of 90.8% (90.5% to 91.1%). Highest precision for multiterm strategies, 57% (54% to 60%), was achieved at a sensitivity of 71% (68% to 74%). The term "cochrane database of systematic reviews.jn." was the most precise single term search strategy (sensitivity of 56% (52% to 60%) and precision of 96% (94% to 98%)). These strategies are available through the "limit" screen of Ovid's search interface for Medline.Systematic reviews can be retrieved from Medline with close to perfect sensitivity or specificity, or with high precision, by using empirical search strategies.
| S-EPMC543864 | biostudies-other
Project description:Rigorous evidence identification is essential for systematic reviews and meta-analyses (evidence syntheses) because the sample selection of relevant studies determines a review's outcome, validity, and explanatory power. Yet, the search systems allowing access to this evidence provide varying levels of precision, recall, and reproducibility and also demand different levels of effort. To date, it remains unclear which search systems are most appropriate for evidence synthesis and why. Advice on which search engines and bibliographic databases to choose for systematic searches is limited and lacking systematic, empirical performance assessments. This study investigates and compares the systematic search qualities of 28 widely used academic search systems, including Google Scholar, PubMed, and Web of Science. A novel, query-based method tests how well users are able to interact and retrieve records with each system. The study is the first to show the extent to which search systems can effectively and efficiently perform (Boolean) searches with regards to precision, recall, and reproducibility. We found substantial differences in the performance of search systems, meaning that their usability in systematic searches varies. Indeed, only half of the search systems analyzed and only a few Open Access databases can be recommended for evidence syntheses without adding substantial caveats. Particularly, our findings demonstrate why Google Scholar is inappropriate as principal search system. We call for database owners to recognize the requirements of evidence synthesis and for academic journals to reassess quality requirements for systematic reviews. Our findings aim to support researchers in conducting better searches for better evidence synthesis.
| S-EPMC7079055 | biostudies-literature
Project description:BackgroundAlthough the methods for conducting systematic reviews of efficacy are well established, there is much less guidance on how systematic reviews of adverse effects should be performed.MethodsIn order to determine where methodological research is most needed to improve systematic reviews of adverse effects of health care interventions, we conducted a descriptive analysis of systematic reviews published between 1994 and 2005. We searched the Database of Abstracts of Reviews of Effects (DARE) and The Cochrane Database of Systematic Reviews (CDSR) to identify systematic reviews in which the primary outcome was an adverse effect or effects. We then extracted data on many of the elements of the systematic review process including: types of interventions studied, adverse effects of interest, resources searched, search strategies, data sources included in reviews, quality assessment of primary data, nature of the data analysis, and source of funding.Results256 reviews were included in our analysis, of which the majority evaluated drug interventions and pre-specified the adverse effect or effects of interest. A median of 3 resources were searched for each review and very few reviews (13/256) provided sufficient information to reproduce their search strategies. Although more than three quarters (185/243) of the reviews sought to include data from sources other than randomised controlled trials, fewer than half (106/256) assessed the quality of the studies that were included. Data were pooled quantitatively in most of the reviews (165/256) but heterogeneity was not always considered. Less than half (123/256) of the reviews reported on the source of funding.ConclusionThere is an obvious need to improve the methodology and reporting of systematic reviews of adverse effects. The methodology around identification and quality assessment of primary data is the main concern.
| S-EPMC1402311 | biostudies-literature
Project description:Manual searches are supplemental approaches to database searches to identify additional primary studies for systematic reviews. The authors argue that these manual approaches, in particular hand-searching and perusing reference lists, are often considered the same yet lead to different outcomes.We conducted a PubMed search for systematic reviews in the top 10 dermatology journals (January 2006-January 2016). After screening, the final sample comprised 292 reviews. Statements related to manual searches were extracted from each review and categorized by the primary and secondary authors. Each statement was categorized as either "Search of Reference List," "Hand Search," "Both," or "Unclear."Of the 292 systematic reviews included in our sample, 143 reviews (48.97%) did not report a hand-search or scan of reference lists. One-hundred thirty-six reviews (46.58%) reported searches of reference lists, while 4 reviews (1.37%) reported systematic hand-searches. Three reviews (1.03%) reported use of both hand-searches and scanning reference lists. Six reviews (2.05%) were classified as unclear due to vague wording.Authors of systematic reviews published in dermatology journals in our study sample scanned reference lists more frequently than they conducted hand-searches, possibly contributing to biased search outcomes. We encourage systematic reviewers to routinely practice hand-searching in order to minimize bias.
| S-EPMC5079492 | biostudies-other
Project description:PurposeThe Electronic Medical Record Search Engine (EMERSE) is a software tool built to aid research spanning cohort discovery, population health, and data abstraction for clinical trials. EMERSE is now live at three academic medical centers, with additional sites currently working on implementation. In this report, we describe how EMERSE has been used to support cancer research based on a variety of metrics.MethodsWe identified peer-reviewed publications that used EMERSE through online searches as well as through direct e-mails to users based on audit logs. These logs were also used to summarize use at each of the three sites. Search terms for two of the sites were characterized using the natural language processing tool MetaMap to determine to which semantic types the terms could be mapped.ResultsWe identified a total of 326 peer-reviewed publications that used EMERSE through August 2019, although this is likely an underestimation of the true total based on the use log analysis. Oncology-related research comprised nearly one third (n = 105; 32.2%) of all research output. The use logs showed that EMERSE had been used by multiple people at each site (nearly 3,500 across all three) who had collectively logged into the system > 100,000 times. Many user-entered search queries could not be mapped to a semantic type, but the most common semantic type for terms that did match was "disease or syndrome," followed by "pharmacologic substance."ConclusionEMERSE has been shown to be a valuable tool for supporting cancer research. It has been successfully deployed at other sites, despite some implementation challenges unique to each deployment environment.
| S-EPMC7265780 | biostudies-literature
Project description:BackgroundWhile writing a scoping review, we needed to update our search strategy. We wanted to capture articles generated by our additional search terms and articles published since our original search. Simultaneously, we strove to optimize project resources by not rescreening articles that had been captured in our original results.Case presentationIn response, we created Open Update Re-run Deduplicate (OUR2D2), a computer application that allows the user to compare search results from a variety of library databases. OUR2D2 supports extensible markup language (XML) files from EndNote and comma-separated values (CSV) files using article titles for comparisons. We conducted unit tests to ensure appropriate functionality as well as accurate data extraction and analysis. We tested OUR2D2 by comparing original and updated search results from PubMed, Embase, Clarivate Web of Science, CINAHL, Scopus, ProQuest Dissertation and Theses, and Lens and estimate that this application saved twenty-one hours of work during the screening process.ConclusionsOUR2D2 could be useful for individuals seeking to update literature review strategies across fields without rescreening articles from previous searches. Because the OUR2D2 source code is freely available with a permissive license, we recommend this application for researchers conducting literature reviews who need to update their search results over time, want a powerful and flexible analysis framework, and may not have access to paid subscription tools.
| S-EPMC8270383 | biostudies-literature
Project description:BackgroundTo evaluate the effectiveness of audit and communication strategies to reduce diagnostic errors made by clinicians.MethodsMEDLINE complete, CINHAL complete, EMBASE, PSNet and Google Advanced. Electronic and manual search of articles on audit systems and communication strategies or interventions, searched for papers published between January 1990 and April 2017. We included studies with interventions implemented by clinicians in a clinical environment with real patients.ResultsA total of 2431 articles were screened of which 26 studies met inclusion criteria. Data extraction was conducted by two groups, each group comprising two independent reviewers. Articles were classified by communication (6) or audit strategies (20) to reduce diagnostic error in clinical settings. The most common interventions were delivered as technology-based systems n = 16 (62%) and within an acute care setting n = 15 (57%). Nine studies reported randomised controlled trials. Three RCT studies on communication interventions and 3 RCTs on audit strategies found the interventions to be effective in reducing diagnostic errors.ConclusionDespite numerous studies on interventions targeting diagnostic errors, our analyses revealed limited evidence on interventions being practically used in clinical settings and a bias of studies originating from the US (n = 19, 73% of included studies). There is some evidence that trigger algorithms, including computer based and alert systems, may reduce delayed diagnosis and improve diagnostic accuracy. In trauma settings, strategies such as additional patient review (e.g. trauma teams) reduced missed diagnosis and in radiology departments review strategies such as team meetings and error documentation may reduce diagnostic error rates over time.Trial registrationThe systematic review was registered in the PROSPERO database under registration number CRD42017067056 .
| S-EPMC6716834 | biostudies-literature