Dataset Information

Optimizing search strategies to identify randomized controlled trials in MEDLINE.

ABSTRACT: BACKGROUND: The Cochrane Highly Sensitive Search Strategy (HSSS), which contains three phases, is widely used to identify Randomized Controlled Trials (RCTs) in MEDLINE. Lefebvre and Clarke suggest that reviewers might consider using four revisions of the HSSS. The objective of this study is to validate these four revisions: combining the free text terms volunteer, crossover, versus, and the Medical Subject Heading CROSS-OVER STUDIES with the top two phases of the HSSS, respectively. METHODS: We replicated the subject search for 61 Cochrane reviews. The included studies of each review that were indexed in MEDLINE were pooled together by review and then combined with the subject search and each of the four proposed search strategies, the top two phases of the HSSS, and all three phases of the HSSS. These retrievals were used to calculate the sensitivity and precision of each of the six search strategies for each review. RESULTS: Across the 61 reviews, the search term versus combined with the top two phases of the HSSS was able to find 3 more included studies than the top two phases of the HSSS alone, or in combination with any of the other proposed search terms, but at the expense of missing 56 relevant articles that would be found if all three phases of the HSSS were used. The estimated time needed to finish a review is 1086 hours for all three phases of the HSSS, 823 hours for the strategy versus, 818 hours for the first two phases of the HSSS or any of the other three proposed strategies. CONCLUSION: This study shows that compared to the first two phases of the HSSS, adding the term versus to the top two phases of the HSSS balances the sensitivity and precision in the reviews studied here to some extent but the differences are very small. It is well known that missing relevant studies may result in bias in systematic reviews. Reviewers need to weigh the trade-offs when selecting the search strategies for identifying RCTs in MEDLINE.

SUBMITTER: Zhang L

PROVIDER: S-EPMC1488863 | biostudies-other | 2006

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Optimizing search strategies to identify randomized controlled trials in MEDLINE.

Zhang Li L Ajiferuke Isola I Sampson Margaret M

BMC medical research methodology 20060509

<h4>Background</h4>The Cochrane Highly Sensitive Search Strategy (HSSS), which contains three phases, is widely used to identify Randomized Controlled Trials (RCTs) in MEDLINE. Lefebvre and Clarke suggest that reviewers might consider using four revisions of the HSSS. The objective of this study is to validate these four revisions: combining the free text terms volunteer, crossover, versus, and the Medical Subject Heading CROSS-OVER STUDIES with the top two phases of the HSSS, respectively.<h4>M ...[more]

PMID: 16684359

Similar Datasets

Project description:BackgroundCluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ("recall") and precision of a well-known electronic search strategy ("randomized controlled trial" as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time.MethodsWe manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews.ResultsThe existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%).ConclusionsCRTs should include the phrase "cluster randomized trial" in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization.

Project description:BackgroundCluster randomized trials (CRTs) are becoming an increasingly important design. However, authors of CRTs do not always adhere to requirements to explicitly identify the design as cluster randomized in titles and abstracts, making retrieval from bibliographic databases difficult. Machine learning algorithms may improve their identification and retrieval. Therefore, we aimed to develop machine learning algorithms that accurately determine whether a bibliographic citation is a CRT report.MethodsWe trained, internally validated, and externally validated two convolutional neural networks and one support vector machine (SVM) algorithm to predict whether a citation is a CRT report or not. We exclusively used the information in an article citation, including the title, abstract, keywords, and subject headings. The algorithms' output was a probability from 0 to 1. We assessed algorithm performance using the area under the receiver operating characteristic (AUC) curves. Each algorithm's performance was evaluated individually and together as an ensemble. We randomly selected 5000 from 87,633 citations to train and internally validate our algorithms. Of the 5000 selected citations, 589 (12%) were confirmed CRT reports. We then externally validated our algorithms on an independent set of 1916 randomized trial citations, with 665 (35%) confirmed CRT reports.ResultsIn internal validation, the ensemble algorithm discriminated best for identifying CRT reports with an AUC of 98.6% (95% confidence interval: 97.8%, 99.4%), sensitivity of 97.7% (94.3%, 100%), and specificity of 85.0% (81.8%, 88.1%). In external validation, the ensemble algorithm had an AUC of 97.8% (97.0%, 98.5%), sensitivity of 97.6% (96.4%, 98.6%), and specificity of 78.2% (75.9%, 80.4%)). All three individual algorithms performed well, but less so than the ensemble.ConclusionsWe successfully developed high-performance algorithms that identified whether a citation was a CRT report with high sensitivity and moderately high specificity. We provide open-source software to facilitate the use of our algorithms in practice.

Project description:BACKGROUND: The identification of health services research in databases such as PubMed/Medline is a cumbersome task. This task becomes even more difficult if the field of interest involves the use of diverse methods and data sources, as is the case with nurse staffing research. This type of research investigates the association between nurse staffing parameters and nursing and patient outcomes. A comprehensively developed search strategy may help identify nurse staffing research in PubMed/Medline. METHODS: A set of relevant references in PubMed/Medline was identified by means of three systematic reviews. This development set was used to detect candidate free-text and MeSH terms. The frequency of these terms was compared to a random sample from PubMed/Medline in order to identify terms specific to nurse staffing research, which were then used to develop a sensitive, precise and balanced search strategy. To determine their precision, the newly developed search strategies were tested against a) the pool of relevant references extracted from the systematic reviews, b) a reference set identified from an electronic journal screening, and c) a sample from PubMed/Medline. Finally, all newly developed strategies were compared to PubMed's Health Services Research Queries (PubMed's HSR Queries). RESULTS: The sensitivities of the newly developed search strategies were almost 100% in all of the three test sets applied; precision ranged from 6.1% to 32.0%. PubMed's HSR queries were less sensitive (83.3% to 88.2%) than the new search strategies. Only minor differences in precision were found (5.0% to 32.0%). CONCLUSIONS: As with other literature on health services research, nurse staffing studies are difficult to identify in PubMed/Medline. Depending on the purpose of the search, researchers can choose between high sensitivity and retrieval of a large number of references or high precision, i.e. and an increased risk of missing relevant references, respectively. More standardized terminology (e.g. by consistent use of the term "nurse staffing") could improve the precision of future searches in this field. Empirically selected search terms can help to develop effective search strategies. The high consistency between all test sets confirmed the validity of our approach.

Dataset Information

Optimizing search strategies to identify randomized controlled trials in MEDLINE.

Publications

Optimizing search strategies to identify randomized controlled trials in MEDLINE.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets