Derivation and validation of a MEDLINE search strategy for research studies that use administrative data.
Ontology highlight
ABSTRACT: Objective
To derive and validate a search strategy that identifies administrative database research (ADR) in the MEDLINE database.Design
Analytical survey.Methods
We downloaded all articles published between January 1, 2008 and October 7, 2009 in 20 top journals in internal medicine, cardiovascular medicine, public health, and health services research. These were reviewed to determine whether they were ADR (in which the study cohort, exposure, or outcome was defined using electronic data created for or during the processing of patients through their health care). We used chi-squared recursive partitioning to create a search strategy that maximized sensitivity based on publication type, MeSH headings, and text words.Main outcome measures
Sensitivity and positive predictive value of the search strategy for true ADR in three samples: derivation (n=5,513); internal validation (n=2,710); and external validation (n=1,500).Results
The prevalence of ADR in the derivation, internal validation, and external validation samples was 2.6, 2.9, and 2.2 percent, respectively. The sensitivity of our search strategy in these samples was 90.9 percent (95 percent confidence interval [CI] 85.0-95.1), 88.5 percent (79.2-94.6), and 100 percent (99.3-100), respectively. The positive predictive value in these samples was 10.7 percent (9.0-12.6), 11.5 percent (9.1-14.4), and 3.3 percent (2.3-4.6), respectively.Conclusion
We derived and validated a search strategy that is highly sensitive for ADR in MEDLINE.
SUBMITTER: Van Walraven C
PROVIDER: S-EPMC3026961 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundEvidence from health services research (HSR) is currently thinly spread through many journals, making it difficult for health services researchers, managers and policy-makers to find research on clinical practice guidelines and the appropriateness, process, outcomes, cost and economics of health care services. We undertook to develop and test search terms to retrieve from the MEDLINE database HSR articles meeting minimum quality standards.MethodsThe retrieval performance of 7445 methodologic search terms and phrases in MEDLINE (the test) were compared with a hand search of the literature (the gold standard) for each issue of 68 journal titles for the year 2000 (a total of 25,936 articles). We determined sensitivity, specificity and precision (the positive predictive value) of the MEDLINE search strategies.ResultsA majority of the articles that were classified as outcome assessment, but fewer than half of those in the other categories, were considered methodologically acceptable (no methodologic criteria were applied for cost studies). Combining individual search terms to maximize sensitivity, while keeping specificity at 50% or more, led to sensitivities in the range of 88.1% to 100% for several categories (specificities ranged from 52.9% to 97.4%). When terms were combined to maximize specificity while keeping sensitivity at 50% or more, specificities of 88.8% to 99.8% were achieved. When terms were combined to maximize sensitivity and specificity while minimizing the differences between the 2 measurements, most strategies for HSR categories achieved sensitivity and specificity of at least 80%.InterpretationSensitive and specific search strategies were validated for retrieval of HSR literature from MEDLINE. These strategies have been made available for public use by the US National Library of Medicine at www.nlm.nih.gov/nichsr/hedges/search.html.
| S-EPMC524948 | biostudies-literature
Project description:BackgroundDegenerative cervical myelopathy (DCM) is a common condition with many unmet clinical needs. Pooled analysis of studies is an important tool for advancing medical understanding. This process starts with a systematic search of the literature. Identification of studies in DCM is challenged by a number of factors, including non-specific terminology and index terms. Search filters or HEDGEs, are search strings developed and validated to optimise medical literature searches. We aimed to develop a search filter for DCM for the MEDLINE database.MethodsThe diagnostic test assessment framework of a "development dataset" and seperate "validation dataset" was used. The development dataset was formed by hand searching four leading spinal journals (Spine, Journal of Neurosurgery Spine, Spinal Cord and Journal of Spinal Disorders and Techniques) in 2005 and 2010. The search filter was initially developed focusing on sensitivity and subsequently refined using NOT functions to improve specificity. One validation dataset was formed from DCM narrative and systematic review articles and the second, articles published in April of 1989, 1993, 1997, 2001, 2005, 2009, 2013 and 2017 retrieved via the search MeSH term 'Spine'. Metrics of sensitivity, specificity, precision and accuracy were used to test performance.ResultsHand searching identified 77/1094 relevant articles for 2005 and 55/1199 for 2010. We developed a search hedge with 100% sensitivity and a precision of 30 and 29% for the 2005 and 2010 development datasets respectively. For the selected time periods, EXP Spine returned 2113 publications and 30 were considered relevant. The search filter identified all 30 relevant articles, with a specificity of 94% and precision of 20%. Of the 255 references listed in the narrative index reviews, 225 were indexed in MEDLINE and 165 (73%) were relevant articles. All relevant articles were identified and accuracy ranged from 67 to 97% over the three reviews. Of the 42 articles returned from 3 recent systematic reviews, all were identified by the filter.ConclusionsWe have developed a highly sensitive hedge for the research of DCM. Whilst precision is similarly low as other hedges, this search filter can be used as an adjunct for DCM search strategies.
| S-EPMC6034255 | biostudies-literature
Project description:Objective:Hypothetically, content in MEDLINE records is consistent across multiple platforms. Though platforms have different interfaces and requirements for query syntax, results should be similar when the syntax is controlled for across the platforms. The authors investigated how search result counts varied when searching records among five MEDLINE platforms. Methods:We created 29 sets of search queries targeting various metadata fields and operators. Within search sets, we adapted 5 distinct, compatible queries to search 5 MEDLINE platforms (PubMed, ProQuest, EBSCOhost, Web of Science, and Ovid), totaling 145 final queries. The 5 queries were designed to be logically and semantically equivalent and were modified only to match platform syntax requirements. We analyzed the result counts and compared PubMed's MEDLINE result counts to result counts from the other platforms. We identified outliers by measuring the result count deviations using modified z-scores centered around PubMed's MEDLINE results. Results:Web of Science and ProQuest searches were the most likely to deviate from the equivalent PubMed searches. EBSCOhost and Ovid were less likely to deviate from PubMed searches. Ovid's results were the most consistent with PubMed's but appeared to apply an indexing algorithm that resulted in lower retrieval sets among equivalent searches in PubMed. Web of Science exhibited problems with exploding or not exploding Medical Subject Headings (MeSH) terms. Conclusion:Platform enhancements among interfaces affect record retrieval and challenge the expectation that MEDLINE platforms should, by default, be treated as MEDLINE. Substantial inconsistencies in search result counts, as demonstrated here, should raise concerns about the impact of platform-specific influences on search results.
| S-EPMC6579582 | biostudies-literature
Project description:ObjectiveTo develop optimal search strategies in Medline for retrieving sound clinical studies on the diagnosis of health disorders.DesignAnalytical survey.SettingMedline, 2000.Participants170 journals for 2000 of which 161 were indexed in Medline.Main outcome measuresThe sensitivity, specificity, precision ("positive predictive value"), and accuracy of 4862 unique terms in 17,287 combinations were determined by comparison with a hand search of all articles (the "gold standard") in 161 journals published during 2000 (49,028 articles).ResultsOnly 147 (18.9%) of 778 articles about diagnostic tests met basic criteria for scientific merit. Combinations of search terms reached peak sensitivities of 98.6% at a specificity of 74.3%. Compared with best single terms, best multiple terms increased sensitivity for sound studies by 6.8% (absolute increase), while also increasing specificity (absolute increase 6.0%) when sensitivity was maximised. When terms were combined to maximise specificity, the single term, specificity.tw. (98.4%), outperformed combinations of terms. The strategies newly reported in this paper outperformed other validated search strategies except for one strategy that had slightly higher sensitivity (99.3% v 98.6%) but lower specificity (54.7% v 74.3%).ConclusionNew empirical search strategies in Medline can optimise retrieval of articles reporting high quality clinical studies of diagnosis.
| S-EPMC403841 | biostudies-literature
Project description:Tools to enhance physician searches of Medline and other bibliographic databases have potential to improve the application of new knowledge in patient care. This is particularly true for articles about glomerular disease, which are published across multiple disciplines and are often difficult to track down. Our objective was to develop and test search filters for PubMed, Ovid Medline, and Embase that allow physicians to search within a subset of the database to retrieve articles relevant to glomerular disease.We used a diagnostic test assessment framework with development and validation phases. We read a total of 22,992 full text articles for relevance and assigned them to the development or validation set to define the reference standard. We then used combinations of search terms to develop 997,298 unique glomerular disease filters. Outcome measures for each filter included sensitivity, specificity, precision, and accuracy. We selected optimal sensitive and specific search filters for each database and applied them to the validation set to test performance.High performance filters achieved at least 93.8% sensitivity and specificity in the development set. Filters optimized for sensitivity reached at least 96.7% sensitivity and filters optimized for specificity reached at least 98.4% specificity. Performance of these filters was consistent in the validation set and similar among all three databases.PubMed, Ovid Medline, and Embase can be filtered for articles relevant to glomerular disease in a reliable manner. These filters can now be used to facilitate physician searching.
| S-EPMC3471011 | biostudies-literature
Project description:Introduction:Administrative data arising via the operation of public service delivery systems hold great benefits for citizens and society by enabling new research questions to be addressed, providing they can be made available in a safe, socially acceptable way. In recognition of this potential, the UK Administrative Data Research Network was established in 2013 to enable new research for public benefit. However, there are considerable challenges to be overcome for effective data use, and many of these are common to administrative data enterprises in general. Using this network as a practical case study, we set out to explore the issues and propose how to share the 'good', suggest solutions to the 'bad', and improve the 'clunky' issues, to lead to improvements in administrative data use. Methods:A qualitative survey representing the data use pathway was carried out across the network, followed by a workshop to discuss the summarised findings and make further suggestions. This led to a set of recommendations to inform the development of an action plan for implementation. Results:The survey respondents (N=27) and workshop participants (N=95) comprised multi disciplinary staff from across the network. The responses were summarised by consensus of three researchers and grouped into six areas: A) Data acquisition pathway; B) Approval processes; C) Controls on access & disclosure; D) Data and metadata; E) Researcher support; and F) Data reuse & retention, leading to an embedded set of 18 recommendations. Key developments promoted by this study were the development of themed research partnerships to progress data acquisition, and a policy of data retention and reuse for research. Conclusions:The network has broken new ground in using administrative data for research. This study informed the development of an evidence-based action plan to address many challenges in the effective use of administrative data. It represents a practical worked example, and the learning is widely relevant to enterprises working with administrative data across the world.
| S-EPMC7479922 | biostudies-literature
Project description:ObjectivesTo create user-friendly search filters with high sensitivity, specificity, and precision to identify articles on geriatric medicine in Medline.DesignA diagnostic test assessment framework was used. A reference set of 2255 articles was created by hand-searching 22 biomedical journals in Medline, and each article was labeled as 'relevant', 'not relevant', or 'possibly relevant' for geriatric medicine. From the relevant articles, search terms were identified to compile different search strategies. The articles retrieved by the various search strategies were compared with articles from the reference set as the index test to create the search filters.MeasuresSensitivity, specificity, precision, accuracy, and number-needed-to-read (NNR) were calculated by comparing the results retrieved by the different search strategies with the reference set.ResultsThe most sensitive search filter had a sensitivity of 94.8%, a specificity of 88.7%, a precision of 73.0%, and an accuracy of 90.2%. It had an NNR of 1.37. The most specific search filter had a specificity of 96.6%, a sensitivity of 69.1%, a precision of 86.6%, and an accuracy of 89.9%. It had an NNR of 1.15.ConclusionThese geriatric search filters simplify searching for relevant literature and therefore contribute to better evidence-based practice. The filters are useful to both the clinician who wants to find a quick answer to a clinical question and the researcher who wants to find as many relevant articles as possible without retrieving too many irrelevant articles.
| S-EPMC3341784 | biostudies-literature
Project description:BACKGROUND: The interest in prognostic reviews is increasing, but to properly review existing evidence an accurate search filer for finding prediction research is needed. The aim of this paper was to validate and update two previously introduced search filters for finding prediction research in Medline: the Ingui filter and the Haynes Broad filter. METHODOLOGY/PRINCIPAL FINDINGS: Based on a hand search of 6 general journals in 2008 we constructed two sets of papers. Set 1 consisted of prediction research papers (n = 71), and set 2 consisted of the remaining papers (n = 1133). Both search filters were validated in two ways, using diagnostic accuracy measures as performance measures. First, we compared studies in set 1 (reference) with studies retrieved by the search strategies as applied in Medline. Second, we compared studies from 4 published systematic reviews (reference) with studies retrieved by the search filter as applied in Medline. Next--using word frequency methods--we constructed an additional search string for finding prediction research. Both search filters were good in identifying clinical prediction models: sensitivity ranged from 0.94 to 1.0 using our hand search as reference, and 0.78 to 0.89 using the systematic reviews as reference. This latter performance measure even increased to around 0.95 (range 0.90 to 0.97) when either search filter was combined with the additional string that we developed. Retrieval rate of explorative prediction research was poor, both using our hand search or our systematic review as reference, and even combined with our additional search string: sensitivity ranged from 0.44 to 0.85. CONCLUSIONS/SIGNIFICANCE: Explorative prediction research is difficult to find in Medline, using any of the currently available search filters. Yet, application of either the Ingui filter or the Haynes broad filter results in a very low number missed clinical prediction model studies.
| S-EPMC3290602 | biostudies-other
Project description:BackgroundSurgery is the primary treatment for colorectal cancer for both curative and palliative intent. Availability of high quality surgery data is essential for assessing many aspects of the quality of colorectal cancer care. The objective of this study was to determine the quality of different administrative data sources in identifying surgery for colorectal cancer with respect to completeness and accuracy.MethodsAll residents in Alberta, Canada who were diagnosed with invasive colorectal cancer in years 2000-2005 were identified from the Alberta Cancer Registry and included in the study. Surgery data for these patients were obtained from the Cancer Registry (which collects the date of surgery for which the primary tumor was removed) and compared to surgery data obtained from two different administrative data sources: Physician Billing and Hospital Inpatient data. Sensitivity, specificity, positive predictive value, negative predictive value and observed agreement were calculated compared to the Cancer Registry data.ResultsThe Physician Billing data alone or combined with Hospital Inpatient data demonstrated equally high sensitivity (97% for both) and observed agreement with the Cancer Registry data (93% for both) for identifying surgeries. The Hospital Inpatient data, however, had the highest specificity (80%). The positive predictive value varied by disease stage and across data sources for stage IV (99% for stages I-III and 83-89% for stage IV), the specificity is better for colon cancer surgeries (72-85%) than for rectal cancer surgeries (60-73%); validation measures did not vary over time.ConclusionPhysician Billing data identify the colorectal cancer surgery more completely than Hospital Inpatient data although both sources have a high level of completeness.
| S-EPMC3406984 | biostudies-literature
Project description:ObjectivesStudies on the impact of long COVID on work capacity are increasing but are difficult to locate in bibliographic databases, due to the heterogeneity of the terms used to describe this new condition and its consequences. This study aims to report on the effectiveness of different search strategies to find studies on the impact of long COVID on work participation in PubMed and to create validated search strings.MethodsWe searched PubMed for articles published on Long COVID and including information about work. Relevant articles were identified and their reference lists were screened. Occupational health journals were manually scanned to identify articles that could have been missed. A total of 885 articles potentially relevant were collected and 120 were finally included in a gold standard database. Recall, Precision, and Number Needed to Read (NNR) of various keywords or combinations of keywords were assessed.ResultsOverall, 123 search-words alone or in combination were tested. The highest Recalls with a single MeSH term or textword were 23 and 90%, respectively. Two different search strings were developed, one optimizing Recall while keeping Precision acceptable (Recall 98.3%, Precision 15.9%, NNR 6.3) and one optimizing Precision while keeping Recall acceptable (Recall 90.8%, Precision 26.1%, NNR 3.8).ConclusionsNo single MeSH term allows to find all relevant studies on the impact of long COVID on work ability in PubMed. The use of various MeSH and non-MeSH terms in combination is required to recover such studies without being overwhelmed by irrelevant articles.
| S-EPMC10940504 | biostudies-literature