Dataset Information

Automatically identifying health outcome information in MEDLINE records.

ABSTRACT:

Objective

Understanding the effect of a given intervention on the patient's health outcome is one of the key elements in providing optimal patient care. This study presents a methodology for automatic identification of outcomes-related information in medical text and evaluates its potential in satisfying clinical information needs related to health care outcomes.

Design

An annotation scheme based on an evidence-based medicine model for critical appraisal of evidence was developed and used to annotate 633 MEDLINE citations. Textual, structural, and meta-information features essential to outcome identification were learned from the created collection and used to develop an automatic system. Accuracy of automatic outcome identification was assessed in an intrinsic evaluation and in an extrinsic evaluation, in which ranking of MEDLINE search results obtained using PubMed Clinical Queries relied on identified outcome statements.

Measurements

The accuracy and positive predictive value of outcome identification were calculated. Effectiveness of the outcome-based ranking was measured using mean average precision and precision at rank 10.

Results

Automatic outcome identification achieved 88% to 93% accuracy. The positive predictive value of individual sentences identified as outcomes ranged from 30% to 37%. Outcome-based ranking improved retrieval accuracy, tripling mean average precision and achieving 389% improvement in precision at rank 10.

Conclusion

Preliminary results in outcome-based document ranking show potential validity of the evidence-based medicine-model approach in timely delivery of information critical to clinical decision support at the point of service.

SUBMITTER: Demner-Fushman D

PROVIDER: S-EPMC1380197 | biostudies-literature | 2006 Jan-Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automatically identifying health outcome information in MEDLINE records.

Demner-Fushman Dina D Few Barbara B Hauser Susan E SE Thoma George G

Journal of the American Medical Informatics Association : JAMIA 20051012 1

<h4>Objective</h4>Understanding the effect of a given intervention on the patient's health outcome is one of the key elements in providing optimal patient care. This study presents a methodology for automatic identification of outcomes-related information in medical text and evaluates its potential in satisfying clinical information needs related to health care outcomes.<h4>Design</h4>An annotation scheme based on an evidence-based medicine model for critical appraisal of evidence was developed ...[more]

PMID: 16221937

Similar Datasets

Project description:BackgroundElectronic health records (EHRs) are a rich source of longitudinal patient data. However, missing information due to clinical care that predated the implementation of EHR system(s) or care that occurred at different medical institutions impedes complete ascertainment of a patient's medical history.ObjectiveThis study aimed to investigate information discrepancies and to quantify information gaps by comparing the gynecological surgical history extracted from an EHR of a single institution by using natural language processing (NLP) techniques with the manually curated surgical history information through chart review of records from multiple independent regional health care institutions.MethodsTo facilitate high-throughput evaluation, we developed a rule-based NLP algorithm to detect gynecological surgery history from the unstructured narrative of the Mayo Clinic EHR. These results were compared to a gold standard cohort of 3870 women with gynecological surgery status adjudicated using the Rochester Epidemiology Project medical records-linkage system. We quantified and characterized the information gaps observed that led to misclassification of the surgical status.ResultsThe NLP algorithm achieved precision of 0.85, recall of 0.82, and F1-score of 0.83 in the test set (n=265) relative to outcomes abstracted from the Mayo EHR. This performance attenuated when directly compared to the gold standard (precision 0.79, recall 0.76, and F1-score 0.76), with the majority of misclassifications being false negatives in nature. We then applied the algorithm to the remaining patients (n=3340) and identified 2 types of information gaps through error analysis. First, 6% (199/3340) of women in this study had no recorded surgery information or partial information in the EHR. Second, 4.3% (144/3340) of women had inconsistent or inaccurate information within the clinical narrative owing to misinterpreted information, erroneous "copy and paste," or incorrect information provided by patients. Additionally, the NLP algorithm misclassified the surgery status of 3.6% (121/3340) of women.ConclusionsAlthough NLP techniques were able to adequately recreate the gynecologic surgical status from the clinical narrative, missing or inaccurately reported and recorded information resulted in much of the misclassification observed. Therefore, alternative approaches to collect or curate surgical history are needed.

Project description:BackgroundAs the availability of interoperable electronic health records (iEHRs) or health information exchanges (HIEs) continues to increase, there is greater need and opportunity to assess the current evidence base on what works and what does not regarding the adoption, use, and impact of iEHRs.ObjectiveThe purpose of this project is to assess the international evidence base on the adoption, use, and impact of iEHRs.MethodsWe conducted a systematic review, searching multiple databases-MEDLINE, Embase, and the Cumulative Index to Nursing and Allied Health Literature (CINAHL)-with supplemental searches conducted in Google Scholar and grey literature sources (ie, Google, Grey Literature Report, and OpenGrey). All searches were conducted in January and February 2017. Articles were eligible for inclusion if they were published in English, were published from 2006 to 2017, and were either an original research study or a literature review. In order to be included, articles needed to focus on iEHRs and HIEs across multiple health care settings, as well as on the impact and effectiveness of iEHR adoption and use.ResultsWe included 130 articles in the synthesis (113 primary studies, 86.9%; 17 reviews, 13.1%), with the majority focused on the United States (88/130, 67.7%). The primary studies focused on a wide range of health care settings; the three most prevalent settings studied included acute care (59/113, 52.2%), primary care (44/113, 38.9%), and emergency departments (34/113, 30.1%). We identified 29 distinct measurement items in the 113 primary studies that were linked to 522 specific measurement outcomes. Productivity and quality were the two evaluation dimensions that received the most attention, accounting for 14 of 29 (48%) measurement items and 306 of 522 (58.6%) measurement outcomes identified. Overall, the majority of the 522 measurement outcomes were positive (298/522, 57.1%). We also identified 17 reviews on iEHR use and impact, 6 (35%) that focused on barriers and facilitators to adoption and implementation and 11 (65%) that focused on benefits and impacts, with the more recent reviews finding little generalizable evidence of benefit and impact.ConclusionsThis review captures the status of an evolving and active field focused on the use and impact of iEHRs. While the overall findings suggest many positive impacts, the quality of the primary studies were not evaluated systematically. When broken down by specific measurement item, the results directed attention both to measurement outcomes that were consistently positive and others that were mostly negative or equivocal.

Project description:BackgroundMost efforts to identify caregivers for research use passive approaches such as self-nomination. We describe an approach in which electronic health records (EHRs) can help identify, recruit, and increase diverse representations of family and other unpaid caregivers.ObjectiveFew health systems have implemented systematic processes for identifying caregivers. This study aimed to develop and evaluate an EHR-driven process for identifying veterans likely to have unpaid caregivers in a caregiver survey study. We additionally examined whether there were EHR-derived veteran characteristics associated with veterans having unpaid caregivers.MethodsWe selected EHR home- and community-based referrals suggestive of veterans' need for supportive care from friends or family. We identified veterans with these referrals across the 8 US Department of Veteran Affairs medical centers enrolled in our study. Phone calls to a subset of these veterans confirmed whether they had a caregiver, specifically an unpaid caregiver. We calculated the screening contact rate for unpaid caregivers of veterans using attempted phone screening and for those who completed phone screening. The veteran characteristics from the EHR were compared across referral and screening groups using descriptive statistics, and logistic regression was used to compare the likelihood of having an unpaid caregiver among veterans who completed phone screening.ResultsDuring the study period, our EHR-driven process identified 12,212 veterans with home- and community-based referrals; 2134 (17.47%) veteran households were called for phone screening. Among the 2134 veterans called, 1367 (64.06%) answered the call, and 813 (38.1%) veterans had a caregiver based on self-report of the veteran, their caregiver, or another person in the household. The unpaid caregiver identification rate was 38.1% and 59.5% among those with an attempted phone screening and completed phone screening, respectively. Veterans had increased odds of having an unpaid caregiver if they were married (adjusted odds ratio [OR] 2.69, 95% CI 1.68-4.34), had respite care (adjusted OR 2.17, 95% CI 1.41-3.41), or had adult day health care (adjusted OR 3.69, 95% CI 1.60-10.00). Veterans with a dementia diagnosis (adjusted OR 1.37, 95% CI 1.00-1.89) or veteran-directed care referral (adjusted OR 1.95, 95% CI 0.97-4.20) were also suggestive of an association with having an unpaid caregiver.ConclusionsThe EHR-driven process to identify veterans likely to have unpaid caregivers is systematic and resource intensive. Approximately 60% (813/1367) of veterans who were successfully screened had unpaid caregivers. In the absence of discrete fields in the EHR, our EHR-driven process can be used to identify unpaid caregivers; however, incorporating caregiver identification fields into the EHR would support a more efficient and systematic identification of caregivers.Trial registrationClincalTrials.gov NCT03474380; https://clinicaltrials.gov/ct2/show/NCT03474380.

Dataset Information

Automatically identifying health outcome information in MEDLINE records.

Objective

Design

Measurements

Results

Conclusion

Publications

Automatically identifying health outcome information in MEDLINE records.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets