Dataset Information

Comparison of UMLS terminologies to identify risk of heart disease using clinical notes.

ABSTRACT: The second track of the 2014 i2b2 challenge asked participants to automatically identify risk factors for heart disease among diabetic patients using natural language processing techniques for clinical notes. This paper describes a rule-based system developed using a combination of regular expressions, concepts from the Unified Medical Language System (UMLS), and freely-available resources from the community. With a performance (F1=90.7) that is significantly higher than the median (F1=87.20) and close to the top performing system (F1=92.8), it was the best rule-based system of all the submissions in the challenge. We also used this system to evaluate the utility of different terminologies in the UMLS towards the challenge task. Of the 155 terminologies in the UMLS, 129 (76.78%) have no representation in the corpus. The Consumer Health Vocabulary had very good coverage of relevant concepts and was the most useful terminology for the challenge task. While segmenting notes into sections and lists has a significant impact on the performance, identifying negations and experiencer of the medical event results in negligible gain.

SUBMITTER: Shivade C

PROVIDER: S-EPMC4973866 | biostudies-other | 2015 Dec

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Comparison of UMLS terminologies to identify risk of heart disease using clinical notes.

Shivade Chaitanya C Malewadkar Pranav P Fosler-Lussier Eric E Lai Albert M AM

Journal of biomedical informatics 20150912

The second track of the 2014 i2b2 challenge asked participants to automatically identify risk factors for heart disease among diabetic patients using natural language processing techniques for clinical notes. This paper describes a rule-based system developed using a combination of regular expressions, concepts from the Unified Medical Language System (UMLS), and freely-available resources from the community. With a performance (F1=90.7) that is significantly higher than the median (F1=87.20) an ...[more]

PMID: 26375493

Similar Datasets

Project description:BackgroundThe electronic medical record contains a wealth of information buried in free text. We created a natural language processing algorithm to identify patients with atrial fibrillation (AF) using text alone.Methods and resultsWe created 3 data sets from patients with at least one AF billing code from 2010 to 2017: a training set (n=886), an internal validation set from site no. 1 (n=285), and an external validation set from site no. 2 (n=276). A team of clinicians reviewed and adjudicated patients as AF present or absent, which served as the reference standard. We trained 54 algorithms to classify each patient, varying the model, number of features, number of stop words, and the method used to create the feature set. The algorithm with the highest F-score (the harmonic mean of sensitivity and positive predictive value) in the training set was applied to the validation sets. F-scores and area under the receiver operating characteristic curves were compared between site no. 1 and site no. 2 using bootstrapping. Adjudicated AF prevalence was 75.1% at site no. 1 and 86.2% at site no. 2. Among 54 algorithms, the best performing model was logistic regression, using 1000 features, 100 stop words, and term frequency-inverse document frequency method to create the feature set, with sensitivity 92.8%, specificity 93.9%, and an area under the receiver operating characteristic curve of 0.93 in the training set. The performance at site no. 1 was sensitivity 92.5%, specificity 88.7%, with an area under the receiver operating characteristic curve of 0.91. The performance at site no. 2 was sensitivity 89.5%, specificity 71.1%, with an area under the receiver operating characteristic curve of 0.80. The F-score was lower at site no. 2 compared with site no. 1 (92.5% [SD, 1.1%] versus 94.2% [SD, 1.1%]; P<0.001).ConclusionsWe developed a natural language processing algorithm to identify patients with AF using text alone, with >90% F-score at 2 separate sites. This approach allows better use of the clinical narrative and creates an opportunity for precise, high-throughput cohort identification.

Project description:Importance:Inpatient violence remains a significant problem despite existing risk assessment methods. The lack of robustness and the high degree of effort needed to use current methods might be mitigated by using routinely registered clinical notes. Objective:To develop and validate a multivariable prediction model for assessing inpatient violence risk based on machine learning techniques applied to clinical notes written in patients' electronic health records. Design, Setting, and Participants:This prognostic study used retrospective clinical notes registered in electronic health records during admission at 2 independent psychiatric health care institutions in the Netherlands. No exclusion criteria for individual patients were defined. At site 1, all adults admitted between January 2013 and August 2018 were included, and at site 2 all adults admitted to general psychiatric wards between June 2016 and August 2018 were included. Data were analyzed between September 2018 and February 2019. Main Outcomes and Measures:Predictive validity and generalizability of prognostic models measured using area under the curve (AUC). Results:Clinical notes recorded during a total of 3189 admissions of 2209 unique individuals at site 1 (mean [SD] age, 34.0 [16.6] years; 1536 [48.2%] male) and 3253 admissions of 1919 unique individuals at site 2 (mean [SD] age, 45.9 [16.6] years; 2097 [64.5%] male) were analyzed. Violent outcome was determined using the Staff Observation Aggression Scale-Revised. Nested cross-validation was used to train and evaluate models that assess violence risk during the first 4 weeks of admission based on clinical notes available after 24 hours. The predictive validity of models was measured at site 1 (AUC = 0.797; 95% CI, 0.771-0.822) and site 2 (AUC = 0.764; 95% CI, 0.732-0.797). The validation of pretrained models in the other site resulted in AUCs of 0.722 (95% CI, 0.690-0.753) at site 1 and 0.643 (95% CI, 0.610-0.675) at site 2; the difference in AUCs between the internally trained model and the model trained on other-site data was significant at site 1 (AUC difference = 0.075; 95% CI, 0.045-0.105; P < .001) and site 2 (AUC difference = 0.121; 95% CI, 0.085-0.156; P < .001). Conclusions and Relevance:Internally validated predictions resulted in AUC values with good predictive validity, suggesting that automatic violence risk assessment using routinely registered clinical notes is possible. The validation of trained models using data from other sites corroborates previous findings that violence risk assessment generalizes modestly to different populations.

Project description:BackgroundCardiovascular disease (CVD) risk communication is a challenge for clinical practice, where physicians find it difficult to explain the absolute risk of a CVD event to patients with varying health literacy. Converting the probability to heart age is increasingly used to promote lifestyle change, but a rapid review of biological age interventions found no clear evidence that they motivate behavior change.ObjectiveIn this review, we aim to identify the content and effects of heart age interventions.MethodsWe conducted a systematic review of studies presenting heart age interventions to adults for CVD risk communication in April 2020 (later updated in March 2021). The Johanna Briggs risk of bias assessment tool was applied to randomized studies. Behavior change techniques described in the intervention methods were coded.ResultsFrom a total of 7926 results, 16 eligible studies were identified; these included 5 randomized web-based experiments, 5 randomized clinical trials, 2 mixed methods studies with quantitative outcomes, and 4 studies with qualitative analysis. Direct comparisons between heart age and absolute risk in the 5 web-based experiments, comprising 5514 consumers, found that heart age increased positive or negative emotional responses (4/5 studies), increased risk perception (4/5 studies; but not necessarily more accurate) and recall (4/4 studies), reduced credibility (2/3 studies), and generally had no effect on lifestyle intentions (4/5 studies). One study compared heart age and absolute risk to fitness age and found reduced lifestyle intentions for fitness age. Heart age combined with additional strategies (eg, in-person or phone counseling) in applied settings for 9582 patients improved risk control (eg, reduced cholesterol levels and absolute risk) compared with usual care in most trials (4/5 studies) up to 1 year. However, clinical outcomes were no different when directly compared with absolute risk (1/1 study). Mixed methods studies identified consultation time and content as important outcomes in actual consultations using heart age tools. There were differences between people receiving an older heart age result and those receiving a younger or equal to current heart age result. The heart age interventions included a wide range of behavior change techniques, and conclusions were sometimes biased in favor of heart age with insufficient supporting evidence. The risk of bias assessment indicated issues with all randomized clinical trials.ConclusionsThe findings of this review provide little evidence that heart age motivates lifestyle behavior change more than absolute risk, but either format can improve clinical outcomes when combined with other behavior change strategies. The label for the heart age concept can affect outcomes and should be pretested with the intended audience. Future research should consider consultation time and differentiate between results of older and younger heart age.International registered report identifier (irrid)NPRR2-10.1101/2020.05.03.20089938.

Project description:Background Cardiovascular risk factors have differential effects on various manifestations of cardiovascular disease, but to date direct formal comparisons are scarce, have been conducted primarily in men, and include only traditional risk factors. Methods and Results Using data from the multi-ethnic Women's Health Initiative Observational Study, we used a case-cohort design to compare 1731 women with incident cardiovascular disease during follow-up to a cohort of 1914 women. The direction of effect of all 24 risk factors (including various apolipoproteins, hemoglobin A1c, high-sensitivity C-reactive protein, N-terminal pro-brain natriuretic peptide, and tissue plasminogen activator antigen) was concordant for coronary heart disease (CHD, defined as myocardial infarction and CHD death) and ischemic stroke; however, associations were generally stronger with CHD. Significant differences for multiple risk factors, including blood pressure, lipid levels, and measures of inflammation, were observed when comparing the effects on hemorrhagic stroke with those on ischemic outcomes. For instance, multivariable adjusted hazard ratios per standard deviation increase in non-high-density lipoprotein cholesterol were 1.16 (95% confidence interval, 1.06-1.28) for CHD, 0.97 (0.88-1.07) for ischemic stroke, and 0.76 (0.63-0.91) for hemorrhagic stroke ( P<0.05 for equal association). Model discrimination was better for models predicting CHD or ischemic stroke than for models predicting hemorrhagic stroke or a combined end point. Conclusions Cardiovascular risk factors have largely similar effects on incidence of CHD and ischemic stroke in women, although the magnitude of association varies. Determinants of ischemic and hemorrhagic stroke substantially differ, underscoring their distinct biology. Cardiovascular disease risk may be more accurately reflected when combined cardiovascular disease or cerebrovascular outcomes are broken down into different first manifestations, or when restricted to ischemic outcomes.

Dataset Information

Comparison of UMLS terminologies to identify risk of heart disease using clinical notes.

Publications

Comparison of UMLS terminologies to identify risk of heart disease using clinical notes.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets