Dataset Information

Detecting Hypoglycemia Incidents Reported in Patients' Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance.

ABSTRACT: BACKGROUND:Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety. OBJECTIVE:We aimed to develop a natural language processing system, called HypoDetect (Hypoglycemia Detector), to automatically identify hypoglycemia incidents reported in patients' secure messages. METHODS:An expert in public health annotated 3000 secure message threads between patients with diabetes and US Department of Veterans Affairs clinical teams as containing patient-reported hypoglycemia incidents or not. A physician independently annotated 100 threads randomly selected from this dataset to determine interannotator agreement. We used this dataset to develop and evaluate HypoDetect. HypoDetect incorporates 3 machine learning algorithms widely used for text classification: linear support vector machines, random forest, and logistic regression. We explored different learning features, including new knowledge-driven features. Because only 114 (3.80%) messages were annotated as positive, we investigated cost-sensitive learning and oversampling methods to mitigate the challenge of imbalanced data. RESULTS:The interannotator agreement was Cohen kappa=.976. Using cross-validation, logistic regression with cost-sensitive learning achieved the best performance (area under the receiver operating characteristic curve=0.954, sensitivity=0.693, specificity 0.974, F1 score=0.590). Cost-sensitive learning and the ensembled synthetic minority oversampling technique improved the sensitivity of the baseline systems substantially (by 0.123 to 0.728 absolute gains). Our results show that a variety of features contributed to the best performance of HypoDetect. CONCLUSIONS:Despite the challenge of data imbalance, HypoDetect achieved promising results for the task of detecting hypoglycemia incidents from secure messages. The system has a great potential to facilitate early detection and treatment of hypoglycemia.

SUBMITTER: Chen J

PROVIDER: S-EPMC6431826 | biostudies-literature | 2019 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Detecting Hypoglycemia Incidents Reported in Patients' Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance.

Chen Jinying J Lalor John J Liu Weisong W Druhl Emily E Granillo Edgard E Vimalananda Varsha G VG Yu Hong H

Journal of medical Internet research 20190311 3

<h4>Background</h4>Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety.<h4>Objective</h4>We aimed to develop a natural language processi ...[more]

PMID: 30855231

Similar Datasets

Project description:ImportanceTelehealth use including secure messages has rapidly expanded since the COVID-19 pandemic, including for multidisciplinary aspects of cancer care. Recent reports described rapid uptake and various benefits for patients and clinicians, suggesting that telehealth may be in standard use after the pandemic.ObjectiveTo examine attitudes and perceptions of multidisciplinary cancer care clinicians toward telehealth and secure messages.Design, setting, and participantsCross-sectional specialty-specific survey (ie, some questions appear only for relevant specialties) among multidisciplinary cancer care clinicians, collected from April 29, 2020, to June 5, 2020. Participants were all 285 clinicians in the fields of medical oncology, radiation oncology, surgical oncology, survivorship, and oncology navigation from all 21 community cancer centers of Kaiser Permanente Northern California.Main outcomes and measuresClinician satisfaction, perceived benefits and challenges of telehealth, perceived quality of telehealth and secure messaging, preferred visit and communication types for different clinical activities, and preferences regarding postpandemic telehealth use.ResultsA total of 202 clinicians (71%) responded (104 of 128 medical oncologists, 34 of 37 radiation oncologists, 16 of 62 breast surgeons, 18 of 28 navigators, and 30 of 30 survivorship experts; 57% (116 of 202) were women; 73% [147 of 202] between ages 36-55 years). Seventy-six percent (n = 154) were satisfied with telehealth without statistically significant variations based on clinician characteristics. In-person visits were thought to promote a strong patient-clinician connection by 99% (n = 137) of respondents compared with 77% (n = 106) for video visits, 43% (n = 59) for telephone, and 14% (n = 19) for secure messages. The most commonly cited benefits of telehealth to clinicians included reduced commute (79%; n = 160), working from home (74%; n = 149), and staying on time (65%; n = 132); the most commonly cited negative factors included internet connection (84%; n = 170) or equipment problems (72%; n = 146), or physical examination needed (64%; n = 131). Most respondents (59%; n = 120) thought that video is adequate to manage the greater part of patient care in general; and most deemed various telehealth modalities suitable for any of the queried types of patient-clinician activities. For some specific activities, less than half of respondents thought that only an in-person visit is acceptable (eg, 49%; n = 66 for end-of-life discussion, 35%; n = 58 for new diagnosis). Most clinicians (82%; n = 166) preferred to maintain or increase use of telehealth after the pandemic.Conclusions and relevanceIn this survey of multidisciplinary cancer care clinicians in the COVID-19 era, telehealth was well received and often preferred by most cancer care clinicians, who deemed it appropriate to manage most aspects of cancer care. As telehealth use becomes routine in some cancer care settings, video and telephone visits and use of asynchronous secure messaging with patients in cancer care has clear potential to extend beyond the pandemic period.

Project description:BACKGROUND:Studies show that combining nicotine replacement therapy (NRT) with tobacco treatment counseling is most effective for smoking cessation. However, tobacco treatment counseling has been underutilized across the nation. A secure email message sent to patients already taking NRT was hypothesized to increase the utilization of tobacco treatment counseling among Veterans in New Jersey. Secure messaging for communication between patients and providers was implemented through a web-based password-protected, secure messaging account, where veterans get notified through their personal email when they have a message awaiting them. OBJECTIVE:The main objective of this project was to determine if there was a significant increase in adoption of tobacco treatment counseling among Veterans who received a secure message describing the options for tobacco treatment counseling available to them. Secondary objectives were to demographically characterize Veterans who were and were not enrolled in secure messaging, as well as those who opened or did not open a message. Finally, because the language and content of the messages were changed across project phases, this project also sought to determine (by analysis of response rates) the type of language that was most effective at eliciting a response. METHODS:Over two phases, messages were sent to two samples of Veterans prescribed NRT within the prior 90 days of each phase. In phase 1, one message was sent in December 2015 (message 1). In phase 2, one message was sent in July 2016 (message 2) and the same message (message 3) was resent in August 2016 to persons who did not open message 2. Messages 2 and 3 were more directive than message 1. Response rates to message 1 versus message 2 were compared. A logistic regression analysis determined effect of age and gender on enrollment in secure messaging across both phases. The effectiveness of each phase at increasing tobacco treatment counseling was analyzed using a McNemar test. RESULTS:Message 2, sent to 423 Veterans, had a significantly higher response rate than message 1, sent to 348 Veterans (18%, 17/93 vs 8%, 6/78, P=.04). Phase 2 (ie, messages 2 and 3) significantly increased utilization of tobacco treatment counseling (net increase of six tobacco treatment counseling adopters, P=.04), whereas phase 1 (ie, message 1) did not (net increase of two tobacco treatment counseling adopters, P=.48). Women (odds ratio [OR] 1.6, 95% CI 1.1-2.3) and those aged 30 to 49 years (compared to other age groups) were more likely to be enrolled in secure messaging. Gender and age were not significant predictors of opening or replying to either message. CONCLUSIONS:Although the effect was small, secure messaging was a useful modality to increase tobacco treatment counseling. Directive content with a follow-up message appeared useful. Female Veterans and/or Veterans aged between 30 and 49 years are more likely to use secure messaging.

Project description:Limited health literacy is a barrier to optimal healthcare delivery and outcomes. Current measures requiring patients to self-report limitations are time-consuming and may be considered intrusive by some. This makes widespread classification of patient health literacy challenging. The objective of this study was to develop and validate "literacy profiles" as automated indicators of patients' health literacy to facilitate a non-intrusive, economic and more comprehensive characterization of health literacy among a health care delivery system's membership. To this end, three literacy profiles were generated based on natural language processing (combining computational linguistics and machine learning) using a sample of 283,216 secure messages sent from 6,941 patients to their primary care physicians. All patients were participants in Kaiser Permanente Northern California's DISTANCE Study. Performance of the three literacy profiles were compared against a gold standard of patient self-reported health literacy. Associations were analyzed between each literacy profile and patient demographics, health outcomes and healthcare utilization. T-tests were used for numeric data such as A1C, Charlson comorbidity index and healthcare utilization rates, and chi-square tests for categorical data such as sex, race, poor adherence and severe hypoglycemia. Literacy profiles varied in their test characteristics, with C-statistics ranging from 0.61-0.74. Relations between literacy profiles and health outcomes revealed patterns consistent with previous health literacy research: patients identified via literacy profiles indicative of limited health literacy: (a) were older and more likely of minority status; (b) had poorer medication adherence and glycemic control; and (c) exhibited higher rates of hypoglycemia, comorbidities and healthcare utilization. This represents the first successful attempt to employ natural language processing to estimate health literacy. Literacy profiles can offer an automated and economical way to identify patients with limited health literacy and greater vulnerability to poor health outcomes.

Dataset Information

Detecting Hypoglycemia Incidents Reported in Patients' Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance.

Publications

Detecting Hypoglycemia Incidents Reported in Patients' Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets