Weakly supervised natural language processing for assessing patient-centered outcome following prostate cancer treatment.
Ontology highlight
ABSTRACT: Background
The population-based assessment of patient-centered outcomes (PCOs) has been limited by the efficient and accurate collection of these data. Natural language processing (NLP) pipelines can determine whether a clinical note within an electronic medical record contains evidence on these data. We present and demonstrate the accuracy of an NLP pipeline that targets to assess the presence, absence, or risk discussion of two important PCOs following prostate cancer treatment: urinary incontinence (UI) and bowel dysfunction (BD).Methods
We propose a weakly supervised NLP approach which annotates electronic medical record clinical notes without requiring manual chart review. A weighted function of neural word embedding was used to create a sentence-level vector representation of relevant expressions extracted from the clinical notes. Sentence vectors were used as input for a multinomial logistic model, with output being either presence, absence or risk discussion of UI/BD. The classifier was trained based on automated sentence annotation depending only on domain-specific dictionaries (weak supervision).Results
The model achieved an average F1 score of 0.86 for the sentence-level, three-tier classification task (presence/absence/risk) in both UI and BD. The model also outperformed a pre-existing rule-based model for note-level annotation of UI with significant margin.Conclusions
We demonstrate a machine learning method to categorize clinical notes based on important PCOs that trains a classifier on sentence vector representations labeled with a domain-specific dictionary, which eliminates the need for manual engineering of linguistic rules or manual chart review for extracting the PCOs. The weakly supervised NLP pipeline showed promising sensitivity and specificity for identifying important PCOs in unstructured clinical text notes compared to rule-based algorithms.
SUBMITTER: Banerjee I
PROVIDER: S-EPMC6482003 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ContextIdentifying tranquil areas is important for landscape planning and policy-making. Research demonstrated discrepancies between modelled potential tranquil areas and where people experience tranquillity based on field surveys. Because surveys are resource-intensive, user-generated text data offers potential for extracting where people experience tranquillity.ObjectivesWe explore and model the relationship between landscape ecological measures and experienced tranquillity extracted from user-generated text descriptions.MethodsGeoreferenced, user-generated landscape descriptions from Geograph.UK were filtered using keywords related to tranquillity. We stratify resulting tranquil locations according to dominant land cover and quantify the influence of landscape characteristics including diversity and naturalness on explaining the presence of tranquillity. Finally, we apply natural language processing to identify terms linked to tranquillity keywords and compare the similarity of these terms across land cover classes.ResultsEvaluation of potential keywords yielded six keywords associated with experienced tranquillity, resulting in 15,350 extracted tranquillity descriptions. The two most common land cover classes associated with tranquillity were arable and horticulture, and improved grassland, followed by urban and suburban. In the logistic regression model across all land cover classes, freshwater, elevation and naturalness were positive predictors of tranquillity. Built-up area was a negative predictor. Descriptions of tranquillity were most similar between improved grassland and arable and horticulture, and most dissimilar between arable and horticulture and urban.ConclusionsThis study highlights the potential of applying natural language processing to extract experienced tranquillity from text, and demonstrates links between landscape ecological measures and tranquillity as a perceived landscape quality.Supplementary informationThe online version of this article (10.1007/s10980-020-01181-8) contains supplementary material, which is available to authorized users.
| S-EPMC8550761 | biostudies-literature
Project description:Background Conventional prognostic scores usually require predefined clinical variables to predict outcome. The advancement of natural language processing has made it feasible to derive meaning from unstructured data. We aimed to test whether using unstructured text in electronic health records can improve the prediction of functional outcome after acute ischemic stroke. Methods and Results Patients hospitalized for acute ischemic stroke were identified from 2 hospital stroke registries (3847 and 2668 patients, respectively). Prediction models developed using the first cohort were externally validated using the second cohort, and vice versa. Free text in the history of present illness and computed tomography reports was used to build machine learning models using natural language processing to predict poor functional outcome at 90 days poststroke. Four conventional prognostic models were used as baseline models. The area under the receiver operating characteristic curves of the model using history of present illness in the internal and external validation sets were 0.820 and 0.792, respectively, which were comparable to the National Institutes of Health Stroke Scale score (0.811 and 0.807). The model using computed tomography reports achieved area under the receiver operating characteristic curves of 0.758 and 0.658. Adding information from clinical text significantly improved the predictive performance of each baseline model in terms of area under the receiver operating characteristic curves, net reclassification improvement, and integrated discrimination improvement indices (all P<0.001). Swapping the study cohorts led to similar results. Conclusions By using natural language processing, unstructured text in electronic health records can provide an alternative tool for stroke prognostication, and even enhance the performance of existing prognostic scores.
| S-EPMC9075227 | biostudies-literature
Project description:Funding institutions often solicit text-based research proposals to evaluate potential recipients. Leveraging the information contained in these documents could help institutions understand the supply of research within their domain. In this work, an end-to-end methodology for semi-supervised document clustering is introduced to partially automate classification of research proposals based on thematic areas of interest. The methodology consists of three stages: (1) manual annotation of a document sample; (2) semi-supervised clustering of documents; (3) evaluation of cluster results using quantitative metrics and qualitative ratings (coherence, relevance, distinctiveness) by experts. The methodology is described in detail to encourage replication and is demonstrated on a real-world data set. This demonstration sought to categorize proposals submitted to the US Army Telemedicine and Advanced Technology Research Center (TATRC) related to technological innovations in military medicine. A comparative analysis of method features was performed, including unsupervised vs. semi-supervised clustering, several document vectorization techniques, and several cluster result selection strategies. Outcomes suggest that pretrained Bidirectional Encoder Representations from Transformers (BERT) embeddings were better suited for the task than older text embedding techniques. When comparing expert ratings between algorithms, semi-supervised clustering produced coherence ratings ~ 25% better on average compared to standard unsupervised clustering with negligible differences in cluster distinctiveness. Last, it was shown that a cluster result selection strategy that balances internal and external validity produced ideal results. With further refinement, this methodological framework shows promise as a useful analytical tool for institutions to unlock hidden insights from untapped archives and similar administrative document repositories.Supplementary informationThe online version contains supplementary material available at 10.1007/s11192-023-04689-3.
| S-EPMC10083066 | biostudies-literature
Project description:BackgroundClinical natural language processing (cNLP) systems are of crucial importance due to their increasing capability in extracting clinically important information from free text contained in electronic health records (EHRs). The conversion of a nonstructured representation of a patient's clinical history into a structured format enables medical doctors to generate clinical knowledge at a level that was not possible before. Finally, the interpretation of the insights gained provided by cNLP systems has a great potential in driving decisions about clinical practice. However, carrying out robust evaluations of those cNLP systems is a complex task that is hindered by a lack of standard guidance on how to systematically approach them.ObjectiveOur objective was to offer natural language processing (NLP) experts a methodology for the evaluation of cNLP systems to assist them in carrying out this task. By following the proposed phases, the robustness and representativeness of the performance metrics of their own cNLP systems can be assured.MethodsThe proposed evaluation methodology comprised five phases: (1) the definition of the target population, (2) the statistical document collection, (3) the design of the annotation guidelines and annotation project, (4) the external annotations, and (5) the cNLP system performance evaluation. We presented the application of all phases to evaluate the performance of a cNLP system called "EHRead Technology" (developed by Savana, an international medical company), applied in a study on patients with asthma. As part of the evaluation methodology, we introduced the Sample Size Calculator for Evaluations (SLiCE), a software tool that calculates the number of documents needed to achieve a statistically useful and resourceful gold standard.ResultsThe application of the proposed evaluation methodology on a real use-case study of patients with asthma revealed the benefit of the different phases for cNLP system evaluations. By using SLiCE to adjust the number of documents needed, a meaningful and resourceful gold standard was created. In the presented use-case, using as little as 519 EHRs, it was possible to evaluate the performance of the cNLP system and obtain performance metrics for the primary variable within the expected CIs.ConclusionsWe showed that our evaluation methodology can offer guidance to NLP experts on how to approach the evaluation of their cNLP systems. By following the five phases, NLP experts can assure the robustness of their evaluation and avoid unnecessary investment of human and financial resources. Besides the theoretical guidance, we offer SLiCE as an easy-to-use, open-source Python library.
| S-EPMC8367121 | biostudies-literature
Project description:BACKGROUND:Over 100 million Americans lack affordable access to behavioral health care. Among these, military veterans are an especially vulnerable population. Military veterans require unique behavioral health services that can address military experiences and challenges transitioning to the civilian sector. Real-world programs to help veterans successfully transition to civilian life must build a sense of community, have the ability to scale, and be able to reach the many veterans who cannot or will not access care. Digitally based behavioral health initiatives have emerged within the past few years to improve this access to care. Our novel behavioral health intervention teaches mindfulness-based cognitive behavioral therapy and narrative therapy using peer support groups as guides, with human-facilitated asynchronous online discussions. Our study applies natural language processing (NLP) analytics to assess effectiveness of our online intervention in order to test whether NLP may provide insights and detect nuances of personal change and growth that are not currently captured by subjective symptom measures. OBJECTIVE:This paper aims to study the value of NLP analytics in assessing progress and outcomes among combat veterans and military sexual assault survivors participating in novel online interventions for posttraumatic growth. METHODS:IBM Watson and Linguistic Inquiry and Word Count tools were applied to the narrative writings of combat veterans and survivors of military sexual trauma who participated in novel online peer-supported group therapies for posttraumatic growth. Participants watched videos, practiced skills such as mindfulness meditation, told their stories through narrative writing, and participated in asynchronous, facilitated online discussions with peers. The writings, including online postings, by the 16 participants who completed the program were analyzed after completion of the program. RESULTS:Our results suggest that NLP can provide valuable insights on shifts in personality traits, personal values, needs, and emotional tone in an evaluation of our novel online behavioral health interventions. Emotional tone analysis demonstrated significant decreases in fear and anxiety, sadness, and disgust, as well as increases in joy. Significant effects were found for personal values and needs, such as needing or desiring closeness and helping others, and for personality traits of openness, conscientiousness, extroversion, agreeableness, and neuroticism (ie, emotional range). Participants also demonstrated increases in authenticity and clout (confidence) of expression. NLP results were generally supported by qualitative observations and analysis, structured data, and course feedback. CONCLUSIONS:The aggregate of results in our study suggest that our behavioral health intervention was effective and that NLP can provide valuable insights on shifts in personality traits, personal values, and needs, as well as measure changes in emotional tone. NLP's sensitivity to changes in emotional tone, values, and personality strengths suggests the efficacy of NLP as a leading indicator of treatment progress.
| S-EPMC7542412 | biostudies-literature
Project description:BackgroundPostoperative complication rates are often assessed through administrative data, although this method has proven to be imprecise. Recently, new developments in natural language processing have shown promise in detecting specific phenotypes from free medical text. Using the clinical challenge of extracting four specific and frequently undercoded postoperative complications (pneumonia, urinary tract infection, sepsis, and septic shock), it was hypothesized that natural language processing would capture postoperative complications on a par with human-level curation from electronic health record free medical text.MethodsElectronic health record data were extracted for surgical cases (across 11 surgical sub-specialties) from 18 hospitals in the Capital and Zealand regions of Denmark that were performed between May 2016 and November 2021. The data set was split into training/validation/test sets (30.0%/48.0%/22.0%). Model performance was compared with administrative data and manual extraction of the test data set.ResultsData were obtained for 17 486 surgical cases. Natural language processing achieved a receiver operating characteristic area under the curve of 0.989 for urinary tract infection, 0.993 for pneumonia, 0.992 for sepsis, and 0.998 for septic shock, whereas administrative data achieved a receiver operating characteristic area under the curve of 0.595 for urinary tract infection, 0.624 for pneumonia, 0.571 for sepsis, and 0.625 for septic shock.ConclusionThe natural language processing approach was able to capture complications with acceptable performance, which was superior to administrative data. In addition, the model performance approached that of manual curation and thereby offers a potential pathway for complete real-time coverage of postoperative complications across surgical procedures based on natural language processing assessment of electronic health record free medical text.
| S-EPMC11003538 | biostudies-literature
Project description:Lifestyle modification, including diet, exercise, and tobacco cessation, is the first-line treatment of many disorders including hypertension, obesity, and diabetes. Lifestyle modification data are not easily retrieved or used in research due to their textual nature. This study addresses this knowledge gap using natural language processing to automatically identify lifestyle modification documentation from electronic health records. Electronic health record notes from hypertension patients were analyzed using an open-source natural language processing tool to retrieve assessment and advice regarding lifestyle modification. These data were classified as lifestyle modification assessment or advice and mapped to a coded standard ontology. Combined lifestyle modification (advice and assessment) recall was 99.27 percent, precision 94.44 percent, and correct classification 88.15 percent. Through extraction and transformation of narrative lifestyle modification data to coded data, this critical information can be used in research, metric development, and quality improvement efforts regarding care delivery for multiple medical conditions that benefit from lifestyle modification.
| S-EPMC6722039 | biostudies-literature
Project description:In this paper we present a new method for automatic transliteration and segmentation of Unicode cuneiform glyphs using Natural Language Processing (NLP) techniques. Cuneiform is one of the earliest known writing system in the world, which documents millennia of human civilizations in the ancient Near East. Hundreds of thousands of cuneiform texts were found in the nineteenth and twentieth centuries CE, most of which are written in Akkadian. However, there are still tens of thousands of texts to be published. We use models based on machine learning algorithms such as recurrent neural networks (RNN) with an accuracy reaching up to 97% for automatically transliterating and segmenting standard Unicode cuneiform glyphs into words. Therefore, our method and results form a major step towards creating a human-machine interface for creating digitized editions. Our code, Akkademia, is made publicly available for use via a web application, a python package, and a github repository.
| S-EPMC7592802 | biostudies-literature
Project description:Recently, researches related to unsupervised disentanglement learning with deep generative models have gained substantial popularity. However, without introducing supervision, there is no guarantee that the factors of interest can be successfully recovered (Locatello et al. 2018). Motivated by a real-world problem, we propose a setting where the user introduces weak supervision by providing similarities between instances based on a factor to be disentangled. The similarity is provided as either a binary (yes/no) or real-valued label describing whether a pair of instances are similar or not. We propose a new method for weakly supervised disentanglement of latent variables within the framework of Variational Autoencoder. Experimental results demonstrate that utilizing weak supervision improves the performance of the disentanglement method substantially.
| S-EPMC7494201 | biostudies-literature
Project description:BackgroundPrehospital care provided by specially trained, physician-based emergency services (P-EMS) is an integrated part of the emergency medical systems in many developed countries. To what extent P-EMS increases survival and favourable outcomes is still unclear. The aim of the study was thus to investigate ambulance runs initially assigned 'life-saving missions' with emphasis on long-term outcome in patients treated by the Mobile Emergency Care Unit (MECU) in Odense, DenmarkMethodsAll MECU runs are registered in a database by the attending physician, stating, among other parameters, the treatment given, outcome of the treatment and the patient's diagnosis. Over a period of 80 months from May 1 2006 to December 31 2012, all missions in which the outcome of the treatment was registered as 'life saving' were scrutinised. Initial outcome, level of competence of the caretaker and diagnosis of each patient were manually established in each case in a combined audit of the prehospital database, the discharge summary of the MECU and the medical records from the hospital. Outcome parameters were final outcome, the aetiology of the life-threatening condition and the level of competences necessary to treat the patient.ResultsOf 25 647 patients treated by the MECU, 701 (2.7%) received prehospital 'life saving treatment'. In 596 (2.3%) patients this treatment exceeded the competences of the attending emergency medical technician or paramedic. Of these patients, 225 (0.9%) were ultimately discharged to their own home.ConclusionsThe present study demonstrates that anaesthesiologist administrated prehospital therapy increases the level of treatment modalities leading to an increased survival in relation to a prehospital system consisting of emergency medical technicians and paramedics alone and thus supports the concept of applying specialists in anaesthesiology in the prehospital setting especially when treating patients with cardiac arrest, patients in need of respiratory support and trauma patients.
| S-EPMC4289732 | biostudies-literature