Dataset Information

Natural language processing (NLP) tools in extracting biomedical concepts from research articles: a case study on autism spectrum disorder.

ABSTRACT:

Background

Natural language processing (NLP) tools can facilitate the extraction of biomedical concepts from unstructured free texts, such as research articles or clinical notes. The NLP software tools CLAMP, cTAKES, and MetaMap are among the most widely used tools to extract biomedical concept entities. However, their performance in extracting disease-specific terminology from literature has not been compared extensively, especially for complex neuropsychiatric disorders with a diverse set of phenotypic and clinical manifestations.

Methods

We comparatively evaluated these NLP tools using autism spectrum disorder (ASD) as a case study. We collected 827 ASD-related terms based on previous literature as the benchmark list for performance evaluation. Then, we applied CLAMP, cTAKES, and MetaMap on 544 full-text articles and 20,408 abstracts from PubMed to extract ASD-related terms. We evaluated the predictive performance using precision, recall, and F1 score.

Results

We found that CLAMP has the best performance in terms of F1 score followed by cTAKES and then MetaMap. Our results show that CLAMP has much higher precision than cTAKES and MetaMap, while cTAKES and MetaMap have higher recall than CLAMP.

Conclusion

The analysis protocols used in this study can be applied to other neuropsychiatric or neurodevelopmental disorders that lack well-defined terminology sets to describe their phenotypic presentations.

SUBMITTER: Peng J

PROVIDER: S-EPMC7772897 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Natural language processing (NLP) tools in extracting biomedical concepts from research articles: a case study on autism spectrum disorder.

Peng Jacqueline J Zhao Mengge M Havrilla James J Liu Cong C Weng Chunhua C Guthrie Whitney W Schultz Robert R Wang Kai K Zhou Yunyun Y

BMC medical informatics and decision making 20201230 Suppl 11

<h4>Background</h4>Natural language processing (NLP) tools can facilitate the extraction of biomedical concepts from unstructured free texts, such as research articles or clinical notes. The NLP software tools CLAMP, cTAKES, and MetaMap are among the most widely used tools to extract biomedical concept entities. However, their performance in extracting disease-specific terminology from literature has not been compared extensively, especially for complex neuropsychiatric disorders with a diverse ...[more]

PMID: 33380331

Similar Datasets

Project description:The cognitive approach to psychotherapy aims to change patients' maladaptive schemas, that is, overly negative views on themselves, the world, or the future. To obtain awareness of these views, they record their thought processes in situations that caused pathogenic emotional responses. The schemas underlying such thought records have, thus far, been largely manually identified. Using recent advances in natural language processing, we take this one step further by automatically extracting schemas from thought records. To this end, we asked 320 healthy participants on Amazon Mechanical Turk to each complete five thought records consisting of several utterances reflecting cognitive processes. Agreement between two raters on manually scoring the utterances with respect to how much they reflect each schema was substantial (Cohen's κ = 0.79). Natural language processing software pretrained on all English Wikipedia articles from 2014 (GLoVE embeddings) was used to represent words and utterances, which were then mapped to schemas using k-nearest neighbors algorithms, support vector machines, and recurrent neural networks. For the more frequently occurring schemas, all algorithms were able to leverage linguistic patterns. For example, the scores assigned to the Competence schema by the algorithms correlated with the manually assigned scores with Spearman correlations ranging between 0.64 and 0.76. For six of the nine schemas, a set of recurrent neural networks trained separately for each of the schemas outperformed the other algorithms. We present our results here as a benchmark solution, since we conducted this research to explore the possibility of automatically processing qualitative mental health data and did not aim to achieve optimal performance with any of the explored models. The dataset of 1600 thought records comprising 5747 utterances is published together with this article for researchers and machine learning enthusiasts to improve upon our outcomes. Based on our promising results, we see further opportunities for using free-text input and subsequent natural language processing in other common therapeutic tools, such as ecological momentary assessments, automated case conceptualizations, and, more generally, as an alternative to mental health scales.

Project description:BackgroundA plethora of publicly available biomedical resources do currently exist and are constantly increasing at a fast rate. In parallel, specialized repositories are been developed, indexing numerous clinical and biomedical tools. The main drawback of such repositories is the difficulty in locating appropriate resources for a clinical or biomedical decision task, especially for non-Information Technology expert users. In parallel, although NLP research in the clinical domain has been active since the 1960s, progress in the development of NLP applications has been slow and lags behind progress in the general NLP domain. The aim of the present study is to investigate the use of semantics for biomedical resources annotation with domain specific ontologies and exploit Natural Language Processing methods in empowering the non-Information Technology expert users to efficiently search for biomedical resources using natural language.MethodsA Natural Language Processing engine which can "translate" free text into targeted queries, automatically transforming a clinical research question into a request description that contains only terms of ontologies, has been implemented. The implementation is based on information extraction techniques for text in natural language, guided by integrated ontologies. Furthermore, knowledge from robust text mining methods has been incorporated to map descriptions into suitable domain ontologies in order to ensure that the biomedical resources descriptions are domain oriented and enhance the accuracy of services discovery. The framework is freely available as a web application at ( http://calchas.ics.forth.gr/ ).ResultsFor our experiments, a range of clinical questions were established based on descriptions of clinical trials from the ClinicalTrials.gov registry as well as recommendations from clinicians. Domain experts manually identified the available tools in a tools repository which are suitable for addressing the clinical questions at hand, either individually or as a set of tools forming a computational pipeline. The results were compared with those obtained from an automated discovery of candidate biomedical tools. For the evaluation of the results, precision and recall measurements were used. Our results indicate that the proposed framework has a high precision and low recall, implying that the system returns essentially more relevant results than irrelevant.ConclusionsThere are adequate biomedical ontologies already available, sufficiency of existing NLP tools and quality of biomedical annotation systems for the implementation of a biomedical resources discovery framework, based on the semantic annotation of resources and the use on NLP techniques. The results of the present study demonstrate the clinical utility of the application of the proposed framework which aims to bridge the gap between clinical question in natural language and efficient dynamic biomedical resources discovery.

Project description:Autism spectrum disorder (ASD) is a neurodevelopmental condition characterized by communication and social interaction deficits, and by restricted interests and stereotyped, repetitive behavior patterns. ASD has a strong genetic component and a complex architecture characterized by the interplay of rare and common genetic variants. Recently, increasing evidence suggest a significant contribution of immune system dysregulation in ASD. The present paper reviews the latest updates regarding the altered immune landscape of this complex disorder highlighting areas with potential for biomarkers discovery as well as personalization of therapeutic approaches. Cross-talk between the central nervous system and immune system has long been envisaged and recent evidence brings insights into the pathways connecting the brain to the immune system. Disturbance of cytokine levels plays an important role in the establishment of a neuroinflammatory milieu in ASD. Several other immune molecules involved in antigen presentation and inflammatory cellular phenotypes are also at play in ASD. Maternal immune activation, the presence of brain-reactive antibodies and autoimmunity are other potential prenatal and postnatal contributors to ASD pathophysiology. The molecular players involved in oxidative-stress response and mitochondrial system function, are discussed as contributors to the pro-inflammatory pattern. The gastrointestinal inflammation pathways proposed to play a role in ASD are also discussed. Moreover, the body of evidence regarding some of the genetic factors linked to the immune system dysregulation is reviewed and discussed. Last, but not least, the epigenetic traits and their interactions with the immune system are reviewed as an expanding field in ASD research. Understanding the immune-mediated pathways that influence brain development and function, metabolism, and intestinal homeostasis, may lead to the identification of robust diagnostic or predictive biomarkers for ASD individuals. Thus, novel therapeutic approaches could be developed, ultimately aiming to improve their quality of life.

Dataset Information

Natural language processing (NLP) tools in extracting biomedical concepts from research articles: a case study on autism spectrum disorder.

Background

Methods

Results

Conclusion

Publications

Natural language processing (NLP) tools in extracting biomedical concepts from research articles: a case study on autism spectrum disorder.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets