Dataset Information

Predictive modeling for classification of positive valence system symptom severity from initial psychiatric evaluation records.

ABSTRACT: In response to the challenges set forth by the CEGS N-GRID 2016 Shared Task in Clinical Natural Language Processing, we describe a framework to automatically classify initial psychiatric evaluation records to one of four positive valence system severities: absent, mild, moderate, or severe. We used a dataset provided by the event organizers to develop a framework comprised of natural language processing (NLP) modules and 3 predictive models (two decision tree models and one Bayesian network model) used in the competition. We also developed two additional predictive models for comparison purpose. To evaluate our framework, we employed a blind test dataset provided by the 2016 CEGS N-GRID. The predictive scores, measured by the macro averaged-inverse normalized mean absolute error score, from the two decision trees and Naïve Bayes models were 82.56%, 82.18%, and 80.56%, respectively. The proposed framework in this paper can potentially be applied to other predictive tasks for processing initial psychiatric evaluation records, such as predicting 30-day psychiatric readmissions.

SUBMITTER: Posada JD

PROVIDER: S-EPMC5705330 | biostudies-literature | 2017 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Predictive modeling for classification of positive valence system symptom severity from initial psychiatric evaluation records.

Posada Jose D JD Barda Amie J AJ Shi Lingyun L Xue Diyang D Ruiz Victor V Kuan Pei-Han PH Ryan Neal D ND Tsui Fuchiang Rich FR

Journal of biomedical informatics 20170529

In response to the challenges set forth by the CEGS N-GRID 2016 Shared Task in Clinical Natural Language Processing, we describe a framework to automatically classify initial psychiatric evaluation records to one of four positive valence system severities: absent, mild, moderate, or severe. We used a dataset provided by the event organizers to develop a framework comprised of natural language processing (NLP) modules and 3 predictive models (two decision tree models and one Bayesian network mode ...[more]

PMID: 28571784

Similar Datasets

Project description:Background and objectiveEfficiently capturing the severity of positive valence symptoms could aid in risk stratification for adverse outcomes among patients with psychiatric disorders and identify optimal treatment strategies for patient subgroups. Motivated by the success of convolutional neural networks (CNNs) in classification tasks, we studied the application of various CNN architectures and their performance in predicting the severity of positive valence symptoms in patients with psychiatric disorders based on initial psychiatric evaluation records.MethodsPsychiatric evaluation records contain unstructured text and semi-structured data such as question-answer pairs. For a given record, we tokenise and normalise the semi-structured content. Pre-processed tokenised words are represented as one-hot encoded word vectors. We then apply different configurations of convolutional and max pooling layers to automatically learn important features from various word representations. We conducted a series of experiments to explore the effect of different CNN architectures on the classification of psychiatric records.ResultsOur best CNN model achieved a mean absolute error (MAE) of 0.539 and a normalized MAE of 0.785 on the test dataset, which is comparable to the other well-known text classification algorithms studied in this work. Our results also suggest that the normalisation step has a great impact on the performance of the developed models.ConclusionsWe demonstrate that normalisation of the semi-structured contents can improve the MAE among all CNN configurations. Without advanced feature engineering, CNN-based approaches can provide a comparable solution for classifying positive valence symptom severity in initial psychiatric evaluation records. Although word embedding is well known for its ability to capture relatively low-dimensional similarity between words, our experimental results show that pre-trained embeddings do not improve the classification performance. This phenomenon may be due to the inability of word embeddings to capture problem specific contextual semantic information implying the quality of the employing embedding is critical for obtaining an accurate CNN model.

Project description:This paper presents a novel method for automatically recognizing symptom severity by using natural language processing of psychiatric evaluation records to extract features that are processed by machine learning techniques to assign a severity score to each record evaluated in the 2016 RDoC for Psychiatry Challenge from CEGS/N-GRID. The natural language processing techniques focused on (a) discerning the discourse information expressed in questions and answers; (b) identifying medical concepts that relate to mental disorders; and (c) accounting for the role of negation. The machine learning techniques rely on the assumptions that (1) the severity of a patient's positive valence symptoms exists on a latent continuous spectrum and (2) all the patient's answers and narratives documented in the psychological evaluation records are informed by the patient's latent severity score along this spectrum. These assumptions motivated our two-step machine learning framework for automatically recognizing psychological symptom severity. In the first step, the latent continuous severity score is inferred from each record; in the second step, the severity score is mapped to one of the four discrete severity levels used in the CEGS/N-GRID challenge. We evaluated three methods for inferring the latent severity score associated with each record: (i) pointwise ridge regression; (ii) pairwise comparison-based classification; and (iii) a hybrid approach combining pointwise regression and the pairwise classifier. The second step was implemented using a tree of cascading support vector machine (SVM) classifiers. While the official evaluation results indicate that all three methods are promising, the hybrid approach not only outperformed the pairwise and pointwise methods, but also produced the second highest performance of all submissions to the CEGS/N-GRID challenge with a normalized MAE score of 84.093% (where higher numbers indicate better performance). These evaluation results enabled us to observe that, for this task, considering pairwise information can produce more accurate severity scores than pointwise regression - an approach widely used in other systems for assigning severity scores. Moreover, our analysis indicates that using a cascading SVM tree outperforms traditional SVM classification methods for the purpose of determining discrete severity levels.

Project description:ObjectiveTo optimally leverage the scalability and unique features of the electronic health records (EHR) for research that would ultimately improve patient care, we need to accurately identify patients and extract clinically meaningful measures. Using multiple sclerosis (MS) as a proof of principle, we showcased how to leverage routinely collected EHR data to identify patients with a complex neurological disorder and derive an important surrogate measure of disease severity heretofore only available in research settings.MethodsIn a cross-sectional observational study, 5,495 MS patients were identified from the EHR systems of two major referral hospitals using an algorithm that includes codified and narrative information extracted using natural language processing. In the subset of patients who receive neurological care at a MS Center where disease measures have been collected, we used routinely collected EHR data to extract two aggregate indicators of MS severity of clinical relevance multiple sclerosis severity score (MSSS) and brain parenchymal fraction (BPF, a measure of whole brain volume).ResultsThe EHR algorithm that identifies MS patients has an area under the curve of 0.958, 83% sensitivity, 92% positive predictive value, and 89% negative predictive value when a 95% specificity threshold is used. The correlation between EHR-derived and true MSSS has a mean R(2) = 0.38±0.05, and that between EHR-derived and true BPF has a mean R(2) = 0.22±0.08. To illustrate its clinical relevance, derived MSSS captures the expected difference in disease severity between relapsing-remitting and progressive MS patients after adjusting for sex, age of symptom onset and disease duration (p = 1.56×10(-12)).ConclusionIncorporation of sophisticated codified and narrative EHR data accurately identifies MS patients and provides estimation of a well-accepted indicator of MS severity that is widely used in research settings but not part of the routine medical records. Similar approaches could be applied to other complex neurological disorders.

Project description:Autism spectrum disorder (ASD) affects 1 in 88 children and is characterized by a complex phenotype, including social, communicative, and sensorimotor deficits. Autism spectrum disorder has been linked with atypical connectivity across multiple brain systems, yet the nature of these differences in young children with the disorder is not well understood.To examine connectivity of large-scale brain networks and determine whether specific networks can distinguish children with ASD from typically developing (TD) children and predict symptom severity in children with ASD.Case-control study performed at Stanford University School of Medicine of 20 children 7 to 12 years old with ASD and 20 age-, sex-, and IQ-matched TD children.Between-group differences in intrinsic functional connectivity of large-scale brain networks, performance of a classifier built to discriminate children with ASD from TD children based on specific brain networks, and correlations between brain networks and core symptoms of ASD.We observed stronger functional connectivity within several large-scale brain networks in children with ASD compared with TD children. This hyperconnectivity in ASD encompassed salience, default mode, frontotemporal, motor, and visual networks. This hyperconnectivity result was replicated in an independent cohort obtained from publicly available databases. Using maps of each individual's salience network, children with ASD could be discriminated from TD children with a classification accuracy of 78%, with 75% sensitivity and 80% specificity. The salience network showed the highest classification accuracy among all networks examined, and the blood oxygen-level dependent signal in this network predicted restricted and repetitive behavior scores. The classifier discriminated ASD from TD in the independent sample with 83% accuracy, 67% sensitivity, and 100% specificity.Salience network hyperconnectivity may be a distinguishing feature in children with ASD. Quantification of brain network connectivity is a step toward developing biomarkers for objectively identifying children with ASD.

Dataset Information

Predictive modeling for classification of positive valence system symptom severity from initial psychiatric evaluation records.

Publications

Predictive modeling for classification of positive valence system symptom severity from initial psychiatric evaluation records.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets