Dataset Information

Text classification models for the automatic detection of nonmedical prescription medication use from social media.

ABSTRACT:

Background

Prescription medication (PM) misuse/abuse has emerged as a national crisis in the United States, and social media has been suggested as a potential resource for performing active monitoring. However, automating a social media-based monitoring system is challenging-requiring advanced natural language processing (NLP) and machine learning methods. In this paper, we describe the development and evaluation of automatic text classification models for detecting self-reports of PM abuse from Twitter.

Methods

We experimented with state-of-the-art bi-directional transformer-based language models, which utilize tweet-level representations that enable transfer learning (e.g., BERT, RoBERTa, XLNet, AlBERT, and DistilBERT), proposed fusion-based approaches, and compared the developed models with several traditional machine learning, including deep learning, approaches. Using a public dataset, we evaluated the performances of the classifiers on their abilities to classify the non-majority "abuse/misuse" class.

Results

Our proposed fusion-based model performs significantly better than the best traditional model (F₁-score [95% CI]: 0.67 [0.64-0.69] vs. 0.45 [0.42-0.48]). We illustrate, via experimentation using varying training set sizes, that the transformer-based models are more stable and require less annotated data compared to the other models. The significant improvements achieved by our best-performing classification model over past approaches makes it suitable for automated continuous monitoring of nonmedical PM use from Twitter.

Conclusions

BERT, BERT-like and fusion-based models outperform traditional machine learning and deep learning models, achieving substantial improvements over many years of past research on the topic of prescription medication misuse/abuse classification from social media, which had been shown to be a complex task due to the unique ways in which information about nonmedical use is presented. Several challenges associated with the lack of context and the nature of social media language need to be overcome to further improve BERT and BERT-like models. These experimental driven challenges are represented as potential future research directions.

SUBMITTER: Al-Garadi MA

PROVIDER: S-EPMC7835447 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Unstructured clinical narratives are continuously being recorded as part of delivery of care in electronic health records, and dedicated tagging staff spend considerable effort manually assigning clinical codes for billing purposes. Despite these efforts, however, label availability and accuracy are both suboptimal. In this retrospective study, we aimed to automate the assignment of top-level International Classification of Diseases version 9 (ICD-9) codes to clinical records from human and veterinary data stores using minimal manual labor and feature curation. Automating top-level annotations could in turn enable rapid cohort identification, especially in a veterinary setting. To this end, we trained long short-term memory (LSTM) recurrent neural networks (RNNs) on 52,722 human and 89,591 veterinary records. We investigated the accuracy of both separate-domain and combined-domain models and probed model portability. We established relevant baseline classification performances by training Decision Trees (DT) and Random Forests (RF). We also investigated whether transforming the data using MetaMap Lite, a clinical natural language processing tool, affected classification performance. We showed that the LSTM-RNNs accurately classify veterinary and human text narratives into top-level categories with an average weighted macro F1 score of 0.74 and 0.68 respectively. In the "neoplasia" category, the model trained on veterinary data had a high validation accuracy in veterinary data and moderate accuracy in human data, with F1 scores of 0.91 and 0.70 respectively. Our LSTM method scored slightly higher than that of the DT and RF models. The use of LSTM-RNN models represents a scalable structure that could prove useful in cohort identification for comparative oncology studies. Digitization of human and veterinary health information will continue to be a reality, particularly in the form of unstructured narratives. Our approach is a step forward for these two domains to learn from and inform one another.

Project description:Depression is a psychological effect of the modern lifestyle on people's thoughts. It is a serious individual and social health problem due to the risk of suicide and loss of workforce, high chronicity, recurrence rates, and prevalence. Therefore, identification, prevention, treatment of depression, and determination of relapse risk factors are of great importance. Depression has traditionally been diagnosed using standardized scales that require clinical diagnoses or patients' subjective responses. However, these classical techniques have some limitations such as cost, uncomfortability, subjectivity, and ineffectiveness. Social media data can be simply and efficiently used for depression detection because it allows instantaneous emotional expression and quick access to various information. Some machine learning-based methods are used for detecting the depression in online social media and networks. Nevertheless, these algorithms suffer from several drawbacks, including data sparsity, dimension explosion, restricted capacity for generalization, and low performance on imbalanced data sets. Furthermore, many machine learning methods work as black-box models, and the constructed depression detection models are not interpretable and explainable. Intelligent metaheuristic optimization algorithms are widely used for different types of complex real-world problems due to their simplicity and high performance. It is aimed to remove the limitations of studies on this problem by increasing the success rate and automatically selecting the relevant features and integrating the explainability. In this study, new chaos-integrated multi-objective optimization algorithms are proposed to increase efficiency. New improved Grey Wolf Optimization algorithms have been proposed by integrating Circle, Logistic, and Iterative chaotic maps into the improved Grey Wolf Optimization algorithm. It is aimed to increase the success rate by proposing a multi-objective fitness function that can optimize the accuracy and the number of features simultaneously. The proposed algorithms are compared with different types of popular supervised machine learning algorithms and current metaheuristic algorithms that are widely and successfully used in depression detection problems. Experimental results show that the proposed models outperform machine learning methods, as evidenced by examining results with accuracy, F-measure, MCC, sensitivity, and precision measures. An accuracy value of 100% was obtained from proposed algorithms. In addition, when the confusion matrices are examined, it is seen that they exhibit a successful distribution. Although it is a new research and application area for optimization theory, promising results have been obtained from the proposed models.