Dataset Information

Applying Machine Learning to Identify Anti-Vaccination Tweets during the COVID-19 Pandemic.

ABSTRACT: Anti-vaccination attitudes have been an issue since the development of the first vaccines. The increasing use of social media as a source of health information may contribute to vaccine hesitancy due to anti-vaccination content widely available on social media, including Twitter. Being able to identify anti-vaccination tweets could provide useful information for formulating strategies to reduce anti-vaccination sentiments among different groups. This study aims to evaluate the performance of different natural language processing models to identify anti-vaccination tweets that were published during the COVID-19 pandemic. We compared the performance of the bidirectional encoder representations from transformers (BERT) and the bidirectional long short-term memory networks with pre-trained GLoVe embeddings (Bi-LSTM) with classic machine learning methods including support vector machine (SVM) and naïve Bayes (NB). The results show that performance on the test set of the BERT model was: accuracy = 91.6%, precision = 93.4%, recall = 97.6%, F1 score = 95.5%, and AUC = 84.7%. Bi-LSTM model performance showed: accuracy = 89.8%, precision = 44.0%, recall = 47.2%, F1 score = 45.5%, and AUC = 85.8%. SVM with linear kernel performed at: accuracy = 92.3%, Precision = 19.5%, Recall = 78.6%, F1 score = 31.2%, and AUC = 85.6%. Complement NB demonstrated: accuracy = 88.8%, precision = 23.0%, recall = 32.8%, F1 score = 27.1%, and AUC = 62.7%. In conclusion, the BERT models outperformed the Bi-LSTM, SVM, and NB models in this task. Moreover, the BERT model achieved excellent performance and can be used to identify anti-vaccination tweets in future studies.

SUBMITTER: To QG

PROVIDER: S-EPMC8069687 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundFamily violence (including intimate partner violence/domestic violence, child abuse, and elder abuse) is a hidden pandemic happening alongside COVID-19. The rates of family violence are rising fast, and women and children are disproportionately affected and vulnerable during this time.ObjectiveThis study aims to provide a large-scale analysis of public discourse on family violence and the COVID-19 pandemic on Twitter.MethodsWe analyzed over 1 million tweets related to family violence and COVID-19 from April 12 to July 16, 2020. We used the machine learning approach Latent Dirichlet Allocation and identified salient themes, topics, and representative tweets.ResultsWe extracted 9 themes from 1,015,874 tweets on family violence and the COVID-19 pandemic: (1) increased vulnerability: COVID-19 and family violence (eg, rising rates, increases in hotline calls, homicide); (2) types of family violence (eg, child abuse, domestic violence, sexual abuse); (3) forms of family violence (eg, physical aggression, coercive control); (4) risk factors linked to family violence (eg, alcohol abuse, financial constraints, guns, quarantine); (5) victims of family violence (eg, the LGBTQ [lesbian, gay, bisexual, transgender, and queer or questioning] community, women, women of color, children); (6) social services for family violence (eg, hotlines, social workers, confidential services, shelters, funding); (7) law enforcement response (eg, 911 calls, police arrest, protective orders, abuse reports); (8) social movements and awareness (eg, support victims, raise awareness); and (9) domestic violence-related news (eg, Tara Reade, Melissa DeRosa).ConclusionsThis study overcomes limitations in the existing scholarship where data on the consequences of COVID-19 on family violence are lacking. We contribute to understanding family violence during the pandemic by providing surveillance via tweets. This is essential for identifying potentially useful policy programs that can offer targeted support for victims and survivors as we prepare for future outbreaks.

Project description:ObjectiveTo investigate determining factors of happiness during the COVID-19 pandemic.DesignObservational study.SettingLarge online surveys in Japan before and during the COVID-19 pandemic.ParticipantsA random sample of 25 482 individuals who are representatives of the Japanese population.Main outcome measureSelf-reported happiness measured using a 10-point Likert scale, where higher scores indicated higher levels of happiness. We defined participants with ≥8 on the scale as having high levels of happiness.ResultsAmong the 25 482 respondents, the median score of self-reported happiness was 7 (IQR 6-8), with 11 418 (45%) reporting high levels of happiness during the pandemic. The multivariable logistic regression model showed that meaning in life, having a spouse, trust in neighbours and female gender were positively associated with happiness (eg, adjusted OR (aOR) for meaning in life 4.17; 95% CI 3.92 to 4.43; p<0.001). Conversely, self-reported poor health, anxiety about future household income, psychiatric diseases except depression and feeling isolated were negatively associated with happiness (eg, aOR for self-reported poor health 0.44; 95% CI 0.39 to 0.48; p<0.001). Using machine-learning methods, we found that meaning in life and social capital (eg, having a spouse and trust in communities) were the strongest positive determinants of happiness, whereas poor health, anxiety about future household income and feeling isolated were important negative determinants of happiness. Among 6965 subjects who responded to questionnaires both before and during the COVID-19 pandemic, there was no systemic difference in the patterns as to determinants of declined happiness during the pandemic.ConclusionUsing machine-learning methods on data from large online surveys in Japan, we found that interventions that have a positive impact on social capital as well as successful pandemic control and economic stimuli may effectively improve the population-level psychological well-being during the COVID-19 pandemic.

Project description:Social media is increasingly being used to express opinions and attitudes toward vaccines. The vaccine stance of social media posts can be classified in almost real-time using machine learning. We describe the use of a Transformer-based machine learning model for analyzing vaccine stance of Italian tweets, and demonstrate the need to address changes over time in vaccine-related language, through periodic model retraining. Vaccine-related tweets were collected through a platform developed for the European Joint Action on Vaccination. Two datasets were collected, the first between November 2019 and June 2020, the second from April to September 2021. The tweets were manually categorized by three independent annotators. After cleaning, the total dataset consisted of 1,736 tweets with 3 categories (promotional, neutral, and discouraging). The manually classified tweets were used to train and test various machine learning models. The model that classified the data most similarly to humans was XLM-Roberta-large, a multilingual version of the Transformer-based model RoBERTa. The model hyper-parameters were tuned and then the model ran five times. The fine-tuned model with the best F-score over the validation dataset was selected. Running the selected fine-tuned model on just the first test dataset resulted in an accuracy of 72.8% (F-score 0.713). Using this model on the second test dataset resulted in a 10% drop in accuracy to 62.1% (F-score 0.617), indicating that the model recognized a difference in language between the datasets. On the combined test datasets the accuracy was 70.1% (F-score 0.689). Retraining the model using data from the first and second datasets increased the accuracy over the second test dataset to 71.3% (F-score 0.713), a 9% improvement from when using just the first dataset for training. The accuracy over the first test dataset remained the same at 72.8% (F-score 0.721). The accuracy over the combined test datasets was then 72.4% (F-score 0.720), a 2% improvement. Through fine-tuning a machine-learning model on task-specific data, the accuracy achieved in categorizing tweets was close to that expected by a single human annotator. Regular training of machine-learning models with recent data is advisable to maximize accuracy.

Dataset Information

Applying Machine Learning to Identify Anti-Vaccination Tweets during the COVID-19 Pandemic.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets