A performance comparison of supervised machine learning models for Covid-19 tweets sentiment analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Rustam F
PROVIDER: S-EPMC7906356 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:COVID-19, caused by SARS-CoV2 infection, varies greatly in its severity but presents with serious respiratory symptoms with vascular and other complications, particularly in older adults. The disease can be spread by both symptomatic and asymptomatic infected individuals. Uncertainty remains over key aspects of the virus infectiousness (particularly the newly emerging variants) and the disease has had severe economic impacts globally. For these reasons, COVID-19 is the subject of intense and widespread discussion on social media platforms including Facebook and Twitter. These public forums substantially influence public opinions and in some cases can exacerbate the widespread panic and misinformation spread during the crisis. Thus, this work aimed to design an intelligent clustering-based classification and topic extracting model named TClustVID that analyzes COVID-19-related public tweets to extract significant sentiments with high accuracy. We gathered COVID-19 Twitter datasets from the IEEE Dataport repository and employed a range of data preprocessing methods to clean the raw data, then applied tokenization and produced a word-to-index dictionary. Thereafter, different classifications were employed on these datasets which enabled the exploration of the performance of traditional classification and TClustVID. Our analysis found that TClustVID showed higher performance compared to traditional methodologies that are determined by clustering criteria. Finally, we extracted significant topics from the clusters, split them into positive, neutral and negative sentiments, and identified the most frequent topics using the proposed model. This approach is able to rapidly identify commonly prevailing aspects of public opinions and attitudes related to COVID-19 and infection prevention strategies spreading among different populations.
| S-EPMC8099549 | biostudies-literature
Project description:BackgroundAs the COVID-19 pandemic progressed, disinformation, fake news, and conspiracy theories spread through many parts of society. However, the disinformation spreading through social media is, according to the literature, one of the causes of increased COVID-19 vaccine hesitancy. In this context, the analysis of social media posts is particularly important, but the large amount of data exchanged on social media platforms requires specific methods. This is why machine learning and natural language processing models are increasingly applied to social media data.ObjectiveThe aim of this study is to examine the capability of the CamemBERT French-language model to faithfully predict the elaborated categories, with the knowledge that tweets about vaccination are often ambiguous, sarcastic, or irrelevant to the studied topic.MethodsA total of 901,908 unique French-language tweets related to vaccination published between July 12, 2021, and August 11, 2021, were extracted using Twitter's application programming interface (version 2; Twitter Inc). Approximately 2000 randomly selected tweets were labeled with 2 types of categorizations: (1) arguments for (pros) or against (cons) vaccination (health measures included) and (2) type of content (scientific, political, social, or vaccination status). The CamemBERT model was fine-tuned and tested for the classification of French-language tweets. The model's performance was assessed by computing the F1-score, and confusion matrices were obtained.ResultsThe accuracy of the applied machine learning reached up to 70.6% for the first classification (pro and con tweets) and up to 90% for the second classification (scientific and political tweets). Furthermore, a tweet was 1.86 times more likely to be incorrectly classified by the model if it contained fewer than 170 characters (odds ratio 1.86; 95% CI 1.20-2.86).ConclusionsThe accuracy of the model is affected by the classification chosen and the topic of the message examined. When the vaccine debate is jostled by contested political decisions, tweet content becomes so heterogeneous that the accuracy of the model drops for less differentiated classes. However, our tests showed that it is possible to improve the accuracy by selecting tweets using a new method based on tweet length.
| S-EPMC9116457 | biostudies-literature
Project description:BackgroundCOVID-19 vaccines are one of the most effective preventive strategies for containing the pandemic. Having a better understanding of the public's conceptions of COVID-19 vaccines may aid in the effort to promptly and thoroughly vaccinate the community. However, because no empirical research has yet fully explored the public's vaccine awareness through sentiment-based topic modeling, little is known about the evolution of public attitude since the rollout of COVID-19 vaccines.ObjectiveIn this study, we specifically focused on tweets about COVID-19 vaccines (Pfizer, Moderna, AstraZeneca, and Johnson & Johnson) after vaccines became publicly available. We aimed to explore the overall sentiments and topics of tweets about COVID-19 vaccines, as well as how such sentiments and main concerns evolved.MethodsWe collected 1,122,139 tweets related to COVID-19 vaccines from December 14, 2020, to April 30, 2021, using Twitter's application programming interface. We removed retweets and duplicate tweets to avoid data redundancy, which resulted in 857,128 tweets. We then applied sentiment-based topic modeling by using the compound score to determine sentiment polarity and the coherence score to determine the optimal topic number for different sentiment polarity categories. Finally, we calculated the topic distribution to illustrate the topic evolution of main concerns.ResultsOverall, 398,661 (46.51%) were positive, 204,084 (23.81%) were negative, 245,976 (28.70%) were neutral, 6899 (0.80%) were highly positive, and 1508 (0.18%) were highly negative sentiments. The main topics of positive and highly positive tweets were planning for getting vaccination (251,979/405,560, 62.13%), getting vaccination (76,029/405,560, 18.75%), and vaccine information and knowledge (21,127/405,560, 5.21%). The main concerns in negative and highly negative tweets were vaccine hesitancy (115,206/205,592, 56.04%), extreme side effects of the vaccines (19,690/205,592, 9.58%), and vaccine supply and rollout (17,154/205,592, 8.34%). During the study period, negative sentiment trends were stable, while positive sentiments could be easily influenced. Topic heatmap visualization demonstrated how main concerns changed during the current widespread vaccination campaign.ConclusionsTo the best of our knowledge, this is the first study to evaluate public COVID-19 vaccine awareness and awareness trends on social media with automated sentiment-based topic modeling after vaccine rollout. Our results can help policymakers and research communities track public attitudes toward COVID-19 vaccines and help them make decisions to promote the vaccination campaign.
| S-EPMC8827037 | biostudies-literature
Project description:How different cultures react and respond given a crisis is predominant in a society's norms and political will to combat the situation. Often, the decisions made are necessitated by events, social pressure, or the need of the hour, which may not represent the nation's will. While some are pleased with it, others might show resentment. Coronavirus (COVID-19) brought a mix of similar emotions from the nations towards the decisions taken by their respective governments. Social media was bombarded with posts containing both positive and negative sentiments on the COVID-19, pandemic, lockdown, and hashtags past couple of months. Despite geographically close, many neighboring countries reacted differently to one another. For instance, Denmark and Sweden, which share many similarities, stood poles apart on the decision taken by their respective governments. Yet, their nation's support was mostly unanimous, unlike the South Asian neighboring countries where people showed a lot of anxiety and resentment. The purpose of this study is to analyze reaction of citizens from different cultures to the novel Coronavirus and people's sentiment about subsequent actions taken by different countries. Deep long short-term memory (LSTM) models used for estimating the sentiment polarity and emotions from extracted tweets have been trained to achieve state-of-the-art accuracy on the sentiment140 dataset. The use of emoticons showed a unique and novel way of validating the supervised deep learning models on tweets extracted from Twitter.
| S-EPMC8545282 | biostudies-literature
Project description:Anti-vaccination attitudes have been an issue since the development of the first vaccines. The increasing use of social media as a source of health information may contribute to vaccine hesitancy due to anti-vaccination content widely available on social media, including Twitter. Being able to identify anti-vaccination tweets could provide useful information for formulating strategies to reduce anti-vaccination sentiments among different groups. This study aims to evaluate the performance of different natural language processing models to identify anti-vaccination tweets that were published during the COVID-19 pandemic. We compared the performance of the bidirectional encoder representations from transformers (BERT) and the bidirectional long short-term memory networks with pre-trained GLoVe embeddings (Bi-LSTM) with classic machine learning methods including support vector machine (SVM) and naïve Bayes (NB). The results show that performance on the test set of the BERT model was: accuracy = 91.6%, precision = 93.4%, recall = 97.6%, F1 score = 95.5%, and AUC = 84.7%. Bi-LSTM model performance showed: accuracy = 89.8%, precision = 44.0%, recall = 47.2%, F1 score = 45.5%, and AUC = 85.8%. SVM with linear kernel performed at: accuracy = 92.3%, Precision = 19.5%, Recall = 78.6%, F1 score = 31.2%, and AUC = 85.6%. Complement NB demonstrated: accuracy = 88.8%, precision = 23.0%, recall = 32.8%, F1 score = 27.1%, and AUC = 62.7%. In conclusion, the BERT models outperformed the Bi-LSTM, SVM, and NB models in this task. Moreover, the BERT model achieved excellent performance and can be used to identify anti-vaccination tweets in future studies.
| S-EPMC8069687 | biostudies-literature
Project description:BackgroundEmerging evidence suggests that people with arthritis are reporting increased physical pain and psychological distress during the COVID-19 pandemic. At the same time, Twitter's daily usage has surged by 23% throughout the pandemic period, presenting a unique opportunity to assess the content and sentiment of tweets. Individuals with arthritis use Twitter to communicate with peers, and to receive up-to-date information from health professionals and services about novel therapies and management techniques.ObjectiveThe aim of this research was to identify proxy topics of importance for individuals with arthritis during the COVID-19 pandemic, and to explore the emotional context of tweets by people with arthritis during the early phase of the pandemic.MethodsFrom March 20 to April 20, 2020, publicly available tweets posted in English and with hashtag combinations related to arthritis and COVID-19 were extracted retrospectively from Twitter. Content analysis was used to identify common themes within tweets, and sentiment analysis was used to examine positive and negative emotions in themes to understand the COVID-19 experiences of people with arthritis.ResultsIn total, 149 tweets were analyzed. The majority of tweeters were female and were from the United States. Tweeters reported a range of arthritis conditions, including rheumatoid arthritis, systemic lupus erythematosus, and psoriatic arthritis. Seven themes were identified: health care experiences, personal stories, links to relevant blogs, discussion of arthritis-related symptoms, advice sharing, messages of positivity, and stay-at-home messaging. Sentiment analysis demonstrated marked anxiety around medication shortages, increased physical symptom burden, and strong desire for trustworthy information and emotional connection.ConclusionsTweets by people with arthritis highlight the multitude of concurrent concerns during the COVID-19 pandemic. Understanding these concerns, which include heightened physical and psychological symptoms in the context of treatment misinformation, may assist clinicians to provide person-centered care during this time of great health uncertainty.
| S-EPMC7746504 | biostudies-literature
Project description:The COVID-19 pandemic has had a significant impact on public life and health worldwide, putting the world's healthcare systems at risk. The first step in stopping this outbreak is to detect the infection in its early stages, which will relieve the risk, control the outbreak's spread, and restore full functionality to the world's healthcare systems. Currently, PCR is the most prevalent diagnosis tool for COVID-19. However, chest X-ray images may play an essential role in detecting this disease, as they are successful for many other viral pneumonia diseases. Unfortunately, there are common features between COVID-19 and other viral pneumonia, and hence manual differentiation between them seems to be a critical problem and needs the aid of artificial intelligence. This research employs deep- and transfer-learning techniques to develop accurate, general, and robust models for detecting COVID-19. The developed models utilize either convolutional neural networks or transfer-learning models or hybridize them with powerful machine-learning techniques to exploit their full potential. For experimentation, we applied the proposed models to two data sets: the COVID-19 Radiography Database from Kaggle and a local data set from Asir Hospital, Abha, Saudi Arabia. The proposed models achieved promising results in detecting COVID-19 cases and discriminating them from normal and other viral pneumonia with excellent accuracy. The hybrid models extracted features from the flatten layer or the first hidden layer of the neural network and then fed these features into a classification algorithm. This approach enhanced the results further to full accuracy for binary COVID-19 classification and 97.8% for multiclass classification.
| S-EPMC8357494 | biostudies-literature
Project description:The COVID-19 pandemic resulted in a large expansion of telehealth, but little is known about user sentiment. Tweets containing the terms "telehealth" and "telemedicine" were extracted (n = 192,430) from the official Twitter API between November 2019 and April 2020. A random subset of 2000 tweets was annotated by trained readers to classify tweets according to their content, including telehealth, sentiment, user type, and relation to COVID-19. A state-of-the-art NLP model (Bidirectional Encoder Representations from Transformers, BERT) was used to categorize the remaining tweets. Following a low and fairly stable level of activity, telehealth tweets rose dramatically beginning the first week of March 2020. The sentiment was overwhelmingly positive or neutral, with only a small percentage of negative tweets. Users included patients, clinicians, vendors (entities that promote the use of telehealth technology or services), and others, which represented the largest category. No significant differences were seen in sentiment across user groups. The COVID-19 pandemic produced a large increase in user tweets related to telehealth and COVID-19, and user sentiment suggests that most people feel positive or neutral about telehealth.
| S-EPMC8230122 | biostudies-literature
Project description:COVID-19 originally known as Corona VIrus Disease of 2019, has been declared as a pandemic by World Health Organization (WHO) on 11th March 2020. Unprecedented pressures have mounted on each country to make compelling requisites for controlling the population by assessing the cases and properly utilizing available resources. The rapid number of exponential cases globally has become the apprehension of panic, fear and anxiety among people. The mental and physical health of the global population is found to be directly proportional to this pandemic disease. The current situation has reported more than twenty four million people being tested positive worldwide as of 27th August, 2020. Therefore, it is the need of the hour to implement different measures to safeguard the countries by demystifying the pertinent facts and information. This paper aims to bring out the fact that tweets containing all handles related to COVID-19 and WHO have been unsuccessful in guiding people around this pandemic outbreak appositely. This study analyzes two types of tweets gathered during the pandemic times. In one case, around twenty three thousand most re-tweeted tweets within the time span from 1st Jan 2019 to 23rd March 2020 have been analyzed and observation says that the maximum number of the tweets portrays neutral or negative sentiments. On the other hand, a dataset containing 226,668 tweets collected within the time span between December 2019 and May 2020 have been analyzed which contrastingly show that there were a maximum number of positive and neutral tweets tweeted by netizens. The research demonstrates that though people have tweeted mostly positive regarding COVID-19, yet netizens were busy engrossed in re-tweeting the negative tweets and that no useful words could be found in WordCloud or computations using word frequency in tweets. The claims have been validated through a proposed model using deep learning classifiers with admissible accuracy up to 81%. Apart from these the authors have proposed the implementation of a Gaussian membership function based fuzzy rule base to correctly identify sentiments from tweets. The accuracy for the said model yields up to a permissible rate of 79%.
| S-EPMC7521435 | biostudies-literature
Project description:Background: Many countries show low COVID-19 vaccination rates despite high levels of readiness and delivery of vaccines. The public's misperceptions, hesitancy, and negative emotions toward vaccines are psychological factors discouraging vaccination. At the individual level, studies have revealed negative perceptual/behavioral outcomes of COVID-19 information exposure via social media where misinformation and vaccine fear flood. Objective: This study extends research context to the global level and investigates social media discourse on the COVID-19 vaccine and its association with vaccination rates of 192 countries in the world. Methods: COVID-19 vaccine tweets were compared by country in terms of (1) the number per million Twitter users, (2) mentions of adverse events-death, side-effects, blood clots, (3) negative sentiment (vs. positive), and (4) fear, sadness, or anger emotions (vs. joy). Artificial intelligence (AI) was adopted to classify sentiment and emotions. Such tweets and covariates (COVID-19 morbidity and mortality rates, GDP, population size and density, literacy rate, democracy index, institutional quality, human development index) were tested as predictors of vaccination rates in countries. Results: Over 21.3 million COVID-19 vaccine tweets posted between November 2020 and August 2021 worldwide were included in our analysis. The global average of COVID-19 vaccine tweets mentioning adverse events was 2% for 'death', 1.15% for 'side-effects', and 0.80% for 'blood clots'. Negative sentiment appeared 1.90 times more frequently than positive sentiment. Fear, anger, or sadness appeared 0.70 times less frequently than joy. The mention of 'side-effects' and fear/sadness/anger emotions appeared as significant predictors of vaccination rates, along with the human development index. Conclusions: Our findings indicate that global efforts to combat misinformation, address negative emotions, and promote positive languages surrounding COVID-19 vaccination on social media may help increase global vaccination uptakes.
| S-EPMC9146864 | biostudies-literature