Dataset Information

Online hate network spreads malicious COVID-19 content outside the control of individual social media platforms.

ABSTRACT: We show that malicious COVID-19 content, including racism, disinformation, and misinformation, exploits the multiverse of online hate to spread quickly beyond the control of any individual social media platform. We provide a first mapping of the online hate network across six major social media platforms. We demonstrate how malicious content can travel across this network in ways that subvert platform moderation efforts. Machine learning topic analysis shows quantitatively how online hate communities are sharpening COVID-19 as a weapon, with topics evolving rapidly and content becoming increasingly coherent. Based on mathematical modeling, we provide predictions of how changes to content moderation policies can slow the spread of malicious content.

SUBMITTER: Velasquez N

PROVIDER: S-EPMC8206165 | biostudies-literature | 2021 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Online hate network spreads malicious COVID-19 content outside the control of individual social media platforms.

Velásquez N N Leahy R R Restrepo N Johnson NJ Lupu Y Y Sear R R Gabriel N N Jha O K OK Goldberg B B Johnson N F NF

Scientific reports 20210615 1

We show that malicious COVID-19 content, including racism, disinformation, and misinformation, exploits the multiverse of online hate to spread quickly beyond the control of any individual social media platform. We provide a first mapping of the online hate network across six major social media platforms. We demonstrate how malicious content can travel across this network in ways that subvert platform moderation efforts. Machine learning topic analysis shows quantitatively how online hate commun ...[more]

PMID: 34131158

Similar Datasets

Project description:BackgroundOnline advertisements on social media platforms are an important tool for engaging relevant populations in public health research. However, little is known about what platforms and ad characteristics are most effective in engaging high-priority HIV populations, including racial/ethnic and sexual minority individuals.MethodsData from this study were drawn from advertising campaigns conducted on popular websites and social media platforms that recruited for several nationwide randomized controlled trials of various HIV prevention and testing strategies among sexual minority men (SMM) from December 2019 until March 2022. Descriptive statistics and LASSO regression models were used to determine which platforms and ad characteristics were associated with significantly higher odds of engagement.ResultsAds on Google search, Facebook, and Instagram yielded the most cost-effective engagement, while gay-oriented dating platforms and TrafficJunky yielded the highest percentage of users who appeared to meet basic eligibility criteria. The highest percentages of Black users were screened through ads on Jack'd, TrafficJunky, and Google search; for Hispanic/Latino users, Google search, Grindr, Facebook, and Instagram. Analyzing ad characteristics, we found ads that used suggestive content, animation, and included study or institution logos were associated with greater engagement. Ads that emphasized convenience of the research (e.g. mentioned participating "from home") and that depicted people of similar races/ethnicities were also associated with greater engagement among Black and Hispanic/Latino sexual minority men.ConclusionsWe found that advertisements on mainstream social media sites are most cost effective. Although gay-oriented dating platforms were much more effective at reaching the target population, they were considerably more expensive. We also identified ad characteristics that were particularly effective in engaging users. These results could inform the design of online public health outreach campaigns for similar populations to improve their engagement and reach. Findings also demonstrated the value of conducting focused research on the effectiveness of various online marketing strategies.

Project description:Disparate biases associated with datasets and trained classifiers in hateful and abusive content identification tasks have raised many concerns recently. Although the problem of biased datasets on abusive language detection has been addressed more frequently, biases arising from trained classifiers have not yet been a matter of concern. In this paper, we first introduce a transfer learning approach for hate speech detection based on an existing pre-trained language model called BERT (Bidirectional Encoder Representations from Transformers) and evaluate the proposed model on two publicly available datasets that have been annotated for racism, sexism, hate or offensive content on Twitter. Next, we introduce a bias alleviation mechanism to mitigate the effect of bias in training set during the fine-tuning of our pre-trained BERT-based model for hate speech detection. Toward that end, we use an existing regularization method to reweight input samples, thereby decreasing the effects of high correlated training set' s n-grams with class labels, and then fine-tune our pre-trained BERT-based model with the new re-weighted samples. To evaluate our bias alleviation mechanism, we employed a cross-domain approach in which we use the trained classifiers on the aforementioned datasets to predict the labels of two new datasets from Twitter, AAE-aligned and White-aligned groups, which indicate tweets written in African-American English (AAE) and Standard American English (SAE), respectively. The results show the existence of systematic racial bias in trained classifiers, as they tend to assign tweets written in AAE from AAE-aligned group to negative classes such as racism, sexism, hate, and offensive more often than tweets written in SAE from White-aligned group. However, the racial bias in our classifiers reduces significantly after our bias alleviation mechanism is incorporated. This work could institute the first step towards debiasing hate speech and abusive language detection systems.

Dataset Information

Online hate network spreads malicious COVID-19 content outside the control of individual social media platforms.

Publications

Online hate network spreads malicious COVID-19 content outside the control of individual social media platforms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets