Dataset Information

Algorithmic bias in social research: A meta-analysis.

ABSTRACT: Both the natural and the social sciences are currently facing a deep "reproducibility crisis". Two important factors in this crisis have been the selective reporting of results and methodological problems. In this article, we examine a fusion of these two factors. More specifically, we demonstrate that the uncritical import of Boolean optimization algorithms from electrical engineering into some areas of the social sciences in the late 1980s has induced algorithmic bias on a considerable scale over the last quarter century. Potentially affected are all studies that have used a method nowadays known as Qualitative Comparative Analysis (QCA). Drawing on replication material for 215 peer-reviewed QCA articles from across 109 high-profile management, political science and sociology journals, we estimate the extent this problem has assumed in empirical work. Our results suggest that one in three studies is affected, one in ten severely so. More generally, our article cautions scientists against letting methods and algorithms travel too easily across disparate disciplines without sufficient prior evaluation of their suitability for the context in hand.

SUBMITTER: Thiem A

PROVIDER: S-EPMC7279593 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Algorithmic bias in social research: A meta-analysis.

Thiem Alrik A Mkrtchyan Lusine L Haesebrouck Tim T Sanchez David D

PloS one 20200608 6

Both the natural and the social sciences are currently facing a deep "reproducibility crisis". Two important factors in this crisis have been the selective reporting of results and methodological problems. In this article, we examine a fusion of these two factors. More specifically, we demonstrate that the uncritical import of Boolean optimization algorithms from electrical engineering into some areas of the social sciences in the late 1980s has induced algorithmic bias on a considerable scale o ...[more]

PMID: 32511249

Similar Datasets

Project description:BackgroundCognitive bias modification (CBM), a set of techniques for modifying bias in information processing-is considered a novel intervention for social anxiety disorder (SAD), which has drawn considerable interest from researchers. However, the effects of CBM on SAD are not consistent. Some studies have demonstrated significant positive effects compared to control groups, while others have found no such effects.AimsWe conducted a meta-analysis aimed at quantitatively assessing the effects of CBM on SAD at post-test.MethodThrough a systematic literature search by two independent raters, 34 articles (36 randomized studies) including 2,550 participants were identified. A multilevel modeling approach was employed to assess the effects of CBM on SAD, and to explore the potentially crucial procedures and sample characteristics that enhance the effectiveness of benign training.ResultsIn general, there were small but significant effects of CBM on the primary symptoms of SAD (g = 0.17), cognitive bias (CB) toward threat (g = 0.32), and reactivity in stressful situations (g = 0.25), but non-significant effects on secondary symptoms. However, the interpretation modification program was more effective than was attentional bias modification in reducing SAD primary symptoms and negative CB. Laboratory training procedures produced larger primary symptom reductions compared to Internet-based training, whereas the percentage of contingency and feedback about training performance boosted cognitive effects only. Finally, the following groups were more likely to benefit from CBM: younger participants (primary symptoms and cognitive effects), women (primary symptom effects), and samples with stronger CB (stressor effects). The quality of the randomized controlled trials was less than desirable, as there was some indication of publication bias in our study.ConclusionsCurrent findings broadly supported cognitive theories of SAD that consider a bidirectional or mutually reinforcing relationship between symptoms and CBs. However, the small therapeutic effect observed here indicates that it is necessary to develop more reliable and efficient CBM interventions that are specific to SAD.

Project description:Background and objectiveResearch collaborations in biomedical research have evolved over time. No studies have addressed research collaboration in network meta-analysis (NMA). In this study, we used social network analysis methods to characterize global collaboration patterns of published NMAs over the past decades.MethodsPubMed, EMBASE, Web of Science and the Cochrane Library were searched (at 9th July, 2015) to include systematic reviews incorporating NMA. Two reviewers independently selected studies and cross-checked the standardized data. Data was analyzed using Ucinet 6.0 and SPSS 17.0. NetDraw software was used to draw social networks.Results771 NMAs published in 336 journals from 3459 authors and 1258 institutions in 49 countries through the period 1997-2015 were included. More than three-quarters (n = 625; 81.06%) of the NMAs were published in the last 5-years. The BMJ (4.93%), Current Medical Research and Opinion (4.67%) and PLOS One (4.02%) were the journals that published the greatest number of NMAs. The UK and the USA (followed by Canada, China, the Netherlands, Italy and Germany) headed the absolute global productivity ranking in number of NMAs. The top 20 authors and institutions with the highest publication rates were identified. Overall, 43 clusters of authors (four major groups: one with 37 members, one with 12 members, one with 11 members and one with 10 members) and 21 clusters of institutions (two major groups: one with 62 members and one with 20 members) were identified. The most prolific authors were affiliated with academic institutions and private consulting firms. 181 consulting firms and pharmaceutical industries (14.39% of institutions) were involved in 199 NMAs (25.81% of total publications). Although there were increases in international and inter-institution collaborations, the research collaboration by authors, institutions and countries were still weak and most collaboration groups were small sizes.ConclusionScientific production on NMA is increasing worldwide with research leadership of Western countries (most notably, the UK, the USA and Canada). More authors, institutions and nations are becoming involved in research collaborations, but frequently with limited international collaborations.

Dataset Information

Algorithmic bias in social research: A meta-analysis.

Publications

Algorithmic bias in social research: A meta-analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets