Algorithmic bias in social research: A meta-analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Thiem A
PROVIDER: S-EPMC7279593 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
Algorithmic bias in social research: A meta-analysis.
Thiem Alrik A Mkrtchyan Lusine L Haesebrouck Tim T Sanchez David D
PloS one 20200608 6
Both the natural and the social sciences are currently facing a deep "reproducibility crisis". Two important factors in this crisis have been the selective reporting of results and methodological problems. In this article, we examine a fusion of these two factors. More specifically, we demonstrate that the uncritical import of Boolean optimization algorithms from electrical engineering into some areas of the social sciences in the late 1980s has induced algorithmic bias on a considerable scale o ...[more]
Similar Datasets
Project description:BackgroundCognitive bias modification (CBM), a set of techniques for modifying bias in information processing-is considered a novel intervention for social anxiety disorder (SAD), which has drawn considerable interest from researchers. However, the effects of CBM on SAD are not consistent. Some studies have demonstrated significant positive effects compared to control groups, while others have found no such effects.AimsWe conducted a meta-analysis aimed at quantitatively assessing the effects of CBM on SAD at post-test.MethodThrough a systematic literature search by two independent raters, 34 articles (36 randomized studies) including 2,550 participants were identified. A multilevel modeling approach was employed to assess the effects of CBM on SAD, and to explore the potentially crucial procedures and sample characteristics that enhance the effectiveness of benign training.ResultsIn general, there were small but significant effects of CBM on the primary symptoms of SAD (g = 0.17), cognitive bias (CB) toward threat (g = 0.32), and reactivity in stressful situations (g = 0.25), but non-significant effects on secondary symptoms. However, the interpretation modification program was more effective than was attentional bias modification in reducing SAD primary symptoms and negative CB. Laboratory training procedures produced larger primary symptom reductions compared to Internet-based training, whereas the percentage of contingency and feedback about training performance boosted cognitive effects only. Finally, the following groups were more likely to benefit from CBM: younger participants (primary symptoms and cognitive effects), women (primary symptom effects), and samples with stronger CB (stressor effects). The quality of the randomized controlled trials was less than desirable, as there was some indication of publication bias in our study.ConclusionsCurrent findings broadly supported cognitive theories of SAD that consider a bidirectional or mutually reinforcing relationship between symptoms and CBs. However, the small therapeutic effect observed here indicates that it is necessary to develop more reliable and efficient CBM interventions that are specific to SAD.
| S-EPMC5383070 | biostudies-literature
Project description:As algorithms are increasingly applied to screen applicants for high-stakes decisions in employment, lending, and other domains, concerns have been raised about the effects of algorithmic monoculture, in which many decision-makers all rely on the same algorithm. This concern invokes analogies to agriculture, where a monocultural system runs the risk of severe harm from unexpected shocks. Here, we show that the dangers of algorithmic monoculture run much deeper, in that monocultural convergence on a single algorithm by a group of decision-making agents, even when the algorithm is more accurate for any one agent in isolation, can reduce the overall quality of the decisions being made by the full collection of agents. Unexpected shocks are therefore not needed to expose the risks of monoculture; it can hurt accuracy even under "normal" operations and even for algorithms that are more accurate when used by only a single decision-maker. Our results rely on minimal assumptions and involve the development of a probabilistic framework for analyzing systems that use multiple noisy estimates of a set of alternatives.
| S-EPMC8179131 | biostudies-literature
Project description:Background and objectiveResearch collaborations in biomedical research have evolved over time. No studies have addressed research collaboration in network meta-analysis (NMA). In this study, we used social network analysis methods to characterize global collaboration patterns of published NMAs over the past decades.MethodsPubMed, EMBASE, Web of Science and the Cochrane Library were searched (at 9th July, 2015) to include systematic reviews incorporating NMA. Two reviewers independently selected studies and cross-checked the standardized data. Data was analyzed using Ucinet 6.0 and SPSS 17.0. NetDraw software was used to draw social networks.Results771 NMAs published in 336 journals from 3459 authors and 1258 institutions in 49 countries through the period 1997-2015 were included. More than three-quarters (n = 625; 81.06%) of the NMAs were published in the last 5-years. The BMJ (4.93%), Current Medical Research and Opinion (4.67%) and PLOS One (4.02%) were the journals that published the greatest number of NMAs. The UK and the USA (followed by Canada, China, the Netherlands, Italy and Germany) headed the absolute global productivity ranking in number of NMAs. The top 20 authors and institutions with the highest publication rates were identified. Overall, 43 clusters of authors (four major groups: one with 37 members, one with 12 members, one with 11 members and one with 10 members) and 21 clusters of institutions (two major groups: one with 62 members and one with 20 members) were identified. The most prolific authors were affiliated with academic institutions and private consulting firms. 181 consulting firms and pharmaceutical industries (14.39% of institutions) were involved in 199 NMAs (25.81% of total publications). Although there were increases in international and inter-institution collaborations, the research collaboration by authors, institutions and countries were still weak and most collaboration groups were small sizes.ConclusionScientific production on NMA is increasing worldwide with research leadership of Western countries (most notably, the UK, the USA and Canada). More authors, institutions and nations are becoming involved in research collaborations, but frequently with limited international collaborations.
| S-EPMC5042468 | biostudies-literature
Project description:The availability of data on digital traces is growing to unprecedented sizes, but inferring actionable knowledge from large-scale data is far from being trivial. This is especially important for computational finance, where digital traces of human behaviour offer a great potential to drive trading strategies. We contribute to this by providing a consistent approach that integrates various datasources in the design of algorithmic traders. This allows us to derive insights into the principles behind the profitability of our trading strategies. We illustrate our approach through the analysis of Bitcoin, a cryptocurrency known for its large price fluctuations. In our analysis, we include economic signals of volume and price of exchange for USD, adoption of the Bitcoin technology and transaction volume of Bitcoin. We add social signals related to information search, word of mouth volume, emotional valence and opinion polarization as expressed in tweets related to Bitcoin for more than 3 years. Our analysis reveals that increases in opinion polarization and exchange volume precede rising Bitcoin prices, and that emotional valence precedes opinion polarization and rising exchange volumes. We apply these insights to design algorithmic trading strategies for Bitcoin, reaching very high profits in less than a year. We verify this high profitability with robust statistical methods that take into account risk and trading costs, confirming the long-standing hypothesis that trading-based social media sentiment has the potential to yield positive returns on investment.
| S-EPMC4593685 | biostudies-literature
Project description:BackgroundCurrent global estimates suggest the proportion of the population with autism spectrum disorder (ASD) who have intellectual disability (ID) is approximately 50%. Our objective was to ascertain the existence of selection bias due to under-inclusion of populations with ID across all fields of autism research. A sub-goal was to evaluate inconsistencies in reporting of findings.MethodsThis review covers all original research published in 2016 in autism-specific journals with an impact factor greater than 3. Across 301 included studies, 100,245 participants had ASD. A random effects meta-analysis was used to estimate the proportion of participants without ID. Selection bias was defined as where more than 75% of participants did not have ID.ResultsMeta-analysis estimated 94% of all participants identified as being on the autism spectrum in the studies reviewed did not have ID (95% CI 0.91-0.97). Eight out of ten studies demonstrated selection bias against participants with ID. The reporting of participant characteristics was generally poor: information about participants' intellectual ability was absent in 38% of studies (n = 114). Where there was selection bias on ID, only 31% of studies mentioned lack of generalisability as a limitation.ConclusionsWe found selection bias against ID throughout all fields of autism research. We recommend transparent reporting about ID and strategies for inclusion for this much marginalised group.
| S-EPMC6397505 | biostudies-literature
Project description:Because negative findings have less chance of getting published, available studies tend to be a biased sample. This leads to an inflation of effect size estimates to an unknown degree. To see how meta-analyses in education account for publication bias, we surveyed all meta-analyses published in the last five years in the Review of Educational Research and Educational Research Review. The results show that meta-analyses usually neglect publication bias adjustment. In the minority of meta-analyses adjusting for bias, mostly non-principled adjustment methods were used, and only rarely were the conclusions based on corrected estimates, rendering the adjustment inconsequential. It is argued that appropriate state-of-the-art adjustment (e.g., selection models) should be attempted by default, yet one needs to take into account the uncertainty inherent in any meta-analytic inference under bias. We conclude by providing practical recommendations on dealing with publication bias.
| S-EPMC8174709 | biostudies-literature
Project description:BackgroundThe validity of research synthesis is threatened if published studies comprise a biased selection of all studies that have been conducted. We conducted a meta-analysis to ascertain the strength and consistency of the association between study results and formal publication.MethodsThe Cochrane Methodology Register Database, MEDLINE and other electronic bibliographic databases were searched (to May 2009) to identify empirical studies that tracked a cohort of studies and reported the odds of formal publication by study results. Reference lists of retrieved articles were also examined for relevant studies. Odds ratios were used to measure the association between formal publication and significant or positive results. Included studies were separated into subgroups according to starting time of follow-up, and results from individual cohort studies within the subgroups were quantitatively pooled.ResultsWe identified 12 cohort studies that followed up research from inception, four that included trials submitted to a regulatory authority, 28 that assessed the fate of studies presented as conference abstracts, and four cohort studies that followed manuscripts submitted to journals. The pooled odds ratio of publication of studies with positive results, compared to those without positive results (publication bias) was 2.78 (95% CI: 2.10 to 3.69) in cohorts that followed from inception, 5.00 (95% CI: 2.01 to 12.45) in trials submitted to regulatory authority, 1.70 (95% CI: 1.44 to 2.02) in abstract cohorts, and 1.06 (95% CI: 0.80 to 1.39) in cohorts of manuscripts.ConclusionDissemination of research findings is likely to be a biased process. Publication bias appears to occur early, mainly before the presentation of findings at conferences or submission of manuscripts to journals.
| S-EPMC2789098 | biostudies-literature
Project description:Randomized clinical trials (RCTs) are usually the preferred strategy with which to generate evidence of comparative effectiveness, but conducting an RCT is not always feasible. Though observational studies and RCTs often provide comparable estimates, the questioning of observational analyses has recently intensified because of randomized-observational discrepancies regarding the effect of postmenopausal hormone replacement therapy on coronary heart disease. Reanalyses of observational data that excluded prevalent users of hormone replacement therapy led to attenuated discrepancies, which begs the question of whether exclusion of prevalent users should be generally recommended. In the current study, the authors evaluated the effect of excluding prevalent users of statins in a meta-analysis of observational studies of persons with cardiovascular disease. The pooled, multivariate-adjusted mortality hazard ratio for statin use was 0.77 (95% confidence interval (CI): 0.65, 0.91) in 4 studies that compared incident users with nonusers, 0.70 (95% CI: 0.64, 0.78) in 13 studies that compared a combination of prevalent and incident users with nonusers, and 0.54 (95% CI: 0.45, 0.66) in 13 studies that compared prevalent users with nonusers. The corresponding hazard ratio from 18 RCTs was 0.84 (95% CI: 0.77, 0.91). It appears that the greater the proportion of prevalent statin users in observational studies, the larger the discrepancy between observational and randomized estimates.
| S-EPMC3271813 | biostudies-other
Project description:We argue that algorithmic modeling is a powerful approach to understanding the collective dynamics of human behavior. We consider the task of pairing up individuals connected over a network, according to the following model: each individual is able to propose to match with and accept a proposal from a neighbor in the network; if a matched individual proposes to another neighbor or accepts another proposal, the current match will be broken; individuals can only observe whether their neighbors are currently matched but have no knowledge of the network topology or the status of other individuals; and all individuals have the common goal of maximizing the total number of matches. By examining the experimental data, we identify a behavioral principle called prudence, develop an algorithmic model, analyze its properties mathematically and by simulations, and validate the model with human subject experiments for various network sizes and topologies. Our results include i) a 1/2-approximate maximum matching is obtained in logarithmic time in the network size for bounded degree networks; ii) for any constant ? > 0, a (1 - ?)-approximate maximum matching is obtained in polynomial time, while obtaining a maximum matching can require an exponential time; and iii) convergence to a maximum matching is slower on preferential attachment networks than on small-world networks. These results allow us to predict that while humans can find a "good quality" matching quickly, they may be unable to find a maximum matching in feasible time. We show that the human subjects largely abide by prudence, and their collective behavior is closely tracked by the above predictions.
| S-EPMC3425504 | biostudies-literature
Project description:Publication bias is a serious problem in systematic reviews and meta-analyses, which can affect the validity and generalization of conclusions. Currently, approaches to dealing with publication bias can be distinguished into two classes: selection models and funnel-plot-based methods. Selection models use weight functions to adjust the overall effect size estimate and are usually employed as sensitivity analyses to assess the potential impact of publication bias. Funnel-plot-based methods include visual examination of a funnel plot, regression and rank tests, and the nonparametric trim and fill method. Although these approaches have been widely used in applications, measures for quantifying publication bias are seldom studied in the literature. Such measures can be used as a characteristic of a meta-analysis; also, they permit comparisons of publication biases between different meta-analyses. Egger's regression intercept may be considered as a candidate measure, but it lacks an intuitive interpretation. This article introduces a new measure, the skewness of the standardized deviates, to quantify publication bias. This measure describes the asymmetry of the collected studies' distribution. In addition, a new test for publication bias is derived based on the skewness. Large sample properties of the new measure are studied, and its performance is illustrated using simulations and three case studies.
| S-EPMC5953768 | biostudies-literature