Dataset Information

Modelling dataset bias in machine-learned theories of economic decision-making.

ABSTRACT: Normative and descriptive models have long vied to explain and predict human risky choices, such as those between goods or gambles. A recent study reported the discovery of a new, more accurate model of human decision-making by training neural networks on a new online large-scale dataset, choices13k. Here we systematically analyse the relationships between several models and datasets using machine-learning methods and find evidence for dataset bias. Because participants' choices in stochastically dominated gambles were consistently skewed towards equipreference in the choices13k dataset, we hypothesized that this reflected increased decision noise. Indeed, a probabilistic generative model adding structured decision noise to a neural network trained on data from a laboratory study transferred best, that is, outperformed all models apart from those trained on choices13k. We conclude that a careful combination of theory and data analysis is still required to understand the complex interactions of machine-learning models and data of human risky choices.

SUBMITTER: Thomas T

PROVIDER: S-EPMC11045447 | biostudies-literature | 2024 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Modelling dataset bias in machine-learned theories of economic decision-making.

Thomas Tobias T Straub Dominik D Tatai Fabian F Shene Megan M Tosik Tümer T Kersting Kristian K Rothkopf Constantin A CA

Nature human behaviour 20240112 4

Normative and descriptive models have long vied to explain and predict human risky choices, such as those between goods or gambles. A recent study reported the discovery of a new, more accurate model of human decision-making by training neural networks on a new online large-scale dataset, choices13k. Here we systematically analyse the relationships between several models and datasets using machine-learning methods and find evidence for dataset bias. Because participants' choices in stochasticall ...[more]

PMID: 38216691

Similar Datasets

Project description:IntroductionThe COVID-19 pandemic had devastating health and socioeconomic effects, partly due to policy decisions to mitigate them. Little evidence exists of approaches that guided decisions in settings with limited pre-pandemic modelling capacity. We thus sought to identify knowledge translation mechanisms, enabling factors and structures needed to effectively translate modelled evidence into policy decisions.MethodsWe used convergent mixed methods in a participatory action approach, with quantitative data from a survey and qualitative data from a scoping review, in-depth interviews and workshop notes. Participants included researchers and policy actors involved in COVID-19 evidence generation and decision-making. They were mostly from lower- and middle-income countries (LMICs) in Africa, Southeast Asia and Latin America. Quantitative and qualitative data integration occurred during data analysis through triangulation and during reporting in a narrative synthesis.ResultsWe engaged 147 researchers and 57 policy actors from 28 countries. We found that the strategies required to use modelled evidence effectively include capacity building of modelling expertise and communication, improved data infrastructure, sustained funding and dedicated knowledge translation platforms. The common knowledge translation mechanisms used during the pandemic included policy briefs, face-to-face debriefings and dashboards. Some enabling factors for knowledge translation comprised solid relationships and open communication between researchers and policymakers, credibility of researchers, co-production of policy questions and embedding researchers in policymaking spaces. Barriers included competition among modellers, negative attitude of policymakers towards research, political influences and demand for quick outputs.ConclusionWe provide a contextualised understanding of knowledge translation for LMICs during the COVID-19 pandemic. Furthermore, we share key lessons on how knowledge translation from mathematical modelling complements the broader learning agenda related to pandemic preparedness and long-term investments in evidence-to-policy translation. Our findings led to the co-development of a knowledge translation framework useful in various settings to guide decision-making, especially for public health emergencies.

Dataset Information

Modelling dataset bias in machine-learned theories of economic decision-making.

Publications

Modelling dataset bias in machine-learned theories of economic decision-making.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets