Dataset Information

A simple model suggesting economically rational sample-size choice drives irreproducibility.

ABSTRACT: Several systematic studies have suggested that a large fraction of published research is not reproducible. One probable reason for low reproducibility is insufficient sample size, resulting in low power and low positive predictive value. It has been suggested that insufficient sample-size choice is driven by a combination of scientific competition and 'positive publication bias'. Here we formalize this intuition in a simple model, in which scientists choose economically rational sample sizes, balancing the cost of experimentation with income from publication. Specifically, assuming that a scientist's income derives only from 'positive' findings (positive publication bias) and that individual samples cost a fixed amount, allows to leverage basic statistical formulas into an economic optimality prediction. We find that if effects have i) low base probability, ii) small effect size or iii) low grant income per publication, then the rational (economically optimal) sample size is small. Furthermore, for plausible distributions of these parameters we find a robust emergence of a bimodal distribution of obtained statistical power and low overall reproducibility rates, both matching empirical findings. Finally, we explore conditional equivalence testing as a means to align economic incentives with adequate sample sizes. Overall, the model describes a simple mechanism explaining both the prevalence and the persistence of small sample sizes, and is well suited for empirical validation. It proposes economic rationality, or economic pressures, as a principal driver of irreproducibility and suggests strategies to change this.

SUBMITTER: Braganza O

PROVIDER: S-EPMC7065751 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A simple model suggesting economically rational sample-size choice drives irreproducibility.

Braganza Oliver O

PloS one 20200311 3

Several systematic studies have suggested that a large fraction of published research is not reproducible. One probable reason for low reproducibility is insufficient sample size, resulting in low power and low positive predictive value. It has been suggested that insufficient sample-size choice is driven by a combination of scientific competition and 'positive publication bias'. Here we formalize this intuition in a simple model, in which scientists choose economically rational sample sizes, ba ...[more]

PMID: 32160229

Similar Datasets

Project description:BackgroundIn spite of the potential impact upon population health and expenditure, interventions promoting medication adherence have been found to be of moderate effectiveness and cost effectiveness. Understanding the relative influence of factors affecting patient medication adherence decisions and the characteristics of individuals associated with variation in adherence will lead to a better understanding of how future interventions should be designed and targeted. This study aims to explore medication-taking decisions that may underpin intentional medication non-adherence behaviour amongst a community sample and the relative importance of medication specific factors and patient background characteristics contributing to those decisions.MethodsA discrete choice experiment conducted through a web-enabled online survey was used to estimate the relative importance of eight medication factors (immediate and long-term medication harms and benefits, cost, regimen, symptom severity, alcohol restrictions) on the preference to continue taking a medication. To reflect more closely what usually occurs in practice, non-disease specific medication and health terms were used to mimic decisions across multiple medications and conditions.161 general community participants, matching the national Australian census data (age, gender) were recruited through an online panel provider (participation rate: 10%) in 2010.ResultsSix of the eight factors (i.e. immediate and long-term medication harms and benefits, cost, and regimen) had a significant influence on medication choice. Patient background characteristics did not improve the model. Respondents with private health insurance appeared less sensitive to cost then those without private health insurance. In general, health outcomes, framed as a side-effect, were found to have a greater influence over adherence than outcomes framed as therapeutic benefits.ConclusionsMedication-taking decisions are the subject of rational choices, influenced by the attributes of treatments and potentially amenable to intervention through education, strategic pricing and the altering of dosing characteristics. Understanding individual treatment preferences is thus an important step to improving adherence support provision in practice. Re-framing future interventions and policies to support rational and informed individual patient choices, is the way forward to realising the full potential health and economic benefits from the efficacious use of medications.

Project description:Genome-wide association studies are revolutionizing the search for the genes underlying human complex diseases. The main decisions to be made at the design stage of these studies are the choice of the commercial genotyping chip to be used and the numbers of case and control samples to be genotyped. The most common method of comparing different chips is using a measure of coverage, but this fails to properly account for the effects of sample size, the genetic model of the disease, and linkage disequilibrium between SNPs. In this paper, we argue that the statistical power to detect a causative variant should be the major criterion in study design. Because of the complicated pattern of linkage disequilibrium (LD) in the human genome, power cannot be calculated analytically and must instead be assessed by simulation. We describe in detail a method of simulating case-control samples at a set of linked SNPs that replicates the patterns of LD in human populations, and we used it to assess power for a comprehensive set of available genotyping chips. Our results allow us to compare the performance of the chips to detect variants with different effect sizes and allele frequencies, look at how power changes with sample size in different populations or when using multi-marker tags and genotype imputation approaches, and how performance compares to a hypothetical chip that contains every SNP in HapMap. A main conclusion of this study is that marked differences in genome coverage may not translate into appreciable differences in power and that, when taking budgetary considerations into account, the most powerful design may not always correspond to the chip with the highest coverage. We also show that genotype imputation can be used to boost the power of many chips up to the level obtained from a hypothetical "complete" chip containing all the SNPs in HapMap. Our results have been encapsulated into an R software package that allows users to design future association studies and our methods provide a framework with which new chip sets can be evaluated.

Dataset Information

A simple model suggesting economically rational sample-size choice drives irreproducibility.

Publications

A simple model suggesting economically rational sample-size choice drives irreproducibility.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets