Dataset Information

The quest for an optimal alpha.

ABSTRACT: Researchers who analyze data within the framework of null hypothesis significance testing must choose a critical "alpha" level, ?, to use as a cutoff for deciding whether a given set of data demonstrates the presence of a particular effect. In most fields, ? = 0.05 has traditionally been used as the standard cutoff. Many researchers have recently argued for a change to a more stringent evidence cutoff such as ? = 0.01, 0.005, or 0.001, noting that this change would tend to reduce the rate of false positives, which are of growing concern in many research areas. Other researchers oppose this proposed change, however, because it would correspondingly tend to increase the rate of false negatives. We show how a simple statistical model can be used to explore the quantitative tradeoff between reducing false positives and increasing false negatives. In particular, the model shows how the optimal ? level depends on numerous characteristics of the research area, and it reveals that although ? = 0.05 would indeed be approximately the optimal value in some realistic situations, the optimal ? could actually be substantially larger or smaller in other situations. The importance of the model lies in making it clear what characteristics of the research area have to be specified to make a principled argument for using one ? level rather than another, and the model thereby provides a blueprint for researchers seeking to justify a particular ? level.

SUBMITTER: Miller J

PROVIDER: S-EPMC6314595 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The quest for an optimal alpha.

Miller Jeff J Ulrich Rolf R

PloS one 20190102 1

Researchers who analyze data within the framework of null hypothesis significance testing must choose a critical "alpha" level, α, to use as a cutoff for deciding whether a given set of data demonstrates the presence of a particular effect. In most fields, α = 0.05 has traditionally been used as the standard cutoff. Many researchers have recently argued for a change to a more stringent evidence cutoff such as α = 0.01, 0.005, or 0.001, noting that this change would tend to reduce the rate of fal ...[more]

PMID: 30601826

Similar Datasets

Project description:BackgroundThe Clarification of Optimal Anticoagulation through Genetics (COAG) trial is a large, multicenter, double-blinded, randomized trial to determine whether use of a genotype-guided dosing algorithm (using clinical and genetic information) to initiate warfarin treatment will improve anticoagulation status when compared to a dosing algorithm using only clinical information.PurposeThis article describes prospective alpha allocation and balanced alpha allocation for the design of the COAG trial.MethodsThe trial involves two possibly heterogeneous populations, which can be distinguished by the difference in warfarin dose as predicted by the two algorithms. A statistical approach is detailed, which allows an overall comparison as well as a comparison of the primary endpoint in the subgroup for which sufficiently different doses are predicted by the two algorithms. Methods of allocating alpha for these analyses are given - a prospective alpha allocation and allocating alpha so that the two analyses have equal power, which we call a 'balanced alpha allocation.'ResultsWe show how to include an analysis of the primary endpoint in a subgroup as a co-primary analysis. Power can be improved by incorporating the correlation between the overall and subgroup analyses in a prospective alpha allocation approach. Balanced alpha allocation for the full cohort and subgroup tests to achieve the same desired power for both of the primary analyses is discussed in detail.LimitationsIn the COAG trial, it is impractical to stratify the randomization on subgroup membership because genetic information may not be available at the time of randomization. If imbalances in the treatment arms in the subgroup are found, they will need to be addressed.ConclusionsThe design of the COAG trial assures that the subgroup in which the largest treatment difference is expected is elevated to a co-primary analysis. Incorporating the correlation between the full cohort and the subgroup analyses provides an improvement in power for the subgroup comparison, and further improvement may be achieved via a balanced alpha allocation approach when the parameters involved in the sample size calculation are reasonably well estimated.

Dataset Information

The quest for an optimal alpha.

Publications

The quest for an optimal alpha.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets