Optimal designs for frequentist model averaging.
Ontology highlight
ABSTRACT:
SUBMITTER: Alhorn K
PROVIDER: S-EPMC6690170 | biostudies-literature | 2019 Sep
REPOSITORIES: biostudies-literature
Publications
Optimal designs for frequentist model averaging.
Alhorn K K Schorning K K Dette H H
Biometrika 20190713 3
We consider the problem of designing experiments for estimating a target parameter in regression analysis when there is uncertainty about the parametric form of the regression function. A new optimality criterion is proposed that chooses the experimental design to minimize the asymptotic mean squared error of the frequentist model averaging estimate. Necessary conditions for the optimal solution of a locally and Bayesian optimal design problem are established. The results are illustrated in seve ...[more]
Similar Datasets
Project description:Postulating that the brain performs approximate Bayesian inference generates principled and empirically testable models of neuronal function-the subject of much current interest in neuroscience and related disciplines. Current formulations address inference and learning under some assumed and particular model. In reality, organisms are often faced with an additional challenge-that of determining which model or models of their environment are the best for guiding behavior. Bayesian model averaging-which says that an agent should weight the predictions of different models according to their evidence-provides a principled way to solve this problem. Importantly, because model evidence is determined by both the accuracy and complexity of the model, optimal inference requires that these be traded off against one another. This means an agent's behavior should show an equivalent balance. We hypothesize that Bayesian model averaging plays an important role in cognition, given that it is both optimal and realizable within a plausible neuronal architecture. We outline model averaging and how it might be implemented, and then explore a number of implications for brain and behavior. In particular, we propose that model averaging can explain a number of apparently suboptimal phenomena within the framework of approximate (bounded) Bayesian inference, focusing particularly upon the relationship between goal-directed and habitual behavior.
| S-EPMC4071291 | biostudies-literature
Project description:Our increased understanding of the mechanistic heterogeneity of diseases has pushed the development of targeted therapeutics. We do not expect all patients with a given disease to benefit from a targeted drug; only those in the target population. That is, those with sufficient dysregulation in the biomolecular pathway targeted by treatment. However, due to complexity of the pathway, and/or technical issues with our characterizing assay, it is often hard to characterize the target population until well into large-scale clinical trials. This has stimulated the development of adaptive enrichment trials; clinical trials in which the target population is adaptively learned; and enrollment criteria are adaptively updated to reflect this growing understanding. This paper proposes a framework for group-sequential adaptive enrichment trials. Building on the work of Simon & Simon (2013). Adaptive enrichment designs for clinical trials. Biostatistics 14(4), 613-625), it includes a frequentist hypothesis test at the end of the trial. However, it uses Bayesian methods to optimize the decisions required during the trial (regarding how to restrict enrollment) and Bayesian methods to estimate effect size, and characterize the target population at the end of the trial. This joint frequentist/Bayesian design combines the power of Bayesian methods for decision making with the use of a formal hypothesis test at the end of the trial to preserve the studywise probability of a type I error.
| S-EPMC6075009 | biostudies-literature
Project description:Alternative designs can increase the feasibility of pediatric trials when compared to classical parallel designs (PaD). In this work we present a model-based approach based on clinical trial simulations for the comparison of PaD with the alternative sequential, crossover, and randomized withdrawal (RWD) designs. Study designs were evaluated in terms of: type I and II errors, sample size per arm (SS), trial duration (TD), treatment exposures, and parameter estimate precision (EP). The crossover requires the lowest SS and TD, although it implies higher placebo and no treatment exposures. RWD maximizes exposure to active treatment while minimizing that to placebo, but requires the largest SS. SS of sequential designs can sometimes be smaller than the crossover one, although with poorer EP. This pharmacometric framework allows a multiscale comparison of alternative study designs that can be used for design selection in future pediatric trials.
| S-EPMC5131885 | biostudies-other
Project description:Clinical trials with adaptive sample size reassessment based on an unblinded analysis of interim results are perhaps the most popular class of adaptive designs (see Elsäßer et al., 2007). Such trials are typically designed by prespecifying a zone for the interim test statistic, termed the promising zone, along with a decision rule for increasing the sample size within that zone. Mehta and Pocock (2011) provided some examples of promising zone designs and discussed several procedures for controlling their type-1 error. They did not, however, address how to choose the promising zone or the corresponding sample size reassessment rule, and proposed instead that the operating characteristics of alternative promising zone designs could be compared by simulation. Jennison and Turnbull (2015) developed an approach based on maximizing expected utility whereby one could evaluate alternative promising zone designs relative to a gold-standard optimal design. In this paper, we show how, by eliciting a few preferences from the trial sponsor, one can construct promising zone designs that are both intuitive and achieve the Jennison and Turnbull (2015) gold-standard for optimality.
| S-EPMC6767001 | biostudies-literature
Project description:The two-phase design is a cost-effective sampling strategy to evaluate the effects of covariates on an outcome when certain covariates are too expensive to be measured on all study subjects. Under such a design, the outcome and inexpensive covariates are measured on all subjects in the first phase and the first-phase information is used to select subjects for measurements of expensive covariates in the second phase. Previous research on two-phase studies has focused largely on the inference procedures rather than the design aspects. We investigate the design efficiency of the two-phase study, as measured by the semiparametric efficiency bound for estimating the regression coefficients of expensive covariates. We consider general two-phase studies, where the outcome variable can be continuous, discrete, or censored, and the second-phase sampling can depend on the first-phase data in any manner. We develop optimal or approximately optimal two-phase designs, which can be substantially more efficient than the existing designs. We demonstrate the improvements of the new designs over the existing ones through extensive simulation studies and two large medical studies.
| S-EPMC7954143 | biostudies-literature
Project description:We consider design issues for cluster randomized trials (CRTs) with a binary outcome where both unit costs and intraclass correlation coefficients (ICCs) in the two arms may be unequal. We first propose a design that maximizes cost efficiency (CE), defined as the ratio of the precision of the efficacy measure to the study cost. Because such designs can be highly sensitive to the unknown ICCs and the anticipated success rates in the two arms, a local strategy based on a single set of best guesses for the ICCs and success rates can be risky. To mitigate this issue, we propose a maximin optimal design that permits ranges of values to be specified for the success rate and the ICC in each arm. We derive maximin optimal designs for three common measures of the efficacy of the intervention, risk difference, relative risk and odds ratio, and study their properties. Using a real cancer control and prevention trial example, we ascertain the efficiency of the widely used balanced design relative to the maximin optimal design and show that the former can be quite inefficient and less robust to mis-specifications of the ICCs and the success rates in the two arms.
| S-EPMC5550375 | biostudies-literature
Project description:In the early phase development of molecularly targeted agents (MTAs), a commonly encountered situation is that the MTA is expected to be more effective for a certain biomarker subgroup, say marker-positive patients, but there is no adequate evidence to show that the MTA does not work for the other subgroup, that is, marker-negative patients. After establishing that marker-positive patients benefit from the treatment, it is often of great clinical interest to determine whether the treatment benefit extends to marker-negative patients. The authors propose optimal sequential enrichment (OSE) designs to address this practical issue in the context of phase II clinical trials. The OSE designs evaluate the treatment effect first in marker-positive patients and then in marker-negative patients if needed. The designs are optimal in the sense that they minimize the expected sample size or the maximum sample size under the null hypothesis that the MTA is futile. An efficient, accurate optimization algorithm is proposed to find the optimal design parameters. One important advantage of the OSE design is that the go/no-go interim decision rules are specified prior to the trial conduct, which makes the design particularly easy to use in practice. A simulation study shows that the OSE designs perform well and are ethically more desirable than the commonly used marker-stratified design. The OSE design is applied to an endometrial carcinoma trial. Copyright © 2016 John Wiley & Sons, Ltd.
| S-EPMC5138118 | biostudies-literature
Project description:Bivalve, ammonite and snail shells are described by a small number of geometrical parameters. Raup noted that the vast majority of theoretically possible shell forms do not occur in nature. The constraint factors that regulate the biased distribution of natural form have long since been an open problem in evolution. The problem of whether natural shell form is a result of optimization remains unsolved despite previous attempts. Here we solve this problem by considering the scaling exponent of shell thickness as a morphological parameter. The scaling exponent has a drastic effect on the optimal design of shell shapes. The observed characteristic shapes of natural shells are explained in a unified manner as a result of optimal utilization of shell material resources, while isometric growth in thickness leads to impossibly tight coiling.
| S-EPMC5301254 | biostudies-literature
Project description:Detecting genetic markers with biologically relevant effects remains a challenge due to multiple testing. Standard analysis methods focus on evidence against the null and protect primarily the type I error. On the other hand, the worthwhile alternative is specified for power calculations at the design stage. The balanced test as proposed by Moerkerke and others (2006) and Moerkerke and Goetghebeur (2006) incorporates this alternative directly in the decision criterion to achieve better power. Genetic markers are selected and ranked in order of the balance of evidence they contain against the null and the target alternative. In this paper, we build on this guiding principle to develop 2-stage designs for screening genetic markers when the cost of measurements is high. For a given marker, a first sample may already provide sufficient evidence for or against the alternative. If not, more data are gathered at the second stage which is then followed by a binary decision based on all available data. By optimizing parameters which determine the decision process over the 2 stages (such as the area of the "gray" zone which leads to the gathering of extra data), the expected cost per marker can be reduced substantially. We also demonstrate that, compared to 1-stage designs, 2-stage designs achieve a better balance between true negatives and positives for the same cost.
| S-EPMC2536725 | biostudies-literature
Project description:In order to have confidence in model-based phylogenetic methods, such as maximum likelihood (ML) and Bayesian analyses, one must use an appropriate model of molecular evolution identified using statistically rigorous criteria. Although model selection methods such as the likelihood ratio test and Akaike information criterion are widely used in the phylogenetic literature, model selection methods lack the ability to reject all models if they provide an inadequate fit to the data. There are two methods, however, that assess absolute model adequacy, the frequentist Goldman-Cox (GC) test and Bayesian posterior predictive simulations (PPSs), which are commonly used in conjunction with the multinomial log likelihood test statistic. In this study, we use empirical and simulated data to evaluate the adequacy of common substitution models using both frequentist and Bayesian methods and compare the results with those obtained with model selection methods. In addition, we investigate the relationship between model adequacy and performance in ML and Bayesian analyses in terms of topology, branch lengths, and bipartition support. We show that tests of model adequacy based on the multinomial likelihood often fail to reject simple substitution models, especially when the models incorporate among-site rate variation (ASRV), and normally fail to reject less complex models than those chosen by model selection methods. In addition, we find that PPSs often fail to reject simpler models than the GC test. Use of the simplest substitution models not rejected based on fit normally results in similar but divergent estimates of tree topology and branch lengths. In addition, use of the simplest adequate substitution models can affect estimates of bipartition support, although these differences are often small with the largest differences confined to poorly supported nodes. We also find that alternative assumptions about ASRV can affect tree topology, tree length, and bipartition support. Our results suggest that using the simplest substitution models not rejected based on fit may be a valid alternative to implementing more complex models identified by model selection methods. However, all common substitution models may fail to recover the correct topology and assign appropriate bipartition support if the true tree shape is difficult to estimate regardless of model adequacy.
| S-EPMC2981515 | biostudies-literature