Dataset Information

Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence Interval-Based Stopping Rules.

ABSTRACT: A well-known stopping rule in adaptive mastery testing is to terminate the assessment once the examinee's ability confidence interval lies entirely above or below the cut-off score. This article proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the test continue. Computation of this probability utilizes normality of an asymptotically equivalent version of the maximum likelihood ability estimate. In two simulation sets, the new procedures showed a substantial reduction in average test length while maintaining similar classification accuracy to the original method.

SUBMITTER: Sie H

PROVIDER: S-EPMC5978543 | biostudies-literature | 2015 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence Interval-Based Stopping Rules.

Sie Haskell H Finkelman Matthew D MD Bartroff Jay J Thompson Nathan A NA

Applied psychological measurement 20141216 4

A well-known stopping rule in adaptive mastery testing is to terminate the assessment once the examinee's ability confidence interval lies entirely above or below the cut-off score. This article proposes new procedures that seek to improve such a variable-length stopping rule by coupling it with curtailment and stochastic curtailment. Under the new procedures, test termination can occur earlier if the probability is high enough that the current classification decision remains the same should the ...[more]

PMID: 29881008

Similar Datasets

Project description:BackgroundAn adequate sample size calculation is essential for designing a successful clinical trial. One way to tackle planning difficulties regarding parameter assumptions required for sample size calculation is to adapt the sample size during the ongoing trial.This can be attained by adaptive group sequential study designs. At a predefined timepoint, the interim effect is tested for significance. Based on the interim test result, the trial is either stopped or continued with the possibility of a sample size recalculation.ObjectivesSample size recalculation rules have different limitations in application like a high variability of the recalculated sample size. Hence, the goal is to provide a tool to counteract this performance limitation.MethodsSample size recalculation rules can be interpreted as functions of the observed interim effect. Often, a "jump" from the first stage's sample size to the maximal sample size at a rather arbitrarily chosen interim effect size is implemented and the curve decreases monotonically afterwards. This jump is one reason for a high variability of the sample size. In this work, we investigate how the shape of the recalculation function can be improved by implementing a smoother increase of the sample size. The design options are evaluated by means of Monte Carlo simulations. Evaluation criteria are univariate performance measures such as the conditional power and sample size as well as a conditional performance score which combines these components.ResultsWe demonstrate that smoothing corrections can reduce variability in conditional power and sample size as well as they increase the performance with respect to a recently published conditional performance score for medium and large standardized effect sizes.ConclusionBased on the simulation study, we present a tool that is easily implemented to improve sample size recalculation rules. The approach can be combined with existing sample size recalculation rules described in the literature.

Dataset Information

Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence Interval-Based Stopping Rules.

Publications

Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence Interval-Based Stopping Rules.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets