Dataset Information

Blinded sample size re-estimation in a comparative diagnostic accuracy study.

ABSTRACT:

Background

The sample size calculation in a confirmatory diagnostic accuracy study is performed for co-primary endpoints because sensitivity and specificity are considered simultaneously. The initial sample size calculation in an unpaired and paired diagnostic study is based on assumptions about, among others, the prevalence of the disease and, in the paired design, the proportion of discordant test results between the experimental and the comparator test. The choice of the power for the individual endpoints impacts the sample size and overall power. Uncertain assumptions about the nuisance parameters can additionally affect the sample size.

Methods

We develop an optimal sample size calculation considering co-primary endpoints to avoid an overpowered study in the unpaired and paired design. To adjust assumptions about the nuisance parameters during the study period, we introduce a blinded adaptive design for sample size re-estimation for the unpaired and the paired study design. A simulation study compares the adaptive design to the fixed design. For the paired design, the new approach is compared to an existing approach using an example study.

Results

Due to blinding, the adaptive design does not inflate type I error rates. The adaptive design reaches the target power and re-estimates nuisance parameters without any relevant bias. Compared to the existing approach, the proposed methods lead to a smaller sample size.

Conclusions

We recommend the application of the optimal sample size calculation and a blinded adaptive design in a confirmatory diagnostic accuracy study. They compensate inefficiencies of the sample size calculation and support to reach the study aim.

SUBMITTER: Stark M

PROVIDER: S-EPMC9019976 | biostudies-literature | 2022 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Blinded sample size re-estimation in a comparative diagnostic accuracy study.

Stark Maria M Hesse Mailin M Brannath Werner W Zapf Antonia A

BMC medical research methodology 20220419 1

<h4>Background</h4>The sample size calculation in a confirmatory diagnostic accuracy study is performed for co-primary endpoints because sensitivity and specificity are considered simultaneously. The initial sample size calculation in an unpaired and paired diagnostic study is based on assumptions about, among others, the prevalence of the disease and, in the paired design, the proportion of discordant test results between the experimental and the comparator test. The choice of the power for the ...[more]

PMID: 35439947

Similar Datasets

Project description:BackgroundCrossover designs are commonly utilised in randomised controlled trials investigating treatments for long-term chronic illnesses. One problem with this design is its inherent repeated measures necessitate the availability of an estimate of the within-person standard deviation (SD) to perform a sample size calculation, which may be rarely available at the design stage of a trial. Interim sample size re-estimation designs can be used to help alleviate this issue by adapting the sample size mid-way through the trial, using accrued information in a statistically robust way.MethodsThe AIM HY-INFORM study is part of the Informative Markers in Hypertension (AIM HY) Programme and comprises two crossover trials, each with a planned recruitment of 600 participants. The objective of the study is to test whether blood pressure response to first line antihypertensive treatment depends on ethnicity. An interim analysis is planned to reassess the assumptions of the planned sample size for the study. The aims of this paper are: (1) to provide a formula for sample size re-estimation in both crossover trials; and (2) to present a simulation study of the planned interim analysis to investigate alternative within-person SDs to that assumed.ResultsThe AIM HY-INFORM protocol sample size calculation fixes the within-person SD to be 8 mmHg, giving > 90% power for a primary treatment effect of 4 mmHg. Using the method developed here and simulating the interim sample size reassessment, if we were to see a larger within-person SD of 9 mmHg at interim, 640 participants for 90% power 90% of the time in the three-period three-treatment design would be required. Similarly, in the four-period four-treatment crossover design, 602 participants would be required.ConclusionsThe formulas presented here provide a method for re-estimating the sample size in crossover trials. In the context of the AIM HY-INFORM study, simulating the interim analysis allows us to explore the results of a possible increase in the within-person SD from that assumed. Simulations show that without increasing the planned sample size of 600 participants, we can reasonably still expect to achieve 80% power with a small increase in the within-person SD from that assumed.Trial registrationClinicalTrials.gov, NCT02847338. Registered on 28 July 2016.

Project description:Background/aimsThe use of adaptive designs has been increasing in randomized clinical trials. Sample size re-estimation is a type of adaptation in which nuisance parameters are estimated at an interim point in the trial and the sample size re-computed based on these estimates. The Secondary Prevention of Small Subcortical Strokes study was a randomized clinical trial assessing the impact of single- versus dual-antiplatelet therapy and control of systolic blood pressure to a higher (130-149 mmHg) versus lower (<130 mmHg) target on recurrent stroke risk in a two-by-two factorial design. A sample size re-estimation was performed during the Secondary Prevention of Small Subcortical Strokes study resulting in an increase from the planned sample size of 2500-3020, and we sought to determine the impact of the sample size re-estimation on the study results.MethodsWe assessed the results of the primary efficacy and safety analyses with the full 3020 patients and compared them to the results that would have been observed had randomization ended with 2500 patients. The primary efficacy outcome considered was recurrent stroke, and the primary safety outcomes were major bleeds and death. We computed incidence rates for the efficacy and safety outcomes and used Cox proportional hazards models to examine the hazard ratios for each of the two treatment interventions (i.e. the antiplatelet and blood pressure interventions).ResultsIn the antiplatelet intervention, the hazard ratio was not materially modified by increasing the sample size, nor did the conclusions regarding the efficacy of mono versus dual-therapy change: there was no difference in the effect of dual- versus monotherapy on the risk of recurrent stroke hazard ratios (n = 3020 HR (95% confidence interval): 0.92 (0.72, 1.2), p = 0.48; n = 2500 HR (95% confidence interval): 1.0 (0.78, 1.3), p = 0.85). With respect to the blood pressure intervention, increasing the sample size resulted in less certainty in the results, as the hazard ratio for higher versus lower systolic blood pressure target approached, but did not achieve, statistical significance with the larger sample (n = 3020 HR (95% confidence interval): 0.81 (0.63, 1.0), p = 0.089; n = 2500 HR (95% confidence interval): 0.89 (0.68, 1.17), p = 0.40). The results from the safety analyses were similar to 3020 and 2500 patients for both study interventions. Other trial-related factors, such as contracts, finances, and study management, were impacted as well.ConclusionAdaptive designs can have benefits in randomized clinical trials, but do not always result in significant findings. The impact of adaptive designs should be measured in terms of both trial results, as well as practical issues related to trial management. More post hoc analyses of study adaptations will lead to better understanding of the balance between the benefits and the costs.

Project description:BackgroundWhile the clinical trials and statistical methodology literature on sample size re-estimation (SSRE) is robust, evaluation of SSRE procedures following the completion of a clinical trial has been sparsely reported. In blinded sample size re-estimation, only nuisance parameters are re-estimated, and the blinding of the current trial treatment effect is preserved. Blinded re-estimation procedures are well-accepted by regulatory agencies and funders. We review our experience of sample size re-estimation in a large international, National Institutes of Health funded clinical trial for adjuvant breast cancer treatment, and evaluate our blinded sample size re-estimation procedure for this time-to-event trial. We evaluated the SSRE procedure by examining assumptions made during the re-estimation process, estimates resulting from re-estimation, and the impact on final trial results with and without the addition of participants, following sample size re-estimation.MethodsWe compared the control group failure probabilities estimated at the time of SSRE to estimates used in the original planning, to the final un-blinded control group failure probability estimates for those included in the SSRE procedure (SSRE cohort), and to the final total control group failure probability estimates. The impact of re-estimation on the final comparison between randomized treatment groups is evaluated for those in the originally planned cohort (n = 340) and for the combination of those recruited in the originally planned cohort and those added after re-estimation (n = 509).ResultsVery little difference is observed between the originally planned cohort and all randomized patients in the control group failure probabilities over time or in the overall hazard ratio estimating treatment effect (originally planned cohort HR 1.25 (0.86, 1.79); all randomized cohort HR 1.24 95% CI (0.91, 1.68)). At the time of blinded SSRE, the estimated control group failure probabilities at 3 years (0.24) and 5 years (0.40) were similar to those for the SSRE cohort once un-blinded (3 years, 0.22 (0.16, 0.30); 5 years, 0.33 (0.26, 0.41)).ConclusionsWe found that our re-estimation procedure performed reasonably well in estimating the control group failure probabilities at the time of re-estimation. Particularly for time-to-event outcomes, pre-planned blinded SSRE procedures may be the best option to aid in maintaining power.Trial registrationClinicalTrials.gov, NCT00201851 . Registered on 9 September 2005. Retrospectively registered.

Project description:BackgroundAn estimated 87% of torture survivors experience chronic pain such as brachial plexopathy from upper extremity suspension or lumbosacral plexus injury from leg hyperextension. However, a vast majority of pain is undetected by evaluators due to a lack of diagnostic tools and confounding psychiatric illness. This diagnostic gap results in exclusive psychological treatment rather than multimodal therapies, substantially limiting rehabilitation. We hypothesized that the United Nations Istanbul Protocol (UNIP) would have a sensitivity of approximately 15% for pain detection, and that the use of a validated pain screen would improve its sensitivity by at least 29%, as compared to the reference standard (pain specialist evaluation).Methods and findingsThis prospective blind-comparison-to-gold-standard study of survivors of torture, as defined by the World Medical Association, took place at Weill Cornell Medicine between February 1, 2017, and June 21, 2019. 11 women and 9 men, for a total of 20 participants, were included in the analysis. Five participants received 2 UNIP evaluations, for a total of 25 unique evaluations included in the analysis. Participants were representative of a global population, with home countries in Africa, Central America, South Asia, the Caribbean, and the Middle East. Methods of torture experienced were homogeneous, following the predictable pattern of systematic torture. Participants first received the standard evaluation protocol for torture survivors (UNIP) by a trained evaluator, and subsequently received a validated pain screen (Brief Pain Inventory-Short Form [BPISF]) followed by a noninvasive examination by a pain specialist physician (reference standard). The primary outcome was the diagnostic and treatment capability of the standard protocol (index test) versus the validated pain screen (BPISF), as compared to the reference standard. Trained evaluators performing the initial assessment with the UNIP (index test) were blinded to the study, and the pain specialist physician (reference standard) was blinded to the outcome of the initial UNIP evaluation and the BPISF; data from the initial UNIP assessment were not gathered by the principal investigator until all other study procedures were completed. Providers using only the UNIP captured pain in a maximum of 16% of evaluations, as compared to 85% of participants being diagnosed with pain by the reference standard. When employed, the validated pain screen had a sensitivity of 100% (95% CI 72%-100%) and a negative predictive value of 100%, as compared to a sensitivity of 24% (95% CI 8%-50%) and a negative predictive value of 19% (95% CI 5%-46%) for the index test. The difference in the sensitivity of the UNIP as compared to the BPISF was significant, with p < 0.001. No adverse events owing to participation in the study were reported by participants. Limitations of the study include small sample size, its single-site nature, and the exclusion of individuals who did not speak 1 of the 5 study languages.ConclusionsThese data indicate that a validated pain screen can supplement the current global standard assessment of torture survivors, the UNIP, to increase the accuracy of pain diagnosis.Trial registrationClinicalTrials.gov NCT03018782.

Dataset Information

Blinded sample size re-estimation in a comparative diagnostic accuracy study.

Background

Methods

Results

Conclusions

Publications

Blinded sample size re-estimation in a comparative diagnostic accuracy study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets