Dataset Information

Incomplete data analysis of non-inferiority clinical trials: Difference between binomial proportions case.

ABSTRACT:

Background

Incomplete data analysis continues to be a major issue for non-inferiority clinical trials. Due to the steadily increasing use of non-inferiority study design, we believe this topic deserves an immediate attention.

Methods

We evaluated the performance of various strategies, including complete case analysis and various imputations techniques for handling incomplete non-inferiority clinical trials when outcome of interest is difference between binomial proportions. Non-inferiority of a new treatment was determined using a fixed margin approach with 95-95% confidence interval method. The methods used to construct the confidence intervals were compared as well and included: Wald, Farrington-Manning and Newcombe methods.

Results

We found that worst-case and best-case scenario imputation methods should not be used for analysis of incomplete data in non-inferiority trial design, since such methods seriously inflate type-I error rates and produce biased estimates. In addition, we report conditions under which complete case analysis is an acceptable strategy for missing at random missingness mechanism. Importantly, we show how two-stage multiple imputation could be successfully applied for incomplete data that follow missing not at random patterns, and thus result in controlled type-I error rates and unbiased estimates.

Conclusion

This thorough simulation study provides a road map for the analysis of incomplete data in non-inferiority clinical trials for different types of missingness. We believe that the results reported in this paper could serve practitioners who encounter missing data problems in their non-inferiority clinical trials.

SUBMITTER: Sidi Y

PROVIDER: S-EPMC7226649 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Incomplete data analysis of non-inferiority clinical trials: Difference between binomial proportions case.

Sidi Yulia Y Harel Ofer O

Contemporary clinical trials communications 20200504

<h4>Background</h4>Incomplete data analysis continues to be a major issue for non-inferiority clinical trials. Due to the steadily increasing use of non-inferiority study design, we believe this topic deserves an immediate attention.<h4>Methods</h4>We evaluated the performance of various strategies, including complete case analysis and various imputations techniques for handling incomplete non-inferiority clinical trials when outcome of interest is difference between binomial proportions. Non-in ...[more]

PMID: 32426549

Similar Datasets

Project description:Background/aims Non-inferiority trials with time-to-event outcomes are becoming increasingly common. Designing non-inferiority trials is challenging, in particular, they require very large sample sizes. We hypothesized that the difference in restricted mean survival time, an alternative to the hazard ratio, could lead to smaller required sample sizes. Methods We show how to convert a margin for the hazard ratio into a margin for the difference in restricted mean survival time and how to calculate the required sample size under a Weibull survival distribution. We systematically selected non-inferiority trials published between 2013 and 2016 in seven major journals. Based on the protocol and article of each trial, we determined the clinically relevant time horizon of interest. We reconstructed individual patient data for the primary outcome and fit a Weibull distribution to the comparator arm. We converted the margin for the hazard ratio into the margin for the difference in restricted mean survival time. We tested for non-inferiority using the difference in restricted mean survival time and hazard ratio. We determined the required sample size based on both measures, using the type I error risk and power from the original trial design. Results We included 35 trials. We found evidence of non-proportional hazards in five (14%) trials. The hazard ratio and the difference in restricted mean survival time were consistent regarding non-inferiority testing, except in one trial where the difference in restricted mean survival time led to evidence of non-inferiority while the hazard ratio did not. The median hazard ratio margin was 1.43 (Q1-Q3, 1.29-1.75). The median of the corresponding margins for the difference in restricted mean survival time was -21?days (Q1-Q3, -36 to -8) for a median time horizon of 2.0?years (Q1-Q3, 1-3?years). The required sample size according to the difference in restricted mean survival time was smaller in 71% of trials, with a median relative decrease of 8.5% (Q1-Q3, 0.4%-38.0%). Across all 35 trials, about 25,000 participants would have been spared from enrollment using the difference in restricted mean survival time compared to hazard ratio for trial design. Conclusion The margins for the hazard ratio may seem large but translate to relatively small differences in restricted mean survival time. The difference in restricted mean survival time offers meaningful interpretation and can result in considerable reductions in sample size. Restricted mean survival time-based measures should be considered more widely in the design and analysis of non-inferiority trials with time-to-event outcomes.

Project description:BackgroundNumerous statistical methods can be used to calculate the confidence interval (CI) of risk differences. There is consensus in previous literature that the Wald method should be discouraged. We compared five statistical methods for estimating the CI of risk difference in terms of CI width and study conclusion in antibiotic non-inferiority trials.MethodsIn a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, reported risk differences for the primary outcome, and described the number of successes and/or failures as well as patients in each arm. For each study, we re-calculated the risk difference CI using the Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and skewness-corrected asymptotic score (SCAS) methods. The CIs by different statistical methods were compared in terms of CI width and conclusion on non-inferiority. A wider CI was considered to be more conservative.ResultsThe analysis included 224 comparisons from 213 studies. The statistical method used to calculate CI was not reported in 134 (59.8%) cases. The median (interquartile range IQR) for CI width by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods was 13.0% (10.8%, 17.4%), 13.3% (10.9%, 18.5%), 13.6% (11.1%, 18.9%), 13.6% (11.1% and 19.0%), and 13.4% (11.1%, 18.9%), respectively. In 216 comparisons that reported a non-inferiority margin, the conclusion on non-inferiority was the same across the five statistical methods in 211 (97.7%) cases. The differences in CI width were more in trials with a sample size of 100 or less in each group and treatment success rate above 90%. Of the 18 trials in this subgroup with a specified non-inferiority margin, non-inferiority was shown in 17 (94.4%), 16 (88.9%), 14 (77.8%), 14 (77.8%), and 15 (83.3%) cases based on CI by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods, respectively.ConclusionsThe statistical method used to calculate CI was not reported in the majority of antibiotic non-inferiority trials. Different statistical methods for CI resulted in different conclusions on non-inferiority in 2.3% cases. The differences in CI widths were highest in trials with a sample size of 100 or less in each group and a treatment success rate above 90%.Trial registrationPROSPERO CRD42020165040 . April 28, 2020.

Project description:BackgroundNon-inferiority trials are increasingly used to evaluate new treatments that are expected to have secondary advantages over standard of care, but similar efficacy on the primary outcome. When designing a non-inferiority trial with a binary primary outcome, the choice of effect measure for the non-inferiority margin (e.g. risk ratio or risk difference) has an important effect on sample size calculations; furthermore, if the control event risk observed is markedly different from that assumed, the trial can quickly lose power or the results become difficult to interpret.MethodsWe propose a new way of designing non-inferiority trials to overcome the issues raised by unexpected control event risks. Our proposal involves using clinical judgement to specify a 'non-inferiority frontier', i.e. a curve defining the most appropriate non-inferiority margin for each possible value of control event risk. Existing trials implicitly use frontiers defined by a fixed risk ratio or a fixed risk difference. We discuss their limitations and propose a fixed arcsine difference frontier, using the power-stabilising transformation for binary outcomes, which may better represent clinical judgement. We propose and compare three ways of designing a trial using this frontier: testing and reporting on the arcsine scale; testing on the arcsine scale but reporting on the risk difference or risk ratio scale; and modifying the margin on the risk difference or risk ratio scale after observing the control event risk according to the power-stabilising frontier.ResultsTesting and reporting on the arcsine scale leads to results which are challenging to interpret clinically. For small values of control event risk, testing on the arcsine scale and reporting results on the risk difference scale produces confidence intervals at a higher level than the nominal one or non-inferiority margins that are slightly smaller than those back-calculated from the power-stabilising frontier alone. However, working on the arcsine scale generally requires a larger sample size compared to the risk difference scale. Therefore, working on the risk difference scale, modifying the margin after observing the control event risk, might be preferable, as it requires a smaller sample size. However, this approach tends to slightly inflate type I error rate; a solution is to use a slightly lower significance level for testing, although this modestly reduces power. When working on the risk ratio scale instead, the same approach based on the modification of the margin leads to power levels above the nominal one, maintaining type I error under control.ConclusionsOur proposed methods of designing non-inferiority trials using power-stabilising non-inferiority frontiers make trial design more resilient to unexpected values of the control event risk, at the only cost of requiring somewhat larger sample sizes when the goal is to report results on the risk difference scale.

Project description:Background. Postoperative morbidity has immediate and delayed consequences for surgical patients, including excess risk of premature death. Capturing these data objectively and routinely in large electronic databases using tools such as the Postoperative Morbidity Survey (POMS) would offer tremendous clinical and translational potential. However, POMS has thus far only utilised prospective data collection by research staff. We hypothesised that retrospective data collection from routinely collated hospital data from paper and electronic charts, medical and nursing notes was non-inferior to prospective data collection requiring research staff capturing POMS-defined morbidity in real-time. Methods. Morbidity was recorded by a trained investigator as defined by POMS prospectively on postoperative days 3 and 7. Separately, an independent investigator blinded to prospectively acquired data retrospectively assessed the same patients' morbidity as defined by POMS criteria, using medical charts, nursing summaries and electronic data. Equivalence was accepted when the confidence limits for both modes of data collection fell completely inside the equivalence bounds, with the maximum equivalence difference (i.e., the largest value of the difference in sensitivities deemed to reach a conclusion of equivalence) set a priori at 0.2. Differences for confidence limits between retrospective and prospective data collection were based on Nam's RMLE method. The relationship between morbidity on postoperative day 3 as recorded by each data collection method on time to become morbidity free and length of hospital stay was compared using the log-rank test. Results. POMS data from 85 patients undergoing elective or emergency surgery were analyzed. At postoperative day 3, POMS-defined morbidity was similar regardless of whether data were collected prospectively or retrospectively (95% CI [-0.13-0.013]; p < 0.001). Non-inferiority for sensitivity was observed for all other POMS domains and timepoints. Time to become morbidity free Kaplan-Meier plots were indistinguishable between POMS obtained prospectively or retrospectively (hazard ratio: 1.09 (95% CI [0.76-1.57]); p = 0.33, log rank test). Similarly, the mode of data collection did not alter the association between early postoperative morbidity on postoperative day 3 and delayed hospital discharge. Conclusions. Postoperative morbidity as defined by the Post Operative Morbidity Survey can be assessed retrospectively. These data may therefore be easily captured using electronic patient record systems, thereby expanding the potential for bioinformatics approaches to generate new clinical and translational insights into recovery from surgery.

Project description:BackgroundIn a non-inferiority (NI) trial, analysis based on the intention-to-treat (ITT) principle is anti-conservative, so current guidelines recommend analysing on a per-protocol (PP) population in addition. However, PP analysis relies on the often implausible assumption of no confounders. Randomisation-based efficacy estimators (RBEEs) allow for treatment non-adherence while maintaining a comparison of randomised groups. Fischer et al. have developed an approach for estimating RBEEs in randomised trials with two active treatments, a common feature of NI trials. The aim of this paper was to demonstrate the use of RBEEs in NI trials using this approach, and to appraise the feasibility of these estimators as the primary analysis in NI trials.MethodsTwo NI trials were used. One comparing two different dosing regimens for the maintenance of remission in people with ulcerative colitis (CODA), and the other comparing an orally administered treatment to an intravenously administered treatment in preventing skeletal-related events in patients with bone metastases from breast cancer (ZICE). Variables that predicted adherence in each of the trial arms, and were also independent of outcome, were sought in each of the studies. Structural mean models (SMMs) were fitted that conditioned on these variables, and the point estimates and confidence intervals compared to that found in the corresponding ITT and PP analyses.ResultsIn the CODA study, no variables were found that differentially predicted treatment adherence while remaining independent of outcome. The SMM, using standard methodology, moved the point estimate closer to 0 (no difference between arms) compared to the ITT and PP analyses, but the confidence interval was still within the NI margin, indicating that the conclusions drawn would remain the same. In the ZICE study, cognitive functioning as measured by the corresponding domain of the QLQ-C30, and use of chemotherapy at baseline were both differentially associated with adherence while remaining independent of outcome. However, while the SMM again moved the point estimate closer to 0, the confidence interval was wide, overlapping with any NI margin that could be justified.ConclusionDeriving RBEEs in NI trials with two active treatments can provide a randomisation-respecting estimate of treatment efficacy that accounts for treatment adherence, is straightforward to implement, but requires thorough planning during the design stage of the study to ensure that strong baseline predictors of treatment are captured. Extension of the approach to handle nonlinear outcome variables is also required.Trial registrationThe CODA study: ClinicalTrials.gov, identifier: NCT00708656 . Registered on 8 April 2008. The ZICE study trial: ClinicalTrials.gov, identifier: NCT00326820 . Registered on 16 May 2006.

Project description:BackgroundIn non-inferiority trials with non-adherence to interventions (or non-compliance), intention-to-treat and per-protocol analyses are often performed; however, non-random non-adherence generally biases these estimates of efficacy.ObjectiveTo identify statistical methods that adjust for the impact of non-adherence and thus estimate the causal effects of experimental interventions in non-inferiority trials.DesignA systematic review was conducted by searching the Ovid MEDLINE database (31 December 2020) to identify (1) randomised trials with a primary analysis for non-inferiority that applied (or planned to apply) statistical methods to account for the impact of non-adherence to interventions, and (2) methodology papers that described such statistical methods and included a non-inferiority trial application.OutcomesThe statistical methods identified, their impacts on non-inferiority conclusions, and their advantages/disadvantages.ResultsA total of 24 papers were included (4 protocols, 13 results papers and 7 methodology papers) reporting relevant methods on 26 occasions. The most common were instrumental variable approaches (n=9), including observed adherence as a covariate within a regression model (n=3), and modelling adherence as a time-varying covariate in a time-to-event analysis (n=3). Other methods included rank preserving structural failure time models and inverse-probability-of-treatment weighting. The methods identified in protocols and results papers were more commonly specified as sensitivity analyses (n=13) than primary analyses (n=3). Twelve results papers included an alternative analysis of the same outcome; conclusions regarding non-inferiority were in agreement on six occasions and could not be compared on six occasions (different measures of effect or results not provided in full).ConclusionsAvailable statistical methods which attempt to account for the impact of non-adherence to interventions were used infrequently. Therefore, firm inferences about their influence on non-inferiority conclusions could not be drawn. Since intention-to-treat and per-protocol analyses do not guarantee unbiased conclusions regarding non-inferiority, the methods identified should be considered for use in sensitivity analyses.Prospero registration numberCRD42020177458.

Dataset Information

Incomplete data analysis of non-inferiority clinical trials: Difference between binomial proportions case.

Background

Methods

Results

Conclusion

Publications

Incomplete data analysis of non-inferiority clinical trials: Difference between binomial proportions case.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets