A practical Bayesian stepped wedge design for community-based cluster-randomized clinical trials: The British Columbia Telehealth Trial.
Ontology highlight
ABSTRACT: Many clinical trial designs are impractical for community-based clinical intervention trials. Stepped wedge trial designs provide practical advantages, but few descriptions exist of their clinical implementational features, statistical design efficiencies, and limitations.Enhance efficiency of stepped wedge trial designs by evaluating the impact of design characteristics on statistical power for the British Columbia Telehealth Trial.The British Columbia Telehealth Trial is a community-based, cluster-randomized, controlled clinical trial in rural and urban British Columbia. To determine the effect of an Internet-based telehealth intervention on healthcare utilization, 1000 subjects with an existing diagnosis of congestive heart failure or type 2 diabetes will be enrolled from 50 clinical practices. Hospital utilization is measured using a composite of disease-specific hospital admissions and emergency visits. The intervention comprises online telehealth data collection and counseling provided to support a disease-specific action plan developed by the primary care provider. The planned intervention is sequentially introduced across all participating practices. We adopt a fully Bayesian, Markov chain Monte Carlo-driven statistical approach, wherein we use simulation to determine the effect of cluster size, sample size, and crossover interval choice on type I error and power to evaluate differences in hospital utilization.For our Bayesian stepped wedge trial design, simulations suggest moderate decreases in power when crossover intervals from control to intervention are reduced from every 3 to 2?weeks, and dramatic decreases in power as the numbers of clusters decrease. Power and type I error performance were not notably affected by the addition of nonzero cluster effects or a temporal trend in hospitalization intensity.Stepped wedge trial designs that intervene in small clusters across longer periods can provide enhanced power to evaluate comparative effectiveness, while offering practical implementation advantages in geographic stratification, temporal change, use of existing data, and resource distribution. Current population estimates were used; however, models may not reflect actual event rates during the trial. In addition, temporal or spatial heterogeneity can bias treatment effect estimates.
<h4>Background</h4>Many clinical trial designs are impractical for community-based clinical intervention trials. Stepped wedge trial designs provide practical advantages, but few descriptions exist of their clinical implementational features, statistical design efficiencies, and limitations.<h4>Objectives</h4>Enhance efficiency of stepped wedge trial designs by evaluating the impact of design characteristics on statistical power for the British Columbia Telehealth Trial.<h4>Methods</h4>The Briti ...[more]
Project description:BackgroundThe Bayesian group sequential design has been applied widely in clinical studies, especially in Phase II and III studies. It allows early termination based on accumulating interim data. However, to date, there lacks development in its application to stepped-wedge cluster randomized trials, which are gaining popularity in pragmatic trials conducted by clinical and health care delivery researchers.MethodsWe propose a Bayesian adaptive design approach for stepped-wedge cluster randomized trials, which makes adaptive decisions based on the predictive probability of declaring the intervention effective at the end of study given interim data. The Bayesian models and the algorithms for posterior inference and trial conduct are presented.ResultsWe present how to determine design parameters through extensive simulations to achieve desired operational characteristics. We further evaluate how various design factors, such as the number of steps, cluster size, random variability in cluster size, and correlation structures, impact trial properties, including power, type I error, and the probability of early stopping. An application example is presented.ConclusionThis study presents the incorporation of Bayesian adaptive strategies into stepped-wedge cluster randomized trials design. The proposed approach provides the flexibility to stop the trial early if substantial evidence of efficacy or futility is observed, improving the flexibility and efficiency of stepped-wedge cluster randomized trials.
Project description:Permutation tests are useful in stepped-wedge trials to provide robust statistical tests of intervention-effect estimates. However, the Stata command permute does not produce valid tests in this setting because individual observations are not exchangeable. We introduce the swpermute command that permutes clusters to sequences to maintain exchangeability. The command provides additional functionality to aid users in performing analyses of stepped-wedge trials. In particular, we include the option "withinperiod" that performs the specified analysis separately in each period of the study with the resulting period-specific intervention-effect estimates combined as a weighted average. We also include functionality to test non-zero null hypotheses to aid the construction of confidence intervals. Examples of the application of swpermute are given using data from a trial testing the impact of a new tuberculosis diagnostic test on bacterial confirmation of a tuberculosis diagnosis.
Project description:BACKGROUND:Cluster randomised trials with unequal sized clusters often have lower precision than with clusters of equal size. To allow for this, sample sizes are inflated by a modified version of the design effect for clustering. These inflation factors are valid under the assumption that randomisation is stratified by cluster size. We investigate the impact of unequal cluster size when that constraint is relaxed, with particular focus on the stepped-wedge cluster randomised trial, where this is more difficult to achieve. METHODS:Assuming a multi-level mixed effect model with exchangeable correlation structure for a cross-sectional design, we use simulation methods to compare the precision for a trial with clusters of unequal size to a trial with clusters of equal size (relative efficiency). For a range of scenarios we illustrate the impact of various design features (the cluster-mean correlation - a function of the intracluster correlation and the cluster size, the number of clusters, number of randomisation sequences) on the average and distribution of the relative efficiency. RESULTS:Simulations confirm that the average reduction in precision, due to varying cluster sizes, is smaller in a stepped-wedge trial compared to the parallel trial. However, the variance of the distribution of the relative efficiency is large; and is larger under the stepped-wedge design compared to the parallel design. This can result in large variations in actual power, depending on the allocation of clusters to sequences. Designs with larger variations in cluster sizes, smaller number of clusters and studies with smaller cluster-mean correlations (smaller cluster sizes or smaller intra-cluster correlation) are particularly at risk. CONCLUSION:The actual realised power in a stepped-wedge trial might be substantially higher or lower than that estimated. This is particularly important when there are a small number of clusters or the variability in cluster sizes is large. Constraining the randomisation on cluster size, where feasible, might mitigate this effect.
Project description:OBJECTIVES:To investigate the extent to which cluster sizes vary in stepped-wedge cluster randomised trials (SW-CRT) and whether any variability is accounted for during the sample size calculation and analysis of these trials. SETTING:Any, not limited to healthcare settings. PARTICIPANTS:Any taking part in an SW-CRT published up to March 2016. PRIMARY AND SECONDARY OUTCOME MEASURES:The primary outcome is the variability in cluster sizes, measured by the coefficient of variation (CV) in cluster size. Secondary outcomes include the difference between the cluster sizes assumed during the sample size calculation and those observed during the trial, any reported variability in cluster sizes and whether the methods of sample size calculation and methods of analysis accounted for any variability in cluster sizes. RESULTS:Of the 101 included SW-CRTs, 48% mentioned that the included clusters were known to vary in size, yet only 13% of these accounted for this during the calculation of the sample size. However, 69% of the trials did use a method of analysis appropriate for when clusters vary in size. Full trial reports were available for 53 trials. The CV was calculated for 23 of these: the median CV was 0.41 (IQR: 0.22-0.52). Actual cluster sizes could be compared with those assumed during the sample size calculation for 14 (26%) of the trial reports; the cluster sizes were between 29% and 480% of that which had been assumed. CONCLUSIONS:Cluster sizes often vary in SW-CRTs. Reporting of SW-CRTs also remains suboptimal. The effect of unequal cluster sizes on the statistical power of SW-CRTs needs further exploration and methods appropriate to studies with unequal cluster sizes need to be employed.
Project description:In stepped-wedge trials (SWTs), the intervention is rolled out in a random order over more than 1 time-period. SWTs are often analysed using mixed-effects models that require strong assumptions and may be inappropriate when the number of clusters is small. We propose a non-parametric within-period method to analyse SWTs. This method estimates the intervention effect by comparing intervention and control conditions in a given period using cluster-level data corresponding to exposure. The within-period intervention effects are combined with an inverse-variance-weighted average, and permutation tests are used. We present an example and, using simulated data, compared the method to (1) a parametric cluster-level within-period method, (2) the most commonly used mixed-effects model, and (3) a more flexible mixed-effects model. We simulated scenarios where period effects were common to all clusters, and when they varied according to a distribution informed by routinely collected health data. The non-parametric within-period method provided unbiased intervention effect estimates with correct confidence-interval coverage for all scenarios. The parametric within-period method produced confidence intervals with low coverage for most scenarios. The mixed-effects models' confidence intervals had low coverage when period effects varied between clusters but had greater power than the non-parametric within-period method when period effects were common to all clusters. The non-parametric within-period method is a robust method for analysing SWT. The method could be used by trial statisticians who want to emphasise that the SWT is a randomised trial, in the common position of being uncertain about whether data will meet the assumptions necessary for mixed-effect models.
Project description:BackgroundStepped-wedge cluster randomized trial (SW-CRT) designs are often used when there is a desire to provide an intervention to all enrolled clusters, because of a belief that it will be effective. However, given there should be equipoise at trial commencement, there has been discussion around whether a pre-trial decision to provide the intervention to all clusters is appropriate. In pharmaceutical drug development, a solution to a similar desire to provide more patients with an effective treatment is to use a response adaptive (RA) design.MethodsWe introduce a way in which RA design could be incorporated in an SW-CRT, permitting modification of the intervention allocation during the trial. The proposed framework explicitly permits a balance to be sought between power and patient benefit considerations. A simulation study evaluates the methodology.ResultsIn one scenario, for one particular RA design, the proportion of cluster-periods spent in the intervention condition was observed to increase from 32.2% to 67.9% as the intervention effect was increased. A cost of this was a 6.2% power drop compared to a design that maximized power by fixing the proportion of time in the intervention condition at 45.0%, regardless of the intervention effect.ConclusionsAn RA approach may be most applicable to settings for which the intervention has substantial individual or societal benefit considerations, potentially in combination with notable safety concerns. In such a setting, the proposed methodology may routinely provide the desired adaptability of the roll-out speed, with only a small cost to the study's power.
Project description:A stepped wedge cluster randomized trial is a type of longitudinal cluster design that sequentially switches clusters to intervention over time until all clusters are treated. While the traditional posttest-only parallel design requires adjustment for a single intraclass correlation coefficient, the stepped wedge design allows multiple outcome measurements from the same cluster and so additional correlation parameters are necessary to characterize the within-cluster correlation structure. Although a number of studies have differentiated between the concepts of within-period and between-period correlations, only a few studies have allowed the between-period correlation to decay over time. In this article, we consider the proportional decay correlation structure for a cohort stepped wedge design, and provide a matrix-adjusted quasi-least squares approach to accurately estimate the correlation parameters along with the marginal intervention effect. We further develop the sample size and power procedures accounting for the correlation decay, and investigate the accuracy of the power procedure with continuous outcomes in a simulation study. We show that the empirical power agrees well with the prediction even with as few as nine clusters, when data are analyzed with matrix-adjusted quasi-least squares concurrently with a suitable bias-corrected sandwich variance. Two trial examples are provided to illustrate the new sample size procedure.
Project description:BACKGROUND:Stepped-wedge cluster randomized trials (SW-CRTs) are increasingly popular in health-related research in both high- and low-resource settings. There may be specific ethical issues that researchers face when designing and conducting SW-CRTs in low-resource settings. Knowledge of these issues can help to improve the ethical conduct of SW-CRTs in a global health context. METHODS:We performed an ethical analysis of two studies using SW-CRT designs in low-resource settings: the Que Vivan Las Madres study conducted from 2014 to 2017 in Guatemala and the Atmiyata study conducted from 2017 to 2018 in rural parts of India. For both case studies, we identified and evaluated the classification of the study as research or nonresearch and the ethical issues regarding the justification of the design, including the delayed rollout of an intervention that had a promising effect. RESULTS:In our case studies, some minor ethical issues surfaced about the registration and stakeholder pressure on the order of randomization, but both included good justification for the design and delayed rollout. Our analysis did, however, demonstrate that careful consideration of the role of randomization and registration of the trials is important. DISCUSSION:SW-CRTs can provide an opportunity for rigorous evaluation of interventions destined to be rolled out on the basis of limited evidence. Furthermore, in SW-CRTs, the underlying objective is often to provide a robust evaluation of the effectiveness for generalized dissemination, and this makes the SW-CRT no less a research study than any other form of cluster randomized trial. CONCLUSION:The design and conduct of stepped-wedge cluster randomized trials raises at least two ethical issues that need special consideration in both high- and low-resource settings: the justification for using the design, specifically the delayed rollout of the intervention to the control group, and the classification of the study as research or nonresearch. In our case studies, these issues did not seem to raise special ethical scrutiny in low-resource settings. Further ethical evaluation will hopefully result in specific ethical guidelines for the use of SW-CRTs in both high- and low-resource settings to contribute to responsible functioning of these trials and adequate protection of participants.
Project description:Stepped-wedge cluster randomized trial (SW-CRT) designs are increasingly employed in pragmatic research; they differ from traditional parallel cluster randomized trials in which an intervention is delivered to a subset of clusters, but not to all. In a SW-CRT, all clusters receive the intervention under investigation by the end of the study. This approach is thought to avoid ethical concerns about the denial of a desired intervention to participants in control groups. Such concerns have been cited in the literature as a primary motivation for choosing SW-CRT design, however SW-CRTs raise additional ethical concerns related to the delayed implementation of an intervention and consent. Yet, PCT investigators may choose SW-CRT designs simply because they are concerned that other study designs are infeasible. In this paper, we examine justifications for the use of SW-CRT study design, over other designs, by drawing on the experience of the National Institutes of Health's Health Care Systems Research Collaboratory (NIH Collaboratory) with five pragmatic SW-CRTs. We found that decisions to use SW-CRT design were justified by practical and epistemic reasons rather than ethical ones. These include concerns about feasibility, the heterogeneity of cluster characteristics, and the desire for simultaneous clinical evaluation and implementation. In this paper we compare the potential benefits of SW-CRTs against the ethical and epistemic challenges brought forth by the design and suggest that the choice of SW-CRT design must balance epistemic, feasibility and ethical justifications. Moreover, given their complexity, such studies need rigorous and informed ethical oversight.
Project description:Stepped wedge cluster randomized trials (SW-CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW-CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model-based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within-cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW-CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW-CRTs might well consider such methods when common modeling assumptions may not hold.
