Dataset Information

Bias and inference from misspecified mixed-effect models in stepped wedge trial analysis.

ABSTRACT: Many stepped wedge trials (SWTs) are analysed by using a mixed-effect model with a random intercept and fixed effects for the intervention and time periods (referred to here as the standard model). However, it is not known whether this model is robust to misspecification. We simulated SWTs with three groups of clusters and two time periods; one group received the intervention during the first period and two groups in the second period. We simulated period and intervention effects that were either common-to-all or varied-between clusters. Data were analysed with the standard model or with additional random effects for period effect or intervention effect. In a second simulation study, we explored the weight given to within-cluster comparisons by simulating a larger intervention effect in the group of the trial that experienced both the control and intervention conditions and applying the three analysis models described previously. Across 500 simulations, we computed bias and confidence interval coverage of the estimated intervention effect. We found up to 50% bias in intervention effect estimates when period or intervention effects varied between clusters and were treated as fixed effects in the analysis. All misspecified models showed undercoverage of 95% confidence intervals, particularly the standard model. A large weight was given to within-cluster comparisons in the standard model. In the SWTs simulated here, mixed-effect models were highly sensitive to departures from the model assumptions, which can be explained by the high dependence on within-cluster comparisons. Trialists should consider including a random effect for time period in their SWT analysis model. © 2017 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

SUBMITTER: Thompson JA

PROVIDER: S-EPMC5600088 | biostudies-literature | 2017 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Bias and inference from misspecified mixed-effect models in stepped wedge trial analysis.

Thompson Jennifer A JA Fielding Katherine L KL Davey Calum C Aiken Alexander M AM Hargreaves James R JR Hayes Richard J RJ

Statistics in medicine 20170528 23

Many stepped wedge trials (SWTs) are analysed by using a mixed-effect model with a random intercept and fixed effects for the intervention and time periods (referred to here as the standard model). However, it is not known whether this model is robust to misspecification. We simulated SWTs with three groups of clusters and two time periods; one group received the intervention during the first period and two groups in the second period. We simulated period and intervention effects that were eithe ...[more]

PMID: 28556355

Similar Datasets

Project description:BACKGROUND:A stepped wedge cluster randomised trial (SWCRT) is a multicentred study which allows an intervention to be rolled out at sites in a random order. Once the intervention is initiated at a site, all participants within that site remain exposed to the intervention for the remainder of the study. The time since the start of the study ("calendar time") may affect outcome measures through underlying time trends or periodicity. The time since the intervention was introduced to a site ("exposure time") may also affect outcomes cumulatively for successful interventions, possibly in addition to a step change when the intervention began. METHODS:Motivated by a SWCRT of self-monitoring for bipolar disorder, we conducted a simulation study to compare model formulations to analyse data from a SWCRT under 36 different scenarios in which time was related to the outcome (improvement in mood score). The aim was to find a model specification that would produce reliable estimates of intervention effects under different scenarios. Nine different formulations of a linear mixed effects model were fitted to these datasets. These models varied in the specification of calendar and exposure times. RESULTS:Modelling the effects of the intervention was best accomplished by including terms for both calendar time and exposure time. Treating time as categorical (a separate parameter for each measurement time-step) achieved the best coverage probabilities and low bias, but at a cost of wider confidence intervals compared to simpler models for those scenarios which were sufficiently modelled by fewer parameters. Treating time as continuous and including a quadratic time term performed similarly well, with slightly larger variations in coverage probability, but narrower confidence intervals and in some cases lower bias. The impact of misspecifying the covariance structure was comparatively small. CONCLUSIONS:We recommend that unless there is a priori information to indicate the form of the relationship between time and outcomes, data from SWCRTs should be analysed with a linear mixed effects model that includes separate categorical terms for calendar time and exposure time. Prespecified sensitivity analyses should consider the different formulations of these time effects in the model, to assess their impact on estimates of intervention effects.

Project description:BackgroundStepped wedge randomised trial designs involve sequential roll-out of an intervention to participants (individuals or clusters) over a number of time periods. By the end of the study, all participants will have received the intervention, although the order in which participants receive the intervention is determined at random. The design is particularly relevant where it is predicted that the intervention will do more good than harm (making a parallel design, in which certain participants do not receive the intervention unethical) and/or where, for logistical, practical or financial reasons, it is impossible to deliver the intervention simultaneously to all participants. Stepped wedge designs offer a number of opportunities for data analysis, particularly for modelling the effect of time on the effectiveness of an intervention. This paper presents a review of 12 studies (or protocols) that use (or plan to use) a stepped wedge design. One aim of the review is to highlight the potential for the stepped wedge design, given its infrequent use to date.MethodsComprehensive literature review of studies or protocols using a stepped wedge design. Data were extracted from the studies in three categories for subsequent consideration: study information (epidemiology, intervention, number of participants), reasons for using a stepped wedge design and methods of data analysis.ResultsThe 12 studies included in this review describe evaluations of a wide range of interventions, across different diseases in different settings. However the stepped wedge design appears to have found a niche for evaluating interventions in developing countries, specifically those concerned with HIV. There were few consistent motivations for employing a stepped wedge design or methods of data analysis across studies. The methodological descriptions of stepped wedge studies, including methods of randomisation, sample size calculations and methods of analysis, are not always complete.ConclusionWhile the stepped wedge design offers a number of opportunities for use in future evaluations, a more consistent approach to reporting and data analysis is required.

Project description:Clinician miscommunication contributes to an estimated 250?000 deaths in US hospitals per year. Efforts to standardize handoff communication may reduce errors and improve patient safety.To determine the effect of a standardized handoff curriculum, UW-IPASS, on interclinician communication and patient outcomes.This cluster randomized stepped-wedge randomized clinical trial was conducted from October 2015 to May 2016 at 8 medical and surgical intensive care units at 2 hospital systems within an academic tertiary referral center. Participants included residents, fellows, advance-practice clinicians, and attending physicians (n?=?106 clinicians, with 1488 handoff events over 8 months) and data were collected from daily text message-based surveys and patient medical records.The UW-IPASS standardized handoff curriculum.The primary aim was to assess the effect of the UW-IPASS handoff curriculum on perceived adequacy of interclinician communication. Patient days of mechanical ventilation, intensive care unit length of stay, reintubations within 24 hours, and order workflow patterns were also analyzed. Mixed-effects logistic regression was used to compute odds ratios and confidence intervals with adjustment for location, time period, and clinician.A total of 63 residents and advance practice clinicians, 13 fellows, and 30 attending physicians participated in the study. During the control period, clinicians reported being unprepared for their shift because of a poor-quality handoff in 35 of 343 handoffs (10.2%), while UW-IPASS-period residents reported being unprepared in 53 of 740 handoffs (7.2%) (odds ratio, 0.19; 95% CI, 0.03-0.74; P?=?.03). Compared with the control phase, the perceived duration of handoffs among clinicians using UW-IPASS was unchanged (+5.5 minutes; 95% CI, 0.34-9.39; P?=?.30). Early morning order entry decreased from 106 per 100 patient-days in the control phase to 78 per 100 patient-days in the intervention period (-28 orders; 95% CI, -55 to -4; P?=?.04). Overall, UW-IPASS was not associated with any changes in intensive care unit length of stay, duration of mechanical ventilation, or the number of reintubations.The UW-IPASS standardized handoff curriculum was perceived to improve intensive care provider preparedness and workflow. IPASS-based curricula represent an important step forward in communication standardization efforts and may help reduce communication errors and omissions.isrctn.org Identifier: ISRCTN14209509.

Project description:BackgroundRisk of overdose, suicide, and other adverse outcomes are elevated among sub-populations prescribed opioid analgesics. To address this, the Veterans Health Administration (VHA) developed the Stratification Tool for Opioid Risk Mitigation (STORM)-a provider-facing dashboard that utilizes predictive analytics to stratify patients prescribed opioids based on risk for overdose/suicide.ObjectiveTo evaluate the impact of the case review mandate on serious adverse events (SAEs) and all-cause mortality among high-risk Veterans.DesignA 23-month stepped-wedge cluster randomized controlled trial in all 140 VHA medical centers between 2018 and 2020.ParticipantsA total of 44,042 patients actively prescribed opioid analgesics with high STORM risk scores (i.e., percentiles 1% to 5%) for an overdose or suicide-related event.InterventionA mandate requiring providers to perform case reviews on opioid analgesic-prescribed patients at high risk of overdose/suicide.Main measuresNine serious adverse events (SAEs), case review completion, number of risk mitigation strategies, and all-cause mortality.Key resultsMandated review inclusion was associated with a significant decrease in all-cause mortality within 4 months of inclusion (OR: 0.78; 95% CI: 0.65-0.94). There was no detectable effect on SAEs. Stepped-wedge analyses found that mandated review patients were five times more likely to receive a case review than non-mandated patients with similar risk (OR: 5.1; 95% CI: 3.64-7.23) and received more risk mitigation strategies than non-mandated patients (0.498; CI: 0.39-0.61).ConclusionsAmong VHA patients prescribed opioid analgesics, identifying high risk patients and mandating they receive an interdisciplinary case review was associated with a decrease in all-cause mortality. Results suggest that providers can leverage predictive analytic-targeted population health approaches and interdisciplinary collaboration to improve patient outcomes.Trial registrationISRCTN16012111.

Dataset Information

Bias and inference from misspecified mixed-effect models in stepped wedge trial analysis.

Publications

Bias and inference from misspecified mixed-effect models in stepped wedge trial analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets