Dataset Information

Treatment effect heterogeneity for univariate subgroups in clinical trials: Shrinkage, standardization, or else.

ABSTRACT: Treatment effect heterogeneity is a well-recognized phenomenon in randomized controlled clinical trials. In this paper, we discuss subgroup analyses with prespecified subgroups of clinical or biological importance. We explore various alternatives to the naive (the traditional univariate) subgroup analyses to address the issues of multiplicity and confounding. Specifically, we consider a model-based Bayesian shrinkage (Bayes-DS) and a nonparametric, empirical Bayes shrinkage approach (Emp-Bayes) to temper the optimism of traditional univariate subgroup analyses; a standardization approach (standardization) that accounts for correlation between baseline covariates; and a model-based maximum likelihood estimation (MLE) approach. The Bayes-DS and Emp-Bayes methods model the variation in subgroup-specific treatment effect rather than testing the null hypothesis of no difference between subgroups. The standardization approach addresses the issue of confounding in subgroup analyses. The MLE approach is considered only for comparison in simulation studies as the "truth" since the data were generated from the same model. Using the characteristics of a hypothetical large outcome trial, we perform simulation studies and articulate the utilities and potential limitations of these estimators. Simulation results indicate that Bayes-DS and Emp-Bayes can protect against optimism present in the naïve approach. Due to its simplicity, the naïve approach should be the reference for reporting univariate subgroup-specific treatment effect estimates from exploratory subgroup analyses. Standardization, although it tends to have a larger variance, is suggested when it is important to address the confounding of univariate subgroup effects due to correlation between baseline covariates. The Bayes-DS approach is available as an R package (DSBayes).

SUBMITTER: Varadhan R

PROVIDER: S-EPMC5226126 | biostudies-literature | 2016 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Treatment effect heterogeneity for univariate subgroups in clinical trials: Shrinkage, standardization, or else.

Varadhan Ravi R Wang Sue-Jane SJ

Biometrical journal. Biometrische Zeitschrift 20151020 1

Treatment effect heterogeneity is a well-recognized phenomenon in randomized controlled clinical trials. In this paper, we discuss subgroup analyses with prespecified subgroups of clinical or biological importance. We explore various alternatives to the naive (the traditional univariate) subgroup analyses to address the issues of multiplicity and confounding. Specifically, we consider a model-based Bayesian shrinkage (Bayes-DS) and a nonparametric, empirical Bayes shrinkage approach (Emp-Bayes) ...[more]

PMID: 26485117

Similar Datasets

Project description:BackgroundInvestigators conducting randomized clinical trials often explore treatment effect heterogeneity to assess whether treatment efficacy varies according to patient characteristics. Identifying heterogeneity is central to making informed personalized healthcare decisions. Treatment effect heterogeneity can be investigated using subpopulation treatment effect pattern plot (STEPP), a non-parametric graphical approach that constructs overlapping patient subpopulations with varying values of a characteristic. Procedures for statistical testing using subpopulation treatment effect pattern plot when the endpoint of interest is survival remain an area of active investigation.MethodsA STEPP analysis was used to explore patterns of absolute and relative treatment effects for varying levels of a breast cancer biomarker, Ki-67, in the phase III Breast International Group 1-98 randomized clinical trial, comparing letrozole to tamoxifen as adjuvant therapy for postmenopausal women with hormone receptor-positive breast cancer. Absolute treatment effects were measured by differences in 4-year cumulative incidence of breast cancer recurrence, while relative effects were measured by the subdistribution hazard ratio in the presence of competing risks using O-E (observed-minus-expected) methodology, an intuitive non-parametric method. While estimation of hazard ratio values based on O-E methodology has been shown, a similar development for the subdistribution hazard ratio has not. Furthermore, we observed that the subpopulation treatment effect pattern plot analysis may not produce results, even with 100 patients within each subpopulation. After further investigation through simulation studies, we observed inflation of the type I error rate of the traditional test statistic and sometimes singular variance-covariance matrix estimates that may lead to results not being produced. This is due to the lack of sufficient number of events within the subpopulations, which we refer to as instability of the subpopulation treatment effect pattern plot analysis. We introduce methodology designed to improve stability of the subpopulation treatment effect pattern plot analysis and generalize O-E methodology to the competing risks setting. Simulation studies were designed to assess the type I error rate of the tests for a variety of treatment effect measures, including subdistribution hazard ratio based on O-E estimation. This subpopulation treatment effect pattern plot methodology and standard regression modeling were used to evaluate heterogeneity of Ki-67 in the Breast International Group 1-98 randomized clinical trial.ResultsWe introduce methodology that generalizes O-E methodology to the competing risks setting and that improves stability of the STEPP analysis by pre-specifying the number of events across subpopulations while controlling the type I error rate. The subpopulation treatment effect pattern plot analysis of the Breast International Group 1-98 randomized clinical trial showed that patients with high Ki-67 percentages may benefit most from letrozole, while heterogeneity was not detected using standard regression modeling.ConclusionThe STEPP methodology can be used to study complex patterns of treatment effect heterogeneity, as illustrated in the Breast International Group 1-98 randomized clinical trial. For the subpopulation treatment effect pattern plot analysis, we recommend a minimum of 20 events within each subpopulation.

Project description:BackgroundRisk of the outcome is a mathematical determinant of the absolute treatment benefit of an intervention, yet this can vary substantially within a trial population, complicating the interpretation of trial results.MethodsWe developed risk models using Cox or logistic regression on a set of large publicly available randomized controlled trials (RCTs). We evaluated risk heterogeneity using the extreme quartile risk ratio (EQRR, the ratio of outcome rates in the lowest risk quartile to that in the highest) and skewness using the median to mean risk ratio (MMRR, the ratio of risk in the median risk patient to the average). We also examined heterogeneity of treatment effects (HTE) across risk strata.ResultsWe describe 39 analyses using data from 32 large trials, with event rates across studies ranging from 3% to 63% (median?=?15%, 25th-75th percentile?=?9-29%). C-statistics of risk models ranged from 0.59 to 0.89 (median?=?0.70, 25th-75th percentile?=?0.65-0.71). The EQRR ranged from 1.8 to 50.7 (median?=?4.3, 25th-75th percentile?=?3.0-6.1). The MMRR ranged from 0.4 to 1.0 (median?=?0.86, 25th-75th percentile?=?0.80-0.92). EQRRs were predictably higher and MMRRs predictably lower as the c-statistic increased or the overall outcome incidence decreased. Among 18 comparisons with a significant overall treatment effect, there was a significant interaction between treatment and baseline risk on the proportional scale in only one. The difference in the absolute risk reduction between extreme risk quartiles ranged from -3.2 to 28.3% (median?=?5.1%; 25th-75th percentile?=?0.3-10.9).ConclusionsThere is typically substantial variation in outcome risk in clinical trials, commonly leading to clinically significant differences in absolute treatment effects Most patients have outcome risks lower than the trial average reflected in the summary result. Risk-stratified trial analyses are feasible and may be clinically informative, particularly when the outcome is predictable and uncommon.

Project description:BACKGROUND:Evidence-based health care is informed by results of randomized clinical trials (RCTs) and their syntheses in meta-analyses. When the trial outcomes measured are not clearly described in trial publications, knowledge synthesis, translation, and decision-making may be impeded. While heterogeneity in outcomes measured in adolescent major depressive disorder (MDD) RCTs has been described, the comprehensiveness of outcome reporting is unknown. This study aimed to assess the reporting of primary outcomes in RCTs evaluating treatments for adolescent MDD. METHODS:RCTs evaluating treatment interventions in adolescents with a diagnosis of MDD published between 2008 and 2017 specifying a single primary outcome were eligible for outcome reporting assessment. Outcome reporting assessment was done independently in duplicate using a comprehensive checklist of 58 reporting items. Primary outcome information provided in each RCT publication was scored as "fully reported", "partially reported", or "not reported" for each checklist item, as applicable. RESULTS:Eighteen of 42 identified articles were found to have a discernable single primary outcome and were included for outcome reporting assessment. Most trials (72%) did not fully report on over half of the 58 checklist items. Items describing masking of outcome assessors, timing and frequency of outcome assessment, and outcome analyses were fully reported in over 70% of trials. Items less frequently reported included outcome measurement instrument properties (ranging from 6 to 17%), justification of timing and frequency of outcome assessment (6%), and justification of criteria used for clinically significant differences (17%). The overall comprehensiveness of reporting appeared stable over time. CONCLUSIONS:Heterogeneous reporting exists in published adolescent MDD RCTs, with frequent omissions of key details about their primary outcomes. These omissions may impair interpretability, replicability, and synthesis of RCTs that inform clinical guidelines and decision-making in this field. Consensus on the minimal criteria for outcome reporting in adolescent MDD RCTs is needed.

Project description:BACKGROUND:Patients recruited in randomized controlled trials (RCTs) for biologic therapies in psoriasis are not fully representative of the real-world psoriasis population. OBJECTIVES:Firstly, to investigate whether patient characteristics are associated with being included in a psoriasis RCT. Secondly, to estimate the differences in the incidence of severe adverse events (SAEs) and the response rate between RCT and real-world populations of patients on biologic therapies for psoriasis using a standardization method. METHODS:Data from the British Association of Dermatologists Biologics and Immunomodulators Register (BADBIR) were appended to individual participant-level data from two RCTs assessing ustekinumab in patients with psoriasis. Baseline variables were assessed for association of being in an RCT using a multivariable logistic regression model. Propensity score weights were derived to reweigh the registry population so that variables had the distribution of the trial population. We measured the C-statistic of the model with trial status as the dependent variable, and the risk differences in the incidence rate of SAEs in the first year and Psoriasis Area and Severity Index (PASI) after 6 months in the BADBIR cohort before and after weighting. RESULTS:In total 6790 registry and 2021 RCT participants were included. The multivariable logistic regression model had a C-statistic of 0.82 [95% confidence interval (CI) 0.81-0.83]. The risk differences for the incidence rate of SAEs and the proportion of patients with PASI < 1.5 were 9.27 (95% CI -3.91-22.5) per 1000 person-years and 0.95 (95% CI -1.98-4.15), respectively. CONCLUSIONS:Our results suggest that RCTs of biologic therapies in patients with psoriasis are not fully representative of the real-world population, but this lack of external validity does not account for the efficacy-effectiveness gap. What's already known about this topic? Patients with psoriasis who would not be eligible for randomized controlled trials (RCTs) investigating biologic therapies have a greater risk of serious adverse events and lower treatment effectiveness than patients who would have been eligible. What does this study add? Baseline patient characteristics were shown to be predictive of whether a patient would have been eligible for enrolment in an RCT for psoriasis biologic therapy. We did not find any efficacy-effectiveness gap between the sample representative of the real-world population of patients with psoriasis and the sample representative of the RCT population. Factors outside of baseline patient characteristics, such as observer effect and higher adherence in RCTs, may be more influential in any efficacy-effectiveness gap between trial and real-world populations of patients with psoriasis.

Dataset Information

Treatment effect heterogeneity for univariate subgroups in clinical trials: Shrinkage, standardization, or else.

Publications

Treatment effect heterogeneity for univariate subgroups in clinical trials: Shrinkage, standardization, or else.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets