Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials.
Ontology highlight
ABSTRACT:
SUBMITTER: Wallach JD
PROVIDER: S-EPMC6657347 | biostudies-literature | 2017 Apr
REPOSITORIES: biostudies-literature
Publications
Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials.
Wallach Joshua D JD Sullivan Patrick G PG Trepanowski John F JF Sainani Kristin L KL Steyerberg Ewout W EW Ioannidis John P A JP
JAMA internal medicine 20170401 4
<h4>Importance</h4>Many published randomized clinical trials (RCTs) make claims for subgroup differences.<h4>Objective</h4>To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses.<h4>Data sources</h4>This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy ( ...[more]
Similar Datasets
Project description: Not available
| S-EPMC8552052 | biostudies-literature
Project description:BACKGROUND:When investigating subgroup effects in meta-analysis, it is unclear whether accounting in meta-regression for between-trial variation in treatment effects, but not between-trial variation in treatment interaction effects when such effects are present, leads to biased estimates, coverage problems, or wrong standard errors, and whether the use of aggregate data (AD) or individual-patient-data (IPD) influences this assessment. METHODS:Seven different models were compared in a simulation study. Models differed regarding the use of AD or IPD, whether they accounted for between-trial variation in interaction effects, and whether they minimized the risk of ecological fallacy. RESULTS:Models that used IPD and that allowed for between-trial variation of the interaction effect had less bias, better coverage, and more accurate standard errors than models that used AD or ignored this variation. The main factor influencing the performance of models was whether they used IPD or AD. The model that used AD had a considerably worse performance than all models that used IPD, especially when a low number of trials was included in the analysis. CONCLUSIONS:The results indicate that IPD models that allow for the between-trial variation in interaction effects should be given preference whenever investigating subgroup effects within a meta-analysis.
| S-EPMC6815379 | biostudies-literature
Project description:Objective To evaluate the frequency, validity, and relevance of statistically significant (P<0.05) sex-treatment interactions in randomized controlled trials in Cochrane meta-analyses.Design Meta-epidemiological study.Data sources Cochrane Database of Systematic Reviews (CDSR) and PubMed.Eligibility criteria for study selection Reviews published in the CDSR with sex-treatment subgroup analyses in the forest plots, using data from randomized controlled trials.Data extraction Information on the study design and sex subgroup data were extracted from reviews and forest plots that met inclusion criteria. For each statistically significant sex-treatment interaction, the potential for biological plausibility and clinical significance was considered.Results Among the 41 reviews with relevant data, there were 109 separate treatment-outcome analyses ("topics"). Among the 109 topics, eight (7%) had a statistically significant sex-treatment interaction. The 109 topics included 311 randomized controlled trials (162 with both sexes, 46 with males only, 103 with females only). Of the 162 individual randomized controlled trials that included both sexes, 15 (9%) had a statistically significant sex-treatment interaction. Of four topics where the first published randomized controlled trial had a statistically significant sex-treatment interaction, no meta-analyses that included other randomized controlled trials retained the statistical significance and no meta-analyses showed statistical significance when data from the first published randomized controlled trial were excluded. Of the eight statistically significant sex-treatment interactions from the overall analyses, only three were discussed by the CDSR reviewers for a potential impact on different clinical management for males compared with females. None of these topics had a sex-treatment interaction that influenced treatment recommendations in recent guidelines. UpToDate, an online physician-authored clinical decision support resource, suggested differential management of men and women for one of these sex-treatment interactions.Conclusion Statistically significant sex-treatment interactions are only slightly more frequent than what would be expected by chance and there is little evidence of subsequent corroboration or clinical relevance of sex-treatment interactions.
| S-EPMC5122320 | biostudies-literature
Project description:Medulloblastoma is a heterogeneous disease with at least four distinct molecular subgroups: wingless (WNT), sonic hedgehog (SHH), Group 3, and Group 4. Recently there has been considerable progress defining the molecular drivers and prognostic factors of each subgroup. However, this information has only rarely been used to stratify risk or impact treatment. The purpose of this work is to provide an update on current clinical trials that provide molecularly stratified treatment paradigms. A search was conducted on ClinicalTrials.gov using the following search terms: "medulloblastoma and subgroup", "medulloblastoma and SHH", "medulloblastoma and WNT", and "medulloblastoma and Non-WNT/Non-SHH". This search resulted in nine distinct clinical trials, five for newly diagnosed medulloblastoma and four for recurrent medulloblastoma. Four trials for newly diagnosed medulloblastoma had a component of craniospinal irradiation reduction for patients with WNT medulloblastoma. Molecularly stratified trials for recurrent medulloblastoma largely focus on SHH. As these trials are ongoing, there are limited data available. A trial in which newly-diagnosed WNT patients received modest chemotherapy without radiation has been closed to accrual due to several early failures. Phase II trials evaluating vismodegib for SHH medulloblastoma in children and adults have been disappointing. In conclusion, although there is an expanding array of clinical trials which incorporate molecular data in prescribing treatment for newly-diagnosed and recurrent medulloblastoma, treatments for these diseases are fairly uniform, with craniospinal radiation dose being the main variable. As the drivers of the distinct subgroups and their associated prognoses are better elucidated, future clinical trials and novel targeted agents are needed to improve outcomes and reduce toxicity where feasible.
| S-EPMC7237968 | biostudies-literature
Project description:Phase I trials often include a dose expansion cohort (DEC), in which additional patients are treated at the estimated maximum tolerated dose (MTD) after dose escalation, with the goal of ensuring that data are available from more than six patients at a single dose level. However, protocols do not always detail how, or even if, the additional toxicity data will be used to reanalyze the MTD or whether observed toxicity in the DEC will warrant changing the assigned dose. A DEC strategy has not been statistically justified. We conducted a simulation study of two phase I designs: the "3+3" and the Continual Reassessment Method (CRM). We quantified how many patients are assigned the true MTD using a 10 to 20 patient DEC and how a sensible reanalysis using the DEC changes the probability of selecting the true MTD. We compared these results with those from an equivalently sized larger CRM that does not include a DEC. With either the 3+3 or CRM, reanalysis with the DEC increased the probability of identifying the true MTD. However, a large CRM without a DEC was more likely to identify the true MTD while still treating 10 or 15 patients at this dose level. Where feasible, a CRM design with no explicit DEC is preferred to designs that fix a dose for all patients in a DEC.
| S-EPMC4565529 | biostudies-literature
Project description:BACKGROUND:Until recently a typical rule that has often been used for the endorsement of new medications by the Food and Drug Administration has been the existence of at least two statistically significant clinical trials favoring the new medication. This rule has consequences for the true positive (endorsement of an effective treatment) and false positive rates (endorsement of an ineffective treatment). METHODS:In this paper, we compare true positive and false positive rates for different evaluation criteria through simulations that rely on (1) conventional p-values; (2) confidence intervals based on meta-analyses assuming fixed or random effects; and (3) Bayes factors. We varied threshold levels for statistical evidence, thresholds for what constitutes a clinically meaningful treatment effect, and number of trials conducted. RESULTS:Our results show that Bayes factors, meta-analytic confidence intervals, and p-values often have similar performance. Bayes factors may perform better when the number of trials conducted is high and when trials have small sample sizes and clinically meaningful effects are not small, particularly in fields where the number of non-zero effects is relatively large. CONCLUSIONS:Thinking about realistic effect sizes in conjunction with desirable levels of statistical evidence, as well as quantifying statistical evidence with Bayes factors may help improve decision-making in some circumstances.
| S-EPMC6882054 | biostudies-literature
Project description:BackgroundClinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools.ObjectiveThis paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care.MethodsA multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior.ResultsTo demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images.ConclusionsThese experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS.Trial registrationFlu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.
| S-EPMC8065554 | biostudies-literature
Project description:A randomized clinical trial is widely regarded as the most rigorous study design to determine the efficacy of intervention because spurious causality and bias associated with other experimental designs can be avoided. The purpose of this article is to provide clinicians and clinical researchers the types of randomized clinical trials used in stroke studies and to discuss the advantages and the limitations for each type of randomized stroke clinical trials.
| S-EPMC2837939 | biostudies-other
Project description:Background A recent focus in the health sciences has been the development of personalized medicine, which includes determining the population for which a given treatment is effective. Due to limited data, identifying the true benefiting population is a challenging task. To tackle this difficulty, the credible subgroups approach provides a pair of bounding subgroups for the true benefiting subgroup, constructed so that one is contained by the benefiting subgroup while the other contains the benefiting subgroup with high probability. However, the method has so far only been developed for parametric linear models. Methods In this article, we develop the details required to follow the credible subgroups approach in more realistic settings by considering nonlinear and semiparametric regression models, supported for regulatory science by conditional power simulations. We also present an improved multiple testing approach using a step-down procedure. We evaluate our approach via simulations and apply it to data from four trials of Alzheimer's disease treatments carried out by AbbVie. Results Semiparametric modeling yields credible subgroups that are more robust to violations of linear treatment effect assumptions, and careful choice of the population of interest as well as the step-down multiple testing procedure result in a higher rate of detection of benefiting types of patients. The approach allows us to identify types of patients that benefit from treatment in the Alzheimer's disease trials. Conclusion Attempts to identify benefiting subgroups of patients in clinical trials are often met with skepticism due to a lack of multiplicity control and unrealistically restrictive assumptions. Our proposed approach merges two techniques, credible subgroups, and semiparametric regression, which avoids these problems and makes benefiting subgroup identification practical and reliable.
| S-EPMC8796253 | biostudies-literature
Project description:In this paper, we illustrate the method of designing a group-sequential randomized clinical trial based on the difference in restricted mean survival time (RMST). The procedure is based on theoretical formulations of Murray and Tsiatis (1999). We also present a numerical example in designing a cardiology surgical trial. Various practical considerations are discussed. R codes are provided in the Supplementary Materials. We conclude that the group-sequential design for RMST is a viable option in practice. A simulation study is performed to compare the proposed method to the Max-Combo and conventional log-rank tests. The simulation result shows that when there is a delayed treatment benefit and the proportional hazards assumption is untrue, the sequential design based on the RMST can be more efficient than that based on the log-rank test but less efficient than that based on the Max-Combo test. Compared with Max-Combo test, the RMST-based study design yield coherent estimand, statistical inference and result interpretation.
| S-EPMC8078843 | biostudies-literature