Dataset Information

Use of external evidence for design and Bayesian analysis of clinical trials: a qualitative study of trialists’ views

ABSTRACT:

Background

Evidence from previous studies is often used relatively informally in the design of clinical trials: for example, a systematic review to indicate whether a gap in the current evidence base justifies a new trial. External evidence can be used more formally in both trial design and analysis, by explicitly incorporating a synthesis of it in a Bayesian framework. However, it is unclear how common this is in practice or the extent to which it is considered controversial. In this qualitative study, we explored attitudes towards, and experiences of, trialists in incorporating synthesised external evidence through the Bayesian design or analysis of a trial.

Methods

Semi-structured interviews were conducted with 16 trialists: 13 statisticians and three clinicians. Participants were recruited across several universities and trials units in the United Kingdom using snowball and purposeful sampling. Data were analysed using thematic analysis and techniques of constant comparison.

Results

Trialists used existing evidence in many ways in trial design, for example, to justify a gap in the evidence base and inform parameters in sample size calculations. However, no one in our sample reported using such evidence in a Bayesian framework. Participants tended to equate Bayesian analysis with the incorporation of prior information on the intervention effect and were less aware of the potential to incorporate data on other parameters. When introduced to the concepts, many trialists felt they could be making more use of existing data to inform the design and analysis of a trial in particular scenarios. For example, some felt existing data could be used more formally to inform background adverse event rates, rather than relying on clinical opinion as to whether there are potential safety concerns. However, several barriers to implementing these methods in practice were identified, including concerns about the relevance of external data, acceptability of Bayesian methods, lack of confidence in Bayesian methods and software, and practical issues, such as difficulties accessing relevant data.

Conclusions

Despite trialists recognising that more formal use of external evidence could be advantageous over current approaches in some areas and useful as sensitivity analyses, there are still barriers to such use in practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05759-8.

SUBMITTER: Clayton G

PROVIDER: S-EPMC8577005 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundRandomized controlled trials are considered the gold standard for evaluating experimental treatments but often require large sample sizes. Single-arm trials require smaller sample sizes but are subject to bias when using historical control data for comparative inferences. This article presents a Bayesian adaptive synthetic-control design that exploits historical control data to create a hybrid of a single-arm trial and a randomized controlled trial.MethodsThe Bayesian adaptive synthetic control design has two stages. In stage 1, a prespecified number of patients are enrolled in a single arm given the experimental treatment. Based on the stage 1 data, applying propensity score matching and Bayesian posterior prediction methods, the usefulness of the historical control data for identifying a pseudo sample of matched synthetic-control patients for making comparative inferences is evaluated. If a sufficient number of synthetic controls can be identified, the single-arm trial is continued. If not, the trial is switched to a randomized controlled trial. The performance of The Bayesian adaptive synthetic control design is evaluated by computer simulation.ResultsThe Bayesian adaptive synthetic control design achieves power and unbiasedness similar to a randomized controlled trial but on average requires a much smaller sample size, provided that the historical control data patients are sufficiently comparable to the trial patients so that a good number of matched controls can be identified in the historical control data. Compared to a single-arm trial, The Bayesian adaptive synthetic control design yields much higher power and much smaller bias.ConclusionThe Bayesian adaptive synthetic-control design provides a useful tool for exploiting historical control data to improve the efficiency of single-arm phase II clinical trials, while addressing the problem of bias when comparing trial results to historical control data. The proposed design achieves power similar to a randomized controlled trial but may require a substantially smaller sample size.

Project description:BackgroundWhen designing and analysing clinical trials, using previous relevant information, perhaps in the form of evidence syntheses, can reduce research waste. We conducted the INVEST (INVestigating the use of Evidence Synthesis in the design and analysis of clinical Trials) survey to summarise the current use of evidence synthesis in trial design and analysis, to capture opinions of trialists and methodologists on such use, and to understand any barriers.MethodsOur sampling frame was all delegates attending the International Clinical Trials Methodology Conference in November 2015. Respondents were asked to indicate (1) their views on the use of evidence synthesis in trial design and analysis, (2) their own use during the past 10 years and (3) the three greatest barriers to use in practice.ResultsOf approximately 638 attendees of the conference, 106 (17%) completed the survey, half of whom were statisticians. Support was generally high for using a description of previous evidence, a systematic review or a meta-analysis in trial design. Generally, respondents did not seem to be using evidence syntheses as often as they felt they should. For example, only 50% (42/84 relevant respondents) had used a meta-analysis to inform whether a trial is needed compared with 74% (62/84) indicating that this is desirable. Only 6% (5/81 relevant respondents) had used a value of information analysis to inform sample size calculations versus 22% (18/81) indicating support for this. Surprisingly large numbers of participants indicated support for, and previous use of, evidence syntheses in trial analysis. For example, 79% (79/100) of respondents indicated that external information about the treatment effect should be used to inform aspects of the analysis. The greatest perceived barrier to using evidence synthesis methods in trial design or analysis was time constraints, followed by a belief that the new trial was the first in the area.ConclusionsEvidence syntheses can be resource-intensive, but their use in informing the design, conduct and analysis of clinical trials is widely considered desirable. We advocate additional research, training and investment in resources dedicated to ways in which evidence syntheses can be undertaken more efficiently, offering the potential for cost savings in the long term.