Dataset Information

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy.

ABSTRACT: BACKGROUND:Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs). METHODS:A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification. RESULTS:Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods). CONCLUSIONS:This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed. TRIAL REGISTRATION:PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.

SUBMITTER: Phillips R

PROVIDER: S-EPMC7708917 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy.

Phillips Rachel R Sauzet Odile O Cornelius Victoria V

BMC medical research methodology 20201130 1

<h4>Background</h4>Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs).<h4>Methods</h4>A scoping review was undertaken to identify articles that proposed original me ...[more]

PMID: 33256641

Similar Datasets

Project description:BackgroundRandomised controlled trials (RCTs) provide valuable information and inform the development of harm profiles of new treatments. Harms are typically assessed through the collection of adverse events (AEs). Despite AEs being routine outcomes collected in trials, analysis and reporting of AEs in journal articles are continually shown to be suboptimal. One key challenge is the large volume of AEs, which can make evaluation and communication problematic. Prominent practice is to report frequency tables of AEs by arm. Visual displays offer an effective solution to assess and communicate complex information; however, they are rarely used and there is a lack of practical guidance on what and how to visually display complex AE data.MethodsIn this article, we demonstrate the use of two plots identified to be beneficial for wide use in RCTs, since both can display multiple AEs and are suitable to display point estimates for binary, count, or time-to-event AE data: the volcano and dot plots. We compare and contrast the use of data visualisations against traditional frequency table reporting, using published AE information in two placebo-controlled trials, of remdesivir for COVID-19 and GDNF for Parkinson disease. We introduce statistical programmes for implementation in Stata.Results/case studyVisualisations of AEs in the COVID-19 trial communicated a risk profile for remdesivir which differed from the main message in the published authors' conclusion. In the Parkinson's disease trial of GDNF, the visualisation provided immediate communication of harm signals, which had otherwise been contained within lengthy descriptive text and tables. Asymmetry in the volcano plot helped flag extreme events that were less obvious from review of the frequency table and dot plot. The dot plot allowed a more comprehensive representation by means of a more detailed summary.ConclusionsVisualisations can better support investigators to assimilate large volumes of data and enable improved informal between-arm comparisons compared to tables. We endorse increased uptake for use in trial publications. Care in construction of visual displays needs to be taken as there can be potential to overemphasise treatment effects in some circumstances.

Project description:IntroductionRandomised controlled trials (RCTs) aim to assess the effect of one (or more) unproven health interventions relative to other reference interventions. RCTs sometimes use an ordinal outcome, which is an endpoint that comprises of multiple, monotonically ordered categories that are not necessarily separated by a quantifiable distance. Ordinal outcomes are appealing in clinical settings as specific disease states can represent meaningful categories that may be of clinical importance to researchers. Ordinal outcomes can also retain information and increase statistical power compared to dichotomised outcomes and can allow multiple clinical outcomes to be comprised in a single endpoint. Target parameters for ordinal outcomes in RCTs may vary depending on the nature of the research question, the modelling assumptions and the expertise of the data analyst. The aim of this scoping review is to systematically describe the use of ordinal outcomes in contemporary RCTs. Specifically, we aim to: [Formula: see text] Identify which target parameters are of interest in trials that use an ordinal outcome, and whether these parameters are explicitly defined. [Formula: see text] Describe how ordinal outcomes are analysed in RCTs to estimate a treatment effect. [Formula: see text] Describe whether RCTs that use an ordinal outcome adequately report key methodological aspects specific to the analysis of the ordinal outcome. Results from this review will outline the current state of practice of the use of ordinal outcomes in RCTs. Ways to improve the analysis and reporting of ordinal outcomes in RCTs will be discussed.Methods and analysisWe will review RCTs that are published in the top four medical journals (British Medical Journal, New England Journal of Medicine, The Lancet and the Journal of the American Medical Association) between 1 January 2012 and 31 July 2022 that use an ordinal outcome as either a primary or a secondary outcome. The review will identify articles through a PubMed-specific search strategy. Our review will adhere to guidelines for scoping reviews as described in the PRISMA-ScR checklist. The study characteristics and details of the study design and analysis, including the target parameter(s) and statistical methods used to analyse the ordinal outcome, will be extracted from eligible studies. The screening, review and data extraction will be conducted using Covidence, a web-based tool for managing systematic reviews. The data will be summarised using descriptive statistics.

Project description:BACKGROUND:In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis. METHODS:We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms. RESULTS:All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms. CONCLUSIONS:In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3-6), small cluster sizes (5-10), and small ICC (?0.05) this model underestimates Type I error rates and there is no optimal model.

Project description:BackgroundRandomised controlled trials (RCTs) provide valuable information for developing harm profiles but current analysis practices to detect between-group differences are suboptimal. Drug trials routinely screen continuous clinical and biological data to monitor participant harm. These outcomes are regularly dichotomised into abnormal/normal values for analysis. Despite the simplicity gained for clinical interpretation, it is well established that dichotomising outcomes results in a considerable reduction in information and thus statistical power. We propose an automated procedure for the routine implementation of the distributional method for the dichotomisation of continuous outcomes proposed by Peacock and Sauzet, which retains the precision of the comparison of means.MethodsWe explored the use of a distributional approach to compare differences in proportions based on the comparison of means which retains the power of the latter. We applied this approach to the screening of clinical and biological data as a means to detect 'signals' for potential adverse drug reactions (ADRs). Signals can then be followed-up in further confirmatory studies. Three distributional methods suitable for different types of distributions are described. We propose the use of an automated approach using the observed data to select the most appropriate distribution as an analysis strategy in a RCT setting for multiple continuous outcomes. We illustrate this approach using data from three RCTs assessing the efficacy of mepolizumab in asthma or COPD. Published reference ranges were used to define the proportions of participants with abnormal values for a subset of 10 blood tests. The between-group distributional and empirical differences in proportions were estimated for each blood test and compared.ResultsWithin trials, the distributions varied across the 10 outcomes demonstrating value in a practical approach to selecting the distributional method in the context of multiple adverse event outcomes. Across trials, there were three outcomes where the method chosen by the automated procedure varied for the same outcome. The distributional approach identified three signals (eosinophils, haematocrit, and haemoglobin) compared to only one when using the Fisher's exact test (eosinophils) and two identified by use of the 95% confidence interval for the difference in proportions (eosinophils and potassium).ConclusionWhen dichotomisation of continuous adverse event outcomes aids clinical interpretation, we advocate use of a distributional approach to retain statistical power. Methods are now easy to implement. Retaining information is especially valuable in the context of the analysis of adverse events in RCTs. The routine implementation of this automated approach requires further evaluation.

Project description:BackgroundMeta-analyses of individual-level data from randomised trials are often required to detect clinically worthwhile effects. The Cholesterol Treatment Trialists' Collaboration, which includes data from numerous large long-term statin trials, is conducting a review of the effects of statin therapy on all adverse events collected in those trials. This article describes the approaches used and challenges faced to systematically capture and categorise the data.MethodsProtocols, statistical analysis plans, case report forms, clinical study reports and datasets were obtained, reviewed and checked. Relevant baseline and follow-up data from each trial was then reorganised into standardised formats based upon the Clinical Data Interchange Standards Consortium Study Data Tabulation Model. Adverse event data were organised and coded (automatically or, where necessary, manually) according to a common medical dictionary based upon the Medical Dictionary for Regulatory Activities.ResultsData from 23 double-blind statin trials and 5 open-label statin trials were provided, either through direct data transfer or through online access platforms. Together, these trials provided 845 datasets containing over 38 million records relating to 30,495 study variables and 181,973 randomised participants. Of the 46 Clinical Data Interchange Standards Consortium Study Data Tabulation Model domains that could potentially have been used to organise the data, the 13 most relevant to the project were identified and utilised, including 6 domains related to post-randomisation adverse events. Nearly 1.2 million adverse events were extracted and mapped to over 45,000 unique adverse event terms. Of these adverse events, 99% were coded to a Medical Dictionary for Regulatory Activities 'lower level term', with the remainder coded to a 'higher level term' or, very rarely, only a 'higher level group term'.ConclusionIn this meta-analysis of adverse event data from the large randomised trials of statins, approaches based on common standards for data organisation and classification have provided a resource capable of allowing reliable and rapid evaluation of any previously unknown benefits or hazards of statin therapy.

Project description:Background/aimsThe demand for simplified data collection within trials to increase efficiency and reduce costs has led to broader interest in repurposing routinely collected administrative data for use in clinical trials research. The aim of this scoping review is to describe how and why administrative data have been used in Australian randomised controlled trial conduct and analyses, specifically the advantages and limitations of their use as well as barriers and enablers to accessing administrative data for use alongside randomised controlled trials.MethodsDatabases were searched to November 2022. Randomised controlled trials were included if they accessed one or more Australian administrative data sets, where some or all trial participants were enrolled in Australia, and where the article was published between January 2000 and November 2022. Titles and abstracts were independently screened by two reviewers, and the full texts of selected studies were assessed against the eligibility criteria by two independent reviewers. Data were extracted from included articles by two reviewers using a data extraction tool.ResultsForty-one articles from 36 randomised controlled trials were included. Trial characteristics, including the sample size, disease area, population, and intervention, were varied; however, randomised controlled trials most commonly linked to government reimbursed claims data sets, hospital admissions data sets and birth/death registries, and the most common reason for linkage was to ascertain disease outcomes or survival status, and to track health service use. The majority of randomised controlled trials were able to achieve linkage in over 90% of trial participants; however, consent and participant withdrawals were common limitations to participant linkage. Reported advantages were the reliability and accuracy of the data, the ease of long term follow-up, and the use of established data linkage units. Common reported limitations were locating participants who had moved outside the jurisdictional area, missing data where consent was not provided, and unavailability of certain healthcare data.ConclusionsAs linked administrative data are not intended for research purposes, detailed knowledge of the data sets is required by researchers, and the time delay in receiving the data is viewed as a barrier to its use. The lack of access to primary care data sets is viewed as a barrier to administrative data use; however, work to expand the number of healthcare data sets that can be linked has made it easier for researchers to access and use these data, which may have implications on how randomised controlled trials will be run in future.

Project description:BackgroundIncreasingly in network meta-analysis (NMA), there is a need to incorporate non-randomised evidence to estimate relative treatment effects, and in particular in cases with limited randomised evidence, sometimes resulting in disconnected networks of treatments. When combining different sources of data, complex NMA methods are required to address issues associated with participant selection bias, incorporating single-arm trials (SATs), and synthesising a mixture of individual participant data (IPD) and aggregate data (AD). We develop NMA methods which synthesise data from SATs and randomised controlled trials (RCTs), using a mixture of IPD and AD, for a dichotomous outcome.MethodsWe propose methods under both contrast-based (CB) and arm-based (AB) parametrisations, and extend the methods to allow for both within- and across-trial adjustments for covariate effects. To illustrate the methods, we use an applied example investigating the effectiveness of biologic disease-modifying anti-rheumatic drugs for rheumatoid arthritis (RA). We applied the methods to a dataset obtained from a literature review consisting of 14 RCTs and an artificial dataset consisting of IPD from two SATs and AD from 12 RCTs, where the artificial dataset was created by removing the control arms from the only two trials assessing tocilizumab in the original dataset.ResultsWithout adjustment for covariates, the CB method with independent baseline response parameters (CBunadjInd) underestimated the effectiveness of tocilizumab when applied to the artificial dataset compared to the original dataset, albeit with significant overlap in posterior distributions for treatment effect parameters. The CB method with exchangeable baseline response parameters produced effectiveness estimates in agreement with CBunadjInd, when the predicted baseline response estimates were similar to the observed baseline response. After adjustment for RA duration, there was a reduction in across-trial heterogeneity in baseline response but little change in treatment effect estimates.ConclusionsOur findings suggest incorporating SATs in NMA may be useful in some situations where a treatment is disconnected from a network of comparator treatments, due to a lack of comparative evidence, to estimate relative treatment effects. The reliability of effect estimates based on data from SATs may depend on adjustment for covariate effects, although further research is required to understand this in more detail.

Dataset Information

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy.

Publications

Statistical methods for the analysis of adverse event data in randomised controlled trials: a scoping review and taxonomy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets