Dataset Information

Advantages of visualisations to evaluate and communicate adverse event information in randomised controlled trials.

ABSTRACT:

Background

Randomised controlled trials (RCTs) provide valuable information and inform the development of harm profiles of new treatments. Harms are typically assessed through the collection of adverse events (AEs). Despite AEs being routine outcomes collected in trials, analysis and reporting of AEs in journal articles are continually shown to be suboptimal. One key challenge is the large volume of AEs, which can make evaluation and communication problematic. Prominent practice is to report frequency tables of AEs by arm. Visual displays offer an effective solution to assess and communicate complex information; however, they are rarely used and there is a lack of practical guidance on what and how to visually display complex AE data.

Methods

In this article, we demonstrate the use of two plots identified to be beneficial for wide use in RCTs, since both can display multiple AEs and are suitable to display point estimates for binary, count, or time-to-event AE data: the volcano and dot plots. We compare and contrast the use of data visualisations against traditional frequency table reporting, using published AE information in two placebo-controlled trials, of remdesivir for COVID-19 and GDNF for Parkinson disease. We introduce statistical programmes for implementation in Stata.

Results/case study

Visualisations of AEs in the COVID-19 trial communicated a risk profile for remdesivir which differed from the main message in the published authors' conclusion. In the Parkinson's disease trial of GDNF, the visualisation provided immediate communication of harm signals, which had otherwise been contained within lengthy descriptive text and tables. Asymmetry in the volcano plot helped flag extreme events that were less obvious from review of the frequency table and dot plot. The dot plot allowed a more comprehensive representation by means of a more detailed summary.

Conclusions

Visualisations can better support investigators to assimilate large volumes of data and enable improved informal between-arm comparisons compared to tables. We endorse increased uptake for use in trial publications. Care in construction of visual displays needs to be taken as there can be potential to overemphasise treatment effects in some circumstances.

SUBMITTER: Cornelius V

PROVIDER: S-EPMC7754702 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundRandomised controlled trials (RCTs) provide valuable information for developing harm profiles but current analysis practices to detect between-group differences are suboptimal. Drug trials routinely screen continuous clinical and biological data to monitor participant harm. These outcomes are regularly dichotomised into abnormal/normal values for analysis. Despite the simplicity gained for clinical interpretation, it is well established that dichotomising outcomes results in a considerable reduction in information and thus statistical power. We propose an automated procedure for the routine implementation of the distributional method for the dichotomisation of continuous outcomes proposed by Peacock and Sauzet, which retains the precision of the comparison of means.MethodsWe explored the use of a distributional approach to compare differences in proportions based on the comparison of means which retains the power of the latter. We applied this approach to the screening of clinical and biological data as a means to detect 'signals' for potential adverse drug reactions (ADRs). Signals can then be followed-up in further confirmatory studies. Three distributional methods suitable for different types of distributions are described. We propose the use of an automated approach using the observed data to select the most appropriate distribution as an analysis strategy in a RCT setting for multiple continuous outcomes. We illustrate this approach using data from three RCTs assessing the efficacy of mepolizumab in asthma or COPD. Published reference ranges were used to define the proportions of participants with abnormal values for a subset of 10 blood tests. The between-group distributional and empirical differences in proportions were estimated for each blood test and compared.ResultsWithin trials, the distributions varied across the 10 outcomes demonstrating value in a practical approach to selecting the distributional method in the context of multiple adverse event outcomes. Across trials, there were three outcomes where the method chosen by the automated procedure varied for the same outcome. The distributional approach identified three signals (eosinophils, haematocrit, and haemoglobin) compared to only one when using the Fisher's exact test (eosinophils) and two identified by use of the 95% confidence interval for the difference in proportions (eosinophils and potassium).ConclusionWhen dichotomisation of continuous adverse event outcomes aids clinical interpretation, we advocate use of a distributional approach to retain statistical power. Methods are now easy to implement. Retaining information is especially valuable in the context of the analysis of adverse events in RCTs. The routine implementation of this automated approach requires further evaluation.

Project description:BACKGROUND:Statistical methods for the analysis of harm outcomes in randomised controlled trials (RCTs) are rarely used, and there is a reliance on simple approaches to display information such as in frequency tables. We aimed to identify whether any statistical methods had been specifically developed to analyse prespecified secondary harm outcomes and non-specific emerging adverse events (AEs). METHODS:A scoping review was undertaken to identify articles that proposed original methods or the original application of existing methods for the analysis of AEs that aimed to detect potential adverse drug reactions (ADRs) in phase II-IV parallel controlled group trials. Methods where harm outcomes were the (co)-primary outcome were excluded. Information was extracted on methodological characteristics such as: whether the method required the event to be prespecified or could be used to screen emerging events; and whether it was applied to individual events or the overall AE profile. Each statistical method was appraised and a taxonomy was developed for classification. RESULTS:Forty-four eligible articles proposing 73 individual methods were included. A taxonomy was developed and articles were categorised as: visual summary methods (8 articles proposing 20 methods); hypothesis testing methods (11 articles proposing 16 methods); estimation methods (15 articles proposing 24 methods); or methods that provide decision-making probabilities (10 articles proposing 13 methods). Methods were further classified according to whether they required a prespecified event (9 articles proposing 12 methods), or could be applied to emerging events (35 articles proposing 61 methods); and if they were (group) sequential methods (10 articles proposing 12 methods) or methods to perform final/one analyses (34 articles proposing 61 methods). CONCLUSIONS:This review highlighted that a broad range of methods exist for AE analysis. Immediate implementation of some of these could lead to improved inference for AE data in RCTs. For example, a well-designed graphic can be an effective means to communicate complex AE data and methods appropriate for counts, time-to-event data and that avoid dichotomising continuous outcomes can improve efficiencies in analysis. Previous research has shown that adoption of such methods in the scientific press is limited and that strategies to support change are needed. TRIAL REGISTRATION:PROSPERO registration: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=97442.

Project description:BackgroundMeta-analyses of individual-level data from randomised trials are often required to detect clinically worthwhile effects. The Cholesterol Treatment Trialists' Collaboration, which includes data from numerous large long-term statin trials, is conducting a review of the effects of statin therapy on all adverse events collected in those trials. This article describes the approaches used and challenges faced to systematically capture and categorise the data.MethodsProtocols, statistical analysis plans, case report forms, clinical study reports and datasets were obtained, reviewed and checked. Relevant baseline and follow-up data from each trial was then reorganised into standardised formats based upon the Clinical Data Interchange Standards Consortium Study Data Tabulation Model. Adverse event data were organised and coded (automatically or, where necessary, manually) according to a common medical dictionary based upon the Medical Dictionary for Regulatory Activities.ResultsData from 23 double-blind statin trials and 5 open-label statin trials were provided, either through direct data transfer or through online access platforms. Together, these trials provided 845 datasets containing over 38 million records relating to 30,495 study variables and 181,973 randomised participants. Of the 46 Clinical Data Interchange Standards Consortium Study Data Tabulation Model domains that could potentially have been used to organise the data, the 13 most relevant to the project were identified and utilised, including 6 domains related to post-randomisation adverse events. Nearly 1.2 million adverse events were extracted and mapped to over 45,000 unique adverse event terms. Of these adverse events, 99% were coded to a Medical Dictionary for Regulatory Activities 'lower level term', with the remainder coded to a 'higher level term' or, very rarely, only a 'higher level group term'.ConclusionIn this meta-analysis of adverse event data from the large randomised trials of statins, approaches based on common standards for data organisation and classification have provided a resource capable of allowing reliable and rapid evaluation of any previously unknown benefits or hazards of statin therapy.

Project description:BackgroundClinicians and researchers should consider the expected benefits and potential harms of an intervention. Parenting programmes are a widely used evidence-based intervention for child behaviour problems. However, few data are available on potential negative effects. The aims of this paper were to increase systematic knowledge of adverse event (AE) assessment in parenting programmes and to provide an AE assessment tool.MethodsAs part of the RISE project (prevention of child mental health problems in South-eastern Europe-adapt, optimise, test and extend parenting for lifelong health), we developed and tested an AE assessment procedure in three sequential studies for parents of children with child behaviour problems aged 2 to 9 years in North Macedonia, Republic of Moldova, and Romania. This paper reports on the development of the assessment tool in phase 1 (N = 140), phase 2 (N = 835), and the final experiences with using the optimised procedures in phase 3 (multisite randomised controlled trial, N = 823) in which AEs were assessed before, three times during intervention delivery, and at 1 year follow-up. At each time point, the participants completed a 12-item AE checklist. If moderate-to-severe problems of parent or child were reported, a structured follow-up interview was conducted.ResultsThe response rate on the AE assessment tool increased from 6% (phase 1) to 100% (phase 3) indicating improvement in collecting these data based on the experiences of each phase. Results of the RCT (phase 3) showed generally low (S)AE frequencies with the finally optimised procedure: During the intervention, no serious adverse events (SAE) were registered; at least one AE was reported by 10% (after the first session), 7% (after the third session), and 4% (after the last fifth session) of participants. None of the identified (S)AEs was causally related to the study or intervention. Cost-benefit considerations are needed to determine the best way to ensure participant safety in parenting programmes.ConclusionThe applied active AE assessment procedure provides a comprehensive AE assessment tool that can be used by others-with adaptations for the specific context, if needed. Based on our experiences, we outline recommendations for future studies.Trial registrationClinicalTrials.gov, registration number phase 1: NCT03552250; phase 2: NCT03865485, phase 3: NCT04721730 . Registered on 13 January 2021.

Project description:ObjectiveTo ascertain contemporary approaches to the collection, reporting and analysis of adverse events (AEs) in randomised controlled trials (RCTs) with a primary efficacy outcome.DesignA review of clinical trials of drug interventions from four high impact medical journals.Data sourcesElectronic contents table of the BMJ, the Journal of the American Medical Association (JAMA), the Lancet and the New England Journal of Medicine (NEJM) were searched for reports of original RCTs published between September 2015 and September 2016.MethodsA prepiloted checklist was used and single data extraction was performed by three reviewers with independent check of a randomly sampled subset to verify quality. We extracted data on collection methods, assessment of severity and causality, reporting criteria, analysis methods and presentation of AE data.ResultsWe identified 184 eligible reports (BMJ n=3; JAMA n=38, Lancet n=62 and NEJM n=81). Sixty-two per cent reported some form of spontaneous AE collection but only 29% included details of specific prompts used to ascertain AE data. Numbers that withdrew from the trial were well reported (80%), however only 35% of these reported whether withdrawals were due to AEs. Results presented and analysis performed was predominantly on 'patients with at least one event' with 84% of studies ignoring repeated events. Despite a lack of power to undertake formal hypothesis testing, 47% performed such tests for binary outcomes.ConclusionsThis review highlighted that the collection, reporting and analysis of AE data in clinical trials is inconsistent and RCTs as a source of safety data are underused. Areas to improve include reducing information loss when analysing at patient level and inappropriate practice of underpowered multiple hypothesis testing. Implementation of standard reporting practices could enable a more accurate synthesis of safety data and development of guidance for statistical methodology to assess causality of AEs could facilitate better statistical practice.

Project description:IntroductionWheeze is an important sign indicating a potentially severe adverse event in vaccine and drug trials, particularly in children. However, there are currently no consensus definitions of wheeze or associated respiratory compromise in randomized controlled trials (RCTs).ObjectiveTo identify definitions and severity grading scales of wheeze as an adverse event in vaccine and drug RCTs enrolling children <5 years and to determine their diagnostic performance based on sensitivity, specificity and inter-observer agreement.MethodsWe performed a systematic review of electronic databases and reference lists with restrictions for trial settings, English language and publication date ≥1970. Wheeze definitions and severity grading were abstracted and ranked by a diagnostic certainty score based on sensitivity, specificity and inter-observer agreement.ResultsOf 1205 articles identified using our broad search terms, we identified 58 eligible trials conducted in 38 countries, mainly in high-income settings. Vaccines made up the majority (90%) of interventions, particularly influenza vaccines (65%). Only 15 trials provided explicit definitions of wheeze. Of 24 studies that described severity, 11 described wheeze severity in the context of an explicit wheeze definition. The remaining 13 studies described wheeze severity where wheeze was defined as part of a respiratory illness or a wheeze equivalent. Wheeze descriptions were elicited from caregiver reports (14%), physical examination by a health worker (45%) or a combination (41%). There were 21/58 studies in which wheeze definitions included combined caregiver report and healthcare worker assessment. The use of these two methods appeared to have the highest combined sensitivity and specificity.ConclusionStandardized wheeze definitions and severity grading scales for use in pediatric vaccine or drug trials are lacking. Standardized definitions of wheeze are needed for assessment of possible adverse events as new vaccines and drugs are evaluated.

Project description:IntroductionAdverse events (AE) in care are recognized as a leading cause of mortality and injury in patients. Improving patients' safety is difficult to achieve. Therefore, innovative research strategies are needed to identify errors in subgroups of patients and related severity of outcomes as well as reliably measured efficiency of reproducible strategies to improve safety. This trial aims to evaluate the impact of a combined multiprofessional education program on the rate of AE in neonatal intensive care units (NICUs).Methods and analysisThis is a stepped-wedge cluster randomised controlled trial with 3 clusters each containing 4 units. The study time period will be 20 months. The education program will be implemented within each cluster following a random sequence with a control period, a 4-month transition period and a post-educational intervention period. Eligibility criteria: for clusters: 6 NICUs from Ile-de-France and 6 NICUs from different regions in France; for patients: in-hospital during the study period (November 23, 2015 and November 2, 2017 [inclusion start dates varying by unit]) in one of the 12 NICUs; corrected gestational age ≤42 weeks upon admission; hospitalization period >2 days; and parents informed and not opposed to the use of their newborn's data. A routine occurrence reporting of medical errors and their consequence will take place during the entire study period. The intervention will combine an education to implement a standardized root cause analysis method, creation of bundles (insertion, daily goals, maintenance bundles) to prevent catheter-associated blood-stream infection and a poster to prevent extravasation injuries.OutcomeWe hypothesize a reduction from 60 (control) to 50 (intervention) AE/1000 patient-days. The primary outcome will be the rate of AE/1000 patient-days in the NICU.Trial registration numberNCT02598609, trial registered November 6, 2015. https://clinicaltrials.gov/ct2/show/NCT02598609.Ethics and disseminationStudy approved by the regional ethic committee CPP Ile-de-France III (no 2014-A01751-46). The results will be published in peer-reviewed journals.

Project description:ObjectivesTo gain a better understanding of current adverse event (AE) analysis practices and the reasons for the lack of use of sophisticated statistical methods for AE data analysis in randomised controlled trials (RCTs), with the aim of identifying priorities and solutions to improve practice.DesignA cross-sectional, online survey of statisticians working in clinical trials, followed up with a workshop of senior statisticians working across the UK.ParticipantsWe aimed to recruit into the survey a minimum of one statistician from each of the 51 UK Clinical Research Collaboration registered clinical trial units (CTUs) and industry statisticians from both pharmaceuticals and clinical research organisations.OutcomesTo gain a better understanding of current AE analysis practices, measure awareness of specialist methods for AE analysis and explore priorities, concerns and barriers when analysing AEs.ResultsThirty-eight (38/51; 75%) CTUs, 5 (5/7; 71%) industry and 21 attendees at the 2019 Promoting Statistical Insights Conference participated in the survey. Of the 64 participants that took part, 46 participants were classified as public sector participants and 18 as industry participants. Participants indicated that they predominantly (80%) rely on subjective comparisons when comparing AEs between treatment groups. Thirty-eight per cent were aware of specialist methods for AE analysis, but only 13% had undertaken such analyses. All participants believed guidance on appropriate AE analysis and 97% thought training specifically for AE analysis is needed. These were both endorsed as solutions by workshop participants.ConclusionsThis research supports our earlier work that identified suboptimal AE analysis practices in RCTs and confirms the underuse of more sophisticated AE analysis approaches. Improvements are needed, and further research in this area is required to identify appropriate statistical methods. This research provides a unanimous call for the development of guidance, as well as training on suitable methods for AE analysis to support change.