Dataset Information

Enabling Data-Driven Clinical Quality Assurance: Predicting Adverse Event Reporting in Clinical Trials Using Machine Learning.

ABSTRACT: INTRODUCTION:Adverse event (AE) under-reporting has been a recurrent issue raised during health authorities Good Clinical Practices (GCP) inspections and audits. Moreover, safety under-reporting poses a risk to patient safety and data integrity. The current clinical Quality Assurance (QA) practices used to detect AE under-reporting rely heavily on investigator site and study audits. Yet several sponsors and institutions have had repeated findings related to safety reporting, and this has led to delays in regulatory submissions. Recent developments in data management and IT systems allow data scientists to apply techniques such as machine learning to detect AE under-reporting in an automated fashion. OBJECTIVE:In this project, we developed a predictive model that enables Roche/Genentech Quality Program Leads oversight of AE reporting at the program, study, site, and patient level. This project was part of a broader effort at Roche/Genentech Product Development Quality to apply advanced analytics to augment and complement traditional clinical QA approaches. METHOD:We used a curated data set from 104 completed Roche/Genentech sponsored clinical studies to train a machine learning model to predict the expected number of AEs. Our final model used 54 features built on patient (e.g., demographics, vitals) and study attributes (e.g., molecule class, disease area). RESULTS:In order to evaluate model performance, we tested how well it would detect simulated test cases based on data not used for model training. For relevant simulation scenarios of 25%, 50%, and 75% under-reporting on the site level, our model scored an area under the curve (AUC) of the receiver operating characteristic (ROC) curve of 0.62, 0.79, and 0.92, respectively. CONCLUSION:The model has been deployed to evaluate safety reporting performance in a set of ongoing studies in the form of a QA/dashboard cockpit available to Roche Quality Program Leads. Applicability and production performance will be assessed over the next 12-24 months in which we will develop a validation strategy to fully integrate our model into Roche QA processes.

SUBMITTER: Menard T

PROVIDER: S-EPMC6689279 | biostudies-other | 2019 Sep

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Enabling Data-Driven Clinical Quality Assurance: Predicting Adverse Event Reporting in Clinical Trials Using Machine Learning.

Ménard Timothé T Barmaz Yves Y Koneswarakantha Björn B Bowling Rich R Popko Leszek L

Drug safety 20190901 9

<h4>Introduction</h4>Adverse event (AE) under-reporting has been a recurrent issue raised during health authorities Good Clinical Practices (GCP) inspections and audits. Moreover, safety under-reporting poses a risk to patient safety and data integrity. The current clinical Quality Assurance (QA) practices used to detect AE under-reporting rely heavily on investigator site and study audits. Yet several sponsors and institutions have had repeated findings related to safety reporting, and this has ...[more]

PMID: 31123940

Similar Datasets

Project description:BackgroundCurrent adverse event reporting practices do not document longitudinal characteristics of adverse effects, and alternative methods are not easily interpretable and have not been employed by clinical trials. Introducing time parameters in the evaluation of safety that are comprehensive yet easily interpretable could allow for a better understanding of treatment quality. In this study, we developed and applied a novel adverse event reporting method based on longitudinal adverse event changes to aid describing, summarizing, and presenting adverse event profile. We termed it the "Adverse Event Load, Onset, and Maximum Grade" method.MethodsWe developed two adverse event summary metrics to complement the traditional maximum grade report. Onset time indicates the time period in which the maximum grade for a specific adverse event occurred and was defined as "early" (i.e. maximum grade happened for the first time before 6 weeks) or "late" (i.e. after the 6th week). Adverse event load indicates the overall severity of a specific adverse event over the entire treatment. Higher adverse event load indicates a worse overall experience. These metrics can be calculated for adverse events with different maximum grades, in treatments with planned changes (e.g. dosage changes), using data sets with different number of adverse event data points between treatments (e.g. treatments with longer cycle lengths may have less adverse event data points) and on data sets with different adverse event data availability (e.g. cycle basis and patient-outcome reports). We tested the utility of this method using individual patient data from two major backbone therapies ("Irinotecan" and "Oxaliplatin") from the N9741 trial available in the Fondation ARCAD database (fondationarcad.org). We investigated profiles of diarrhea, neutropenia/leukopenia, and nausea/vomiting.ResultsOur method provided additional information compared to traditional adverse event reports. For example, for nausea/vomiting, while patients in Irinotecan had a higher risk of experiencing maximum grade 3-4 (15.6% vs 7.6%, respectively; p < 0.001), patients in both groups experienced similar severity over time (adverse effect load = 0.102 and 0.096, respectively; p = 0.26), suggesting that patients in Oxaliplatin experienced a lower-grade but more persistent nausea/vomiting. For neutropenia/leukopenia, more patients in Irinotecan experienced their maximum grade for the first time early in the treatment compared to patients in Oxaliplatin (67.9% vs 41.7%; p < 0.001), regardless of maximum grade. Longitudinal information can help compare treatments or guide clinicians on choosing appropriate interventions for low-grade but persistent adverse event or early adverse event onset.ConclusionWe developed an adverse event reporting method that provides clinically relevant information about treatment toxicity by incorporating two longitudinal adverse event metrics to the traditional maximum grade approach. Future research should establish clinical benchmarks for metrics included in this adverse event reporting method.

Project description:Study Design?Systematic review. Objectives?(1) To compare the quality of adverse event (AE) methodology and reporting among randomized trials comparing lumbar fusion with lumbar total disk replacement (TDR) using established AE reporting systems; (2) to compare the AEs and reoperations of lumbar spinal fusion with those from lumbar TDR; (3) to make recommendations on how to report AEs in randomized controlled trials (RCTs) so that surgeons and patients have more-detailed and comprehensive information when making treatment decisions. Methods?A systematic search of PubMed, the Cochrane collaboration database, and the National Guideline Clearinghouse through May 2015 was conducted. Randomized controlled trials with at least 2 years of follow-up comparing lumbar artificial disk replacement with lumbar fusion were included. Patients were required to have axial or mechanical low back pain of ?3 months' duration due to degenerative joint disease defined as degenerative disk disease, facet joint disease, or spondylosis. Outcomes included the quality of AE acquisition methodology and results reporting, and AEs were defined as those secondary to the procedure and reoperations. Individual and pooled relative risks and their 95% confidence intervals comparing lumbar TDR with fusion were calculated. Results?RCTs demonstrated a generally poor description of methods for assessing AEs. There was a consistent lack of clear definition or grading for these events. Furthermore, there was a high degree of variation in reporting of surgery-related AEs. Most studies lacked adequate reporting of the timing of AEs, and there were no clear distinctions between acute or chronic AEs. Meta-analysis of the pooled data demonstrated a twofold increased risk of AEs in patients having lumbar fusion compared with patients having lumbar TDR at 2-year follow-up, and this relative risk was maintained at 5 years. Furthermore, the pooled data demonstrated a 1.7 times greater relative risk of reoperation in the fusion group compared with lumbar TDR, although this risk decreased to 1.1 at 5-year follow-up. However, given the lack of quality and consistency in the methods of recording and reporting of AEs, we are unable to make a clear recommendation of one treatment over the other. Conclusions?Based on the currently available literature, lumbar TDR appears to be comparable in safety to lumbar fusion. However, due to lack of consistency in reporting of AEs, it is difficult to make conclusions regarding the true safety profile of lumbar TDR. Standardization in AE reporting will significantly improve the reliability of the current literature.

Dataset Information

Enabling Data-Driven Clinical Quality Assurance: Predicting Adverse Event Reporting in Clinical Trials Using Machine Learning.

Publications

Enabling Data-Driven Clinical Quality Assurance: Predicting Adverse Event Reporting in Clinical Trials Using Machine Learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets