Bayesian Modeling for the Detection of Adverse Events Underreporting in Clinical Trials.
Ontology highlight
ABSTRACT: Introduction
Safety underreporting is a recurrent issue in clinical trials that can impact patient safety and data integrity. Clinical quality assurance (QA) practices used to detect underreporting rely on on-site audits; however, adverse events (AEs) underreporting remains a recurrent issue. In a recent project, we developed a predictive model that enables oversight of AE reporting for clinical quality program leads (QPLs). However, there were limitations to using solely a machine learning model.Objective
Our primary objective was to propose a robust method to compute the probability of AE underreporting that could complement our machine learning model. Our model was developed to enhance patients' safety while reducing the need for on-site and manual QA activities in clinical trials.Methods
We used a Bayesian hierarchical model to estimate the site reporting rates and assess the risk of underreporting. We designed the model with Project Data Sphere clinical trial data that are public and anonymized.Results
We built a model that infers the site reporting behavior from patient-level observations and compares them across a study to enable a robust detection of outliers between clinical sites.Conclusion
The new model will be integrated into the current dashboard designed for clinical QPLs. This approach reduces the need for on-site audits, shifting focus from source data verification to pre-identified, higher risk areas. It will enhance further QA activities for safety reporting from clinical trials and generate quality evidence during pre-approval inspections.
SUBMITTER: Barmaz Y
PROVIDER: S-EPMC8278191 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundClinician reporting of symptomatic adverse events (AEs) in phase I trials uses the Common Terminology Criteria for Adverse Events (CTCAE). The utility of the patient-reported outcomes (PROs) version of the CTCAE (PRO-CTCAE) in this setting is unknown. This prospective, observational study compared patient- and clinician-reported symptomatic AEs in phase I patients.MethodsPhase I study-eligible patients at Princess Margaret were surveyed with the PRO-CTCAE full-item library (78 symptomatic AEs) at baseline (BL), mid-cycle 1, and mid-cycle 2 (C2). Patient and trial characteristics, best response, and survival data were collected. Presence or absence of patient- (PRO-CTCAE) or clinician-reported symptomatic AEs were compared (kappa) at defined timepoints and overall (BL+ mid-cycle 1 + C2).ResultsOf 292 patients approached from May 2017 to January 2019, a total of 265 (90.8%) were consented, with 243 (91.7%) evaluable and 552 PRO-CTCAE surveys (completion rate = 98.7%) included in analyses. Evaluation of overall patient-reported symptomatic AEs identified 50 PRO-CTCAE and 11 CTCAE items with 10% or greater reporting frequency. Nineteen CTCAE items were reported as 1% or less despite matched PRO-CTCAE items reporting as 10% or greater. Underreported categories included sexual health, bodily emissions, and cognition. Clinician- relative to patient-reporting frequency (ratio) demonstrated 9 symptomatic AEs with a 50-fold or more lower clinician reporting rate. Overall patient-clinician agreement for individual symptomatic AEs ranged from poor (κ = 0.00-0.19) to moderate (κ = 0.40-0.59), with discordance driven by lack of clinician reporting. Dyspnea (κ = 0.54) and peripheral neuropathy (κ = 0.63) at BL and limb edema (κ = 0.55) at C2 demonstrated the highest patient-clinician agreement.ConclusionsPoor to moderate patient-clinician agreement for symptomatic AEs suggests clinician underreporting in phase I trials. Analyses of severity and interference PRO categories are ongoing.
| S-EPMC8502480 | biostudies-literature
Project description:Background/aimsRegulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events.MethodsIn this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes.ResultsOur simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals.ConclusionIn the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.
| S-EPMC8041270 | biostudies-literature
Project description:It is an important and yet challenging task to identify true signals from many adverse events that may be reported during the course of a clinical trial. One unique feature of drug safety data from clinical trials, unlike data from post-marketing spontaneous reporting, is that many types of adverse events are reported by only very few patients leading to rare events. Due to the limited study size, the p-values of testing whether the rate is higher in the treatment group across all types of adverse events are in general not uniformly distributed under the null hypothesis that there is no difference between the treatment group and the placebo group. A consequence is that typically fewer than 100α percent of the hypotheses are rejected under the null at the nominal significance level of α . The other challenge is multiplicity control. Adverse events from the same body system may be correlated. There may also be correlations between adverse events from different body systems. To tackle these challenging issues, we develop Monte-Carlo-based methods for the signal identification from patient-reported adverse events in clinical trials. The proposed methodologies account for the rare events and arbitrary correlation structures among adverse events within and/or between body systems. Extensive simulation studies demonstrate that the proposed method can accurately control the family-wise error rate and is more powerful than existing methods under many practical situations. Application to two real examples is provided.
| S-EPMC6661211 | biostudies-literature
Project description:Purpose: Research biopsies are frequently incorporated within clinical trials in oncology and are often a mandatory requirement for trial enrollment. However, limited information is available regarding the extent and completeness of research biopsy reporting.Experimental Design: We identified a cohort of therapeutic clinical trials where research biopsies were performed between January 2005 and October 2010 from an IR database at our institution. Clinical trial protocols were compared with the highest level of corresponding publication as a manuscript or registry report.Results: A total of 866 research biopsies were performed across 46 clinical trials, with a median of 8 patients biopsied/trial and 19 biopsies collected/trial. After a median follow-up time of 4.3 years from trial completion, 36 of 46 trials (78%) reported trial results: published manuscripts (n = 35), or registry report (n = 1). A total of 635 conducted biopsies were reported in 18 of the 46 trials (39%). Six (33%) of these 18 trials underreported the number of biopsies performed. Of 33 trials with mandatory research biopsies, 13 (39%) trials reported on these biopsies. Biopsy complications occurred in 8 trials [n = 39 patients, 6 grade 3/4 adverse events (AE)] but only 1 trial reported these. Factors associated with biopsy reporting included a larger number of biopsies (P ≤ 0.001) and serial biopsies (P < 0.001). Twelve of 16 (75%) trials with >12 biopsies performed reported on these biopsies compared with only 20% (6/30) that performed ≤12 biopsies.Conclusions: Despite ethical obligations to report research biopsies, the majority (61%) of trials do not report results from research biopsies. Complications are rarely reported in these studies. Improved reporting of results and AEs from research biopsies is needed. Clin Cancer Res; 23(21); 6450-7. ©2017 AACR.
| S-EPMC5668146 | biostudies-literature
Project description:Adverse events (AEs) derived from nonspecific activity of treatments can impair the validity of trials, and even make it difficult to identify specific AEs associated with treatments. To better understand these nonspecific AEs, we investigated the AEs in placebo groups by using knee osteoarthritis clinical trials.Randomized, placebo-controlled, knee osteoarthritis trials were identified by searching electronic databases. We determined the rate of patients with AEs and the rate of dropouts caused by AEs in the active and placebo groups. Furthermore, we calculated the rate of patients for individual AEs in the placebo groups. Finally, we performed secondary analyses to identify the factors associated with these rates.Overall, 272 papers reporting 281 trials were included in the analysis. The rates of patients with AEs were 31.8% in the active groups and 27.4% in the placebo groups. The rate of the placebo groups accounted for 86.2% of the rate of the active groups. The rates of dropouts caused by AEs were 5.2% in the active groups and 4.8% in the placebo groups. The rate of the placebo groups accounted for 92.3% of the rate of the active groups. AEs in the placebo groups included a number of clinical conditions, with elevated alanine aminotransferase (0.59%; 95% CI: 0.46 to 0.77) being the most common objective outcome and headache (4.48%; 95% CI: 4.20 to 4.79) being the most frequent subjective outcome. The rate of patients with AEs and the rate of dropouts caused by AEs were associated with the treatment type, delivery route, and study design.The nonspecific AEs substantially accounted for the development of AEs in the active groups and included conditions involving the entire body.
| S-EPMC4218813 | biostudies-literature
Project description:BACKGROUND: Misclassification of adverse events in clinical trials can sometimes have serious consequences. Therefore, each of the many steps involved, from a patient's adverse experience to presentation in tables in publications, should be as standardised as possible, minimising the scope for interpretation. Adverse events are categorised by a predefined dictionary, e.g. MedDRA, which is updated biannually with many new categories. The objective of this paper is to study interobserver variation and other challenges of coding. METHODS: Systematic review using PRISMA. We searched PubMed, EMBASE and The Cochrane Library. All studies were screened for eligibility by two authors. RESULTS: Our search returned 520 unique studies of which 12 were included. Only one study investigated interobserver variation. It reported that 12% of the codes were evaluated differently by two coders. Independent physicians found that 8% of all the codes deviated from the original description. Other studies found that product summaries could be greatly affected by the choice of dictionary. With the introduction of MedDRA, it seems to have become harder to identify adverse events statistically because each code is divided in subgroups. To account for this, lumping techniques have been developed but are rarely used, and guidance on when to use them is vague. An additional challenge is that adverse events are censored if they already occurred in the run-in period of a trial. As there are more than 26 ways of determining whether an event has already occurred, this can lead to bias, particularly because data analysis is rarely performed blindly. CONCLUSION: There is a lack of evidence that coding of adverse events is a reliable, unbiased and reproducible process. The increase in categories has made detecting adverse events harder, potentially compromising safety. It is crucial that readers of medical publications are aware of these challenges. Comprehensive interobserver studies are needed.
| S-EPMC3401103 | biostudies-literature
Project description:This article reviews ocular adverse events (AEs) reported in association with administration of antibody-drug conjugates (ADCs) in human clinical trials. References reporting ocular toxicity or AEs associated with ADCs were collected using online publication searches. Articles, abstracts, or citations were included if they cited ocular toxicities or vision-impairing AEs with a confirmed or suspected association with ADC administration. Twenty-two references were found citing ocular or vision-impairing AEs in association with ADC administration. All references reported use of ADCs in human clinical trials for treatment of various malignancies. The molecular target and cytotoxic agent varied depending on the ADC used. Ocular AEs affected a diversity of ocular tissues. The most commonly reported AEs involved the ocular surface and included blurred vision, dry eye, and corneal abnormalities (including microcystic corneal disease). Most ocular AEs were not severe (? grade 2) or dose limiting. Clinical outcomes were not consistently reported, but when specified, most AEs improved or resolved with cessation of treatment or with ameliorative therapy. A diverse range of ocular AEs are reported in association with administration of ADCs for the treatment of cancer. The toxicologic mechanism(s) and pathogenesis of such events are not well understood, but most are mild in severity and reversible. Drug development and medical professionals should be aware of the clinical features of these events to facilitate early recognition and intervention in the assessment of preclinical development programs and in human clinical trials.
| S-EPMC4677113 | biostudies-literature
Project description:BACKGROUND AND PURPOSE:Research on stem cells (SC) is growing rapidly in neurology, but clinical applications of SC for neurological disorders remain to be proven effective and safe. Human clinical trials need to be registered in registries in order to reduce publication bias and selective reporting. METHODS:We searched three databases-clinicaltrials.gov, the Clinical Research Information System (CRIS), and PubMed-for neurologically relevant SC-based human trials and articles in Korea. The registration of trials, posting and publication of results, and registration of published SC articles were examined. RESULTS:There were 17 completed trials registered at clinicaltrials.gov and the CRIS website, with results articles having been published for 5 of them. Our study found 16 publications, of which 1 was a review article, 1 was a protocol article, and 8 contained registered trial information. CONCLUSIONS:Many registered SC trials related to neurological disorders are not reported, while many SC-related publications are not registered in a public registry. These results support the presence of biased reporting and publication bias in SC trials related to neurological disorders in Korea.
| S-EPMC5897206 | biostudies-literature
Project description:Bayesian inference of distinct DNA methylation patterns (epialleles) in RRBS bisulfite sequencing data.
| PRJEB21102 | ENA
Project description:Introduction: Improving adverse drug event (ADE) detection is important for post-marketing drug safety surveillance. Existing statistical approaches can be further optimized owing to their high efficiency and low cost. Objective: The objective of this study was to evaluate the proposed approach for use in pharmacovigilance, the early detection of potential ADEs, and the improvement of drug safety. Methods: We developed a novel integrated approach, the Bayesian signal detection algorithm, based on the pharmacological network model (ICPNM) using the FDA Adverse Event Reporting System (FAERS) data published from 2004 to 2009 and from 2014 to 2019Q2, PubChem, and DrugBank database. First, we used a pharmacological network model to generate the probabilities for drug-ADE associations, which comprised the proper prior information component (IC). We then defined the probability of the propensity score adjustment based on a logistic regression model to control for the confounding bias. Finally, we chose the Side Effect Resource (SIDER) and the Observational Medical Outcomes Partnership (OMOP) data to evaluate the detection performance and robustness of the ICPNM compared with the statistical approaches [disproportionality analysis (DPA)] by using the area under the receiver operator characteristics curve (AUC) and Youden's index. Results: Of the statistical approaches implemented, the ICPNM showed the best performance (AUC, 0.8291; Youden's index, 0.5836). Meanwhile, the AUCs of the IC, EBGM, ROR, and PRR were 0.7343, 0.7231, 0.6828, and 0.6721, respectively. Conclusion: The proposed ICPNM combined the strengths of the pharmacological network model and the Bayesian signal detection algorithm and performed better in detecting true drug-ADE associations. It also detected newer ADE signals than a DPA and may be complementary to the existing statistical approaches.
| S-EPMC8762263 | biostudies-literature