Dataset Information

Reporting of harms in oncological clinical study reports submitted to the European Medicines Agency compared to trial registries and publications-a methodological review.

ABSTRACT:

Background

An accurate and comprehensive assessment of harms is a fundamental part of an accurate weighing of benefits and harms of an intervention when making treatment decisions; however, harms are known to be underreported in journal publications. Therefore, we sought to compare the completeness of reporting of harm data, discrepancies in harm data reported, and the delay to access results of oncological clinical trials between three sources: clinical study reports (CSRs), clinical trial registries and journal publications.

Methods

We used the EMA clinical data website to identify all trials submitted to the EMA between 2015 and 2018. We retrieved all CSRs and included all phase II, II/III or III randomised controlled trials (RCTs) assessing targeted therapy and immunotherapy for cancer. We then identified related records in clinical trial registries and journals. We extracted harms data for eight pre-specified variables and determined the completeness of reporting of harm data in each of the three sources.

Results

We identified 42 RCTs evaluating 13 different drugs. Results were available on the EMA website in CSRs for 37 (88%) RCTs, ClinicalTrials.gov for 36 (86%), the European Clinical Trials Register (EUCTR) for 20 (48%) and in journal publications for 32 (76%). Harms reporting was more complete in CSRs than other sources. We identified marked discrepancies in harms data between sources, e.g. the number of patients discontinuing due to adverse events differed in CSRs and clinical trial registers for 88% of trials with data in both sources. For CSRs and publications, the corresponding number was 90%. The median (interquartile range) delay between the primary trial completion date and access to results was 4.34 (3.09-7.22) years for CSRs, 2.94 (1.16-4.52) years for ClinicalTrials.gov, 5.39 (4.18-7.33) years for EUCTR and 2.15 (0.64-5.04) years for publications.

Conclusions

Harms of recently approved oncological drugs were reported more frequently and in more detail in CSRs than in trial registries and journal publications. Systematic reviews seeking to address harms of oncological treatments should ideally use CSRs as the primary source of data; however, due to problems with access, this is currently not feasible.

SUBMITTER: Paludan-Muller AS

PROVIDER: S-EPMC8028762 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo determine, using research on duloxetine for major depressive disorder as an example, if there are inconsistencies between protocols, clinical study reports, and main publicly available sources (journal articles and trial registries), and within clinical study reports themselves, with respect to benefits and major harms.DesignData on primary efficacy analysis and major harms extracted from each data source and compared.SettingNine randomised placebo controlled trials of duloxetine (total 2878 patients) submitted to the European Medicines Agency (EMA) for marketing approval for major depressive disorder.Data sourcesClinical study reports, including protocols as appendices (total 13,729 pages), were obtained from the EMA in May 2011. Journal articles were identified through relevant literature databases and contacting the manufacturer, Eli Lilly. Clinicaltrials.gov and the manufacturer's online clinical trial registry were searched for trial results.ResultsClinical study reports fully described the primary efficacy analysis and major harms (deaths (including suicides), suicide attempts, serious adverse events, and discontinuations because of adverse events). There were minor inconsistencies in the population in the primary efficacy analysis between the protocol and clinical study report and within the clinical study report for one trial. Furthermore, we found contradictory information within the reports for seven serious adverse events and eight adverse events that led to discontinuation but with no apparent bias. In each trial, a median of 406 (range 177-645) and 166 (100-241) treatment emergent adverse events (adverse events that emerged or worsened after study drug was started) in the randomised phase were not reported in journal articles and Lilly trial registry reports, respectively. We also found publication bias in relation to beneficial effects.ConclusionClinical study reports contained extensive data on major harms that were unavailable in journal articles and in trial registry reports. There were inconsistencies between protocols and clinical study reports and within clinical study reports. Clinical study reports should be used as the data source for systematic reviews of drugs, but they should first be checked against protocols and within themselves for accuracy and consistency.

Project description:Objectives: We aimed to study the prevalence of selective reporting of primary and secondary outcomes in tinnitus trials and to examine if selective reporting of outcome measures is influenced by the nature and direction of its results. Background: Selective reporting of outcome measures has been reported in several biomedical fields and can influence the clinical usefulness and implementation of outcomes of clinical trials. It is reported as one of the obstacles in finding an effective intervention for tinnitus. Methods: ClinicalTrials.gov (CT.gov) was used to identify all registered interventional tinnitus trials up to December 2015. A standardized search was used to find corresponding publications up to March 2018. The prespecified outcomes in CT.gov were compared with the outcomes reported in corresponding publication(s). The effects of the (lack of) statistical significance of trial results and the effects of funding source on record adherence were evaluated. Changes in registration elements were assessed with the Archive site of CT.gov. Results: We found corresponding publications for 60 (64.5%) of 93 eligible tinnitus trials registered in CT.gov. Of all the publications, five (7.5%) fully reported outcome measures entirely in line with the prespecified outcome measures. Discrepancies between the prespecified and reported outcomes were found in a total of 51 (76.1%) of the studies for primary outcomes, whereas 62 (92.5%) of the studies had discrepancies in secondary outcomes. In secondary outcomes, statistical significance of trial results influenced CT.gov record adherence. In addition, there was a statistically significant difference in the rate of discrepancy in industry-funded [n = 98 (87.5%) discrepant outcomes] and non-industry funded trials [n = 172 (74.5%) discrepant outcomes] (p = 0.01). Finally, 15 (25.9%) trialists made modifications in registered outcome measures during or after the trial period. Conclusion: Tinnitus trials suffer from substantial outcome reporting bias. Awareness of its presence must be raised to limit the obstacles of finding an effective intervention for tinnitus.

Project description:BACKGROUND:Clinical study reports (CSRs) have been increasingly utilised within academic research in recent years. European Medicines Agency (EMA) Policy 0070 'Phase 1,' which came into effect in January 2015, requires the publication of regulatory documents such as CSRs from central applications in an anonymised format. EMA Policy 0070 requires sponsors to demonstrate careful consideration of data utility within anonymised CSRs published within the scope of the policy, yet the concept of data utility is not clearly defined in the associated anonymisation guidance. OBJECTIVE:To review the use of data from CSRs in published academic research and to hypothesise the potential data utility of CSRs anonymised under the objectives of EMA Policy 0070 for future academic research. METHODS:Review of the objectives, research methodologies and findings of academic research reports using unpublished data from CSRs (prior to EMA Policy 0070). Semi-structured interviews with authors of academic research reports, including questions related to data utility of anonymised CSRs published under EMA Policy 0070. RESULTS:Thirteen academic research reports were identified and reviewed. The research purposes ranged from assessment of reporting bias, comparison of methods and results with published data sources, detailed evaluation of harms and adverse events, re-analysis and novel analyses including systematic reviews and meta-analysis. All of the examples identified required access to the methods and results sections of CSRs (including aggregated summary tables) and research purposes relating to evaluation of adverse events also required access to participant narratives. Retaining anonymised participant narratives relating to interventions, findings and events, while maintaining an acceptably low risk of participant re-identification, may provide an important gain in data utility and further understanding of drug safety profiles. CONCLUSIONS:This work provides an initial insight into the previous use of CSR data and current practices for including regulatory data in academic research. This work also provides early guidance to qualitatively assess and document data utility within anonymised CSRs published under EMA Policy 0070.

Project description:IntroductionStudies measuring the effectiveness of risk minimization measures (RMMs) submitted by pharmaceutical companies to the European Medicines Agency are part of the post-authorization regulatory requirements and represent an important source of data covering a range of medicinal products and safety-related issues. Their objectives, design, and the associated regulatory outcomes were reviewed, and conclusions were drawn that may support future progress in risk minimization evaluation.MethodsInformation was obtained from risk management plans, study protocols, clinical study reports, and assessment reports of 157 medicinal products authorized for cardiovascular, endocrinology, and metabolic indications. We selected observational studies measuring, as outcomes of interest, the relationship between the RMMs in place and (1) implementation measures, such as clinical knowledge or physicians` compliance to recommendations contained in the RMMs; and (2) occurrence or reduced severity of the adverse drug reactions for which the RMMs were required.ResultsOf 59 eligible studies (24 completed, 35 ongoing), 44 assessed implementation measures, whereas only 15 assessed safety outcomes (1 study as a single endpoint and 14 studies with other endpoints). Fifty-one studies used non-experimental designs and 25 studies employed electronic healthcare databases for analysis. Of the 24 completed studies, 17 were considered satisfactory and supported immediate regulatory decision making, 6 were considered inconclusive and required new evaluations, and 1 was terminated early because new safety restrictions were required, thereby necessitating a new evaluation. Compliance with agreed deadlines was considered acceptable in 21 of 24 completed studies; the average time for a submission was 37 months (standard deviation ± 17), with differences observed by type of data source employed.ConclusionsThree important gaps in the evaluation plans of RMMs were identified: lack of early feedback on implementation, limited evaluation of safety outcomes, and inability to provide information on the effectiveness from an integrated measurement of different elements of a set of risk minimization tools. More robust evidence is needed to advance regulatory science and support more rapid adjustment of risk minimization strategies as needed.

Project description:AimsTo systematically assess the level of evidence for psychotropic drugs approved by the European Medicines Agency (EMA).MethodsCross-sectional analysis of all European Public Assessment Reports (EPARs) and meta-analyses of the many studies reported in these EPARs. Eligible EPARs were identified from the EMA's website and individual study reports were requested from the Agency when necessary. All marketing authorisation applications (defined by the drug, the route of administration and given indications) for psychotropic medications for adults (including drugs used in psychiatry and addictology) were considered. EPARs solely based on bioequivalence studies were excluded. Our primary outcome measure was the presence of robust evidence of comparative effectiveness, defined as at least two 'positive' superiority studies against an active comparator. Various other features of the approvals were assessed, such as evidence of non-inferiority v. active comparator and superiority v. placebo. For studies with available data, effect sizes were computed and pooled using a random effect meta-analysis for each dose of each drug in each indication.ResultsTwenty-seven marketing authorisations were identified. For one, comparative effectiveness was explicitly considered as not needed in the EPAR. Of those remaining, 21/26 (81%) did not provide any evidence of superiority against an active comparator, 2/26 (8%) were based on at least two trials showing superiority against active comparator and three (11%) were based on one positive trial; 1/26 provided evidence for two positive non-inferiority analyses v. active comparator and seven (26%) provided evidence for one. In total, 20/27 (74%) evaluations reported evidence of superiority v. placebo with two or more trials. Among the meta-analyses of initiation studies against active comparator (57 available comparisons), the median effect size was 0.051 (range -0.503; 0.318). Twenty approved evaluations (74%) reported evidence of superiority v. placebo on the basis of two or more initiation trials and seven based on a single trial. Among meta-analyses of initiation studies against placebo (125 available comparisons), the median effect size was -0.283 (range -0.820; 0.091). Importantly, among the 89 study reports requested on the EMA website, only 19 were made available 1 year after our requests.ConclusionsThe evidence for psychiatric drug approved by the EMA was in general poor. Small to modest effects v. placebo were considered sufficient in indications where an earlier drug exists. Data retrieval was incomplete after 1 year despite EMA's commitment to transparency. Improvements are needed.

Project description:ImportanceClinical trial registries are intended to increase clinical research transparency by nonselectively identifying and documenting clinical trial designs and outcomes. Inconsistencies in reported data undermine the utility of such registries and have previously been noted in general medical literature.ObjectiveTo assess whether inconsistencies in reported data exist between ophthalmic literature and clinical trial registries.Design, setting, and participantsIn this retrospective, cross-sectional study, interventional clinical trials published from January 1, 2014, to December 31, 2014, in the American Journal of Ophthalmology, JAMA Ophthalmology, and Ophthalmology were reviewed. Observational, retrospective, uncontrolled, and post hoc reports were excluded, yielding a sample size of 106 articles. Data collection was performed from January through September 2016. Data review and adjudication continued through January 2017.Main outcomes and measuresIf possible, articles were matched to registry entries listed in the ClinicalTrials.gov database or in 1 of 16 international registries indexed by the World Health Organization International Clinical Trials Registry Platform version 3.2 search engine. Each article-registry pair was assessed for inconsistencies in design, results, and funding (each of which was further divided into subcategories) by 2 reviewers and adjudicated by a third.ResultsOf 106 trials that met the study criteria, matching registry entries were found for 68 (64.2%), whereas no matching registry entries were found for 38 (35.8%). Inconsistencies were identified in study design, study results, and funding sources, including specific interventions in 8 (11.8%), primary outcome measure (POM) designs in 32 (47.1%), and POM results in 48 (70.6%). In addition, numerous data pieces were unreported, including analysis methods in 52 (76.5%) and POM results in 38 (55.9%).Conclusions and relevanceClinical trial registries were underused in this sample of ophthalmology clinical trials. For studies with registry data, inconsistency rates between published and registered data were similar to those previously reported for general medical literature. In most cases, inconsistencies involved missing data, but explicit discrepancies in methods and/or data were also found. Transparency and credibility of published trials may be improved by closer attention to their registration and reporting.

Project description:BACKGROUND:No study has looked at differences of pooled estimates-such as meta-analyses-of corresponding study documents of the same intervention. In this study, we compared meta-analyses of human papillomavirus (HPV) vaccine trial data from clinical study reports with trial data from corresponding trial register entries and journal publications. METHODS:We obtained clinical study reports from the European Medicines Agency and GlaxoSmithKline, corresponding trial register entries from ClinicalTrials.gov and corresponding journal publications via the Cochrane Collaboration's Central Register of Controlled Trials, Google Scholar and PubMed. Two researchers extracted data. We compared reporting of trial design aspects and 20 prespecified benefit and harm outcomes extracted from each study document type. Risk ratios were calculated with the random effects inverse variance method. RESULTS:We included study documents from 22 randomized clinical trials and 2 follow-up studies with 95,670 healthy participants and non-HPV vaccine comparators (placebo, HPV vaccine adjuvants and hepatitis vaccines). We obtained 24 clinical study reports, 24 corresponding trial register entries and 23 corresponding journal publications; the median number of pages was 1351 (range 357 to 11,456), 32 (range 11 to 167) and 11 (range 7 to 83), respectively. All 24 (100%) clinical study reports, no (0%) trial register entries and 9 (39%) journal publications reported on all six major design-related biases defined by the Cochrane Handbook version 2011. The clinical study reports reported more inclusion criteria (mean 7.0 vs. 5.8 [trial register entries] and 4.0 [journal publications]) and exclusion criteria (mean 17.8 vs. 11.7 and 5.0) but fewer primary outcomes (mean 1.6 vs. 3.5 and 1.2) and secondary outcomes (mean 8.8 vs. 13.0 and 3.2) than the trial register entries. Results were posted for 19 trial register entries (79%). Compared to the clinical study reports, the trial register entries and journal publications contained 3% and 44% of the seven assessed benefit data points (6879 vs. 230 and 3015) and 38% and 31% of the 13 assessed harm data points (167,550 vs. 64,143 and 51,899). No meta-analysis estimate differed significantly when we compared pooled risk ratio estimates of corresponding study document data as ratios of relative risk. CONCLUSION:There were no significant differences in the meta-analysis estimates of the assessed outcomes from corresponding study documents. The clinical study reports were the superior study documents in terms of the quantity and the quality of the data they contained and should be used as primary data sources in systematic reviews. SYSTEMATIC REVIEW REGISTRATION:The protocol for our comparison is registered on PROSPERO as an addendum to our systematic review of the benefits and harms of the HPV vaccines: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20180320.pdf: CRD42017056093. Our systematic review protocol was registered on PROSPERO on January 2017: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20170030.pdf. Two protocol amendments were registered on PROSPERO on November 2017: https://www.crd.york.ac.uk/PROSPEROFILES/56093_PROTOCOL_20171116.pdf. Our index of the HPV vaccine studies was published in Systematic Reviews on January 2018: https://doi.org/10.1186/s13643-018-0675-z. A description of the challenges obtaining the data was published on September 2018: https://doi.org/10.1136/bmj.k3694.