Dataset Information

P-hacking in clinical trials and how incentives shape the distribution of results across phases.

ABSTRACT: Clinical research should conform to high standards of ethical and scientific integrity, given that human lives are at stake. However, economic incentives can generate conflicts of interest for investigators, who may be inclined to withhold unfavorable results or even tamper with data in order to achieve desired outcomes. To shed light on the integrity of clinical trial results, this paper systematically analyzes the distribution of P values of primary outcomes for phase II and phase III drug trials reported to the ClinicalTrials.gov registry. First, we detect no bunching of results just above the classical 5% threshold for statistical significance. Second, a density-discontinuity test reveals an upward jump at the 5% threshold for phase III results by small industry sponsors. Third, we document a larger fraction of significant results in phase III compared to phase II. Linking trials across phases, we find that early favorable results increase the likelihood of continuing into the next phase. Once we take into account this selective continuation, we can explain almost completely the excess of significant results in phase III for trials conducted by large industry sponsors. For small industry sponsors, instead, part of the excess remains unexplained.

SUBMITTER: Adda J

PROVIDER: S-EPMC7306753 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

P-hacking in clinical trials and how incentives shape the distribution of results across phases.

Adda Jérôme J Decker Christian C Ottaviani Marco M

Proceedings of the National Academy of Sciences of the United States of America 20200602 24

Clinical research should conform to high standards of ethical and scientific integrity, given that human lives are at stake. However, economic incentives can generate conflicts of interest for investigators, who may be inclined to withhold unfavorable results or even tamper with data in order to achieve desired outcomes. To shed light on the integrity of clinical trial results, this paper systematically analyzes the distribution of <i>P</i> values of primary outcomes for phase II and phase III d ...[more]

PMID: 32487730

Similar Datasets

Project description:ImportanceIncentivizing research participation is controversial and variably regulated because of uncertainty regarding whether financial incentives serve as undue inducements by diminishing peoples' sensitivity to research risks or unjust inducements by preferentially increasing enrollment among underserved individuals.ObjectiveTo determine whether incentives improve enrollment in real randomized clinical trials (RCTs) or serve as undue or unjust inducements.Design, setting, and participantsTwo RCTs of incentives that were embedded in 2 parent RCTs, 1 comparing smoking cessation interventions (conducted at smoking cessation clinics in 2 health systems) and 1 evaluating an ambulation intervention (conducted across wards of the Hospital of the University of Pennsylvania) included all persons eligible for the parent trials who did not have prior knowledge of the incentives trials. Recruitment occurred from September 2017 to August 2019 for the smoking trial and January 2018 through May 2019 for the ambulation trial; data were analyzed from January 2020 to July 2020.InterventionsPatients were randomly assigned to incentives of $0, $200, or $500 for participating in the smoking cessation trial and $0, $100, or $300 for the ambulation trial.Main outcomes and measuresThe primary outcome of each incentive trial was the proportion of people assigned to each recruitment strategy that consented to participate. Each trial was powered to test the hypotheses that incentives served neither as undue inducements (based on the interaction between incentive size and perceived research risk, as measured using a 10-point scale, on the primary outcome), nor unjust inducements (based on the interaction between incentive size and participants' self-reported income). Noninferiority methods were used to test whether the data were compatible with these 2 effects of incentives and superiority methods to compare the primary and other secondary outcomes.ResultsThere were a total of 654 participants (327 women [50.0%]; mean [SD] age, 50.6 [12.1] years; 394 Black/African American [60.2%], 214 White [32.7%], and 24 multiracial individuals [3.7%]) in the smoking trial, and 642 participants (364 women [56.7%]; mean [SD] age, 46.7 [15.6] years; 224 Black/African American [34.9%], 335 White [52.2%], and 5 multiracial individuals [0.8%]) in the ambulation trial. Incentives significantly increased consent rates among those in the smoking trial in 47 of 216 (21.8%), 78 of 217 (35.9%), and 104 of 221 (47.1%) in the $0, $200, and $500 groups, respectively (adjusted odds ratio [aOR] for each increase in incentive, 1.70; 95% CI, 1.34-2.17; P < .001). Incentives did not increase consent among those in the ambulation trial: 98 of 216 (45.4%), 102 of 212 (48.1%), and 92 of 214 (43.0%) in the $0, $100, and $300 groups, respectively (aOR, 0.88; 95% CI, 0.64-1.22; P = .45). In neither trial was there evidence of undue or unjust inducement (upper confidence limits of ORs for undue inducement, 1.15 and 0.99; P < .001 showing noninferiority; upper confidence limits of ORs for unjust inducement, 1.21 and 1.26; P = .01 and P < .001, respectively). There were no significant effects of incentive size on the secondary outcomes in either trial, including time spent reviewing the risk sections of consent forms, perceived research risks, trial understanding, perceived coercion, or therapeutic misconceptions.Conclusions and relevanceIn these 2 randomized clinical trials, financial incentives increased trial enrollment in 1 of 2 trials and did not produce undue or unjust inducement or other unintended consequences in either trial.Trial registrationClinicalTrials.gov Identifier: NCT02697799.

Project description:ObjectiveTo identify the availability of results for trials registered on the European Union Clinical Trials Register (EUCTR) compared with other dissemination routes to understand its value as a results repository.DesignCross sectional audit study.SettingEUCTR protocols and results sections, data extracted 1-3 December 2020.PopulationRandom sample of 500 trials registered on EUCTR with a completion date of more than two years from the beginning of searches (ie, 1 December 2018).Main outcome measuresProportion of trials with results across the examined dissemination routes (EUCTR, ClinicalTrials.gov, ISRCTN registry, and journal publications), and for each dissemination route individually. Prespecified secondary outcomes were number and proportion of unique results, and the timing of results, for each dissemination route.ResultsIn the sample of 500 trials, availability of results on EUCTR (53.2%, 95% confidence interval 48.8% to 57.6%) was similar to the peer reviewed literature (58.6%, 54.3% to 62.9%) and exceeded the proportion of results available on other registries with matched records. Among the 383 trials with any results, 55 (14.4%, 10.9% to 17.9%) were only available on EUCTR. Also, after the launch of the EUCTR results database, median time to results was fastest on EUCTR (1142 days, 95% confidence interval 812 to 1492), comparable with journal publications (1226 days, 1074 to 1551), and exceeding ClinicalTrials.gov (3321 days, 1653 to undefined). For 117 trials (23.4%, 19.7% to 27.1%), however, results were published elsewhere but not submitted to the EUCTR registry, and no results were located in any dissemination route for 117 trials (23.4%, 19.7% to 27.1).ConclusionsEUCTR should be considered in results searches for systematic reviews and can help researchers and the public to access the results of clinical trials, unavailable elsewhere, in a timely way. Reporting requirements, such as the EU's, can help in avoiding research waste by ensuring results are reported. The registry's true value, however, is unrealised because of inadequate compliance with EU guidelines, and problems with data quality that complicate the routine use of the registry. As the EU transitions to a new registry, continuing to emphasise the importance of EUCTR and the provision of timely and complete data is critical. For the future, EUCTR will still hold important information from the past two decades of clinical research in Europe. With increased efforts from sponsors and regulators, the registry can continue to grow as a source of results of clinical trials, many of which might be unavailable from other dissemination routes.

Dataset Information

P-hacking in clinical trials and how incentives shape the distribution of results across phases.

Publications

P-hacking in clinical trials and how incentives shape the distribution of results across phases.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets