Dataset Information

Effect of revealing authors' conflicts of interests in peer review: randomized controlled trial.

ABSTRACT: OBJECTIVE:To assess the effect of disclosing authors' conflict of interest declarations to peer reviewers at a medical journal. DESIGN:Randomized controlled trial. SETTING:Manuscript review process at the Annals of Emergency Medicine. PARTICIPANTS: Reviewers (n=838) who reviewed manuscripts submitted between 2 June 2014 and 23 January 2018 inclusive (n=1480 manuscripts). INTERVENTION:Reviewers were randomized to either receive (treatment) or not receive (control) authors' full International Committee of Medical Journal Editors format conflict of interest disclosures before reviewing manuscripts. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain "counterfactual scores"-that is, the scores they believed they would have given had they been assigned to the opposite arm-as well as attitudes toward conflicts of interest. MAIN OUTCOME MEASURE:Overall quality score that reviewers assigned to the manuscript on submitting their review (1 to 5 scale). Secondary outcomes were scores the reviewers submitted for the seven more specific quality ratings and counterfactual scores elicited in the follow-up survey. RESULTS:Providing authors' conflict of interest disclosures did not affect reviewers' mean ratings of manuscript quality (Mcontrol=2.70 (SD 1.11) out of 5; Mtreatment=2.74 (1.13) out of 5; mean difference 0.04, 95% confidence interval -0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol= 2.85 (1.12) out of 5; Mtreatment=2.96 (1.16) out of 5; mean difference 0.11, -0.05 to 0.26). Similarly, no effect of the treatment was seen on any of the other seven quality ratings that the reviewers assigned. Reviewers acknowledged conflicts of interest as an important matter and believed that they could correct for them when they were disclosed. However, their counterfactual scores did not differ from actual scores (Mactual=2.69; Mcounterfactual=2.67; difference in means 0.02, 0.01 to 0.02). When conflicts were reported, a comparison of different source types (for example, government, for-profit corporation) found no difference in effect. CONCLUSIONS:Current ethical standards require disclosure of conflicts of interest for all scientific reports. As currently implemented, this practice had no effect on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.

SUBMITTER: John LK

PROVIDER: S-EPMC6833969 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMuch research worldwide is focussed on cost containment and better adherence to guidelines in healthcare. The research focussing on professional behaviour is often performed in a well-controlled research setting. In this study a large-scale implementation of a peer review strategy was tested on both test ordering and prescribing behaviour in primary care in the normal quality improvement setting.MethodsWe planned a cluster-RCT in existing local quality improvement collaboratives (LQICs) in primary care. The study ran from January 2008 to January 2011. LQICs were randomly assigned to one of two trial arms, with each arm receiving the same intervention of audit and feedback combined with peer review. Both arms were offered five different clinical topics and acted as blind controls for the other arm. The differences in test ordering rates and prescribing rates between both arms were analysed in an intention-to-treat pre-post analysis and a per-protocol analysis.ResultsTwenty-one LQIC groups, including 197 GPs working in 88 practices, entered the trial. The intention-to-treat analysis did not show a difference in the changes in test ordering or prescribing performance between intervention and control groups. The per-protocol analysis showed positive results for half of the clinical topics. The increase in total tests ordered was 3% in the intervention arm and 15% in the control arm. For prescribing the increase in prescriptions was 20% in the intervention arm and 66% in the control group. It was observed that the groups with the highest baseline test ordering and prescription volumes showed the largest improvements.ConclusionsOur study shows that the results from earlier work could not be confirmed by our attempt to implement the strategy in the field. We did not see a decrease in the volumes of tests ordered or of the drugs prescribed but were able to show a lesser increase instead. Implementing the peer review with audit and feedback proved to be not feasible in primary care in the Netherlands.Trial registrationThis trial was registered at the Dutch trial register under number ISRCTN40008171 on August 7th 2007.

Project description:BackgroundVast sums are distributed based on grant peer review, but studies show that interrater reliability is often low. In this study, we tested the effect of receiving two short individual feedback reports compared to one short general feedback report on the agreement between reviewers.MethodsA total of 42 reviewers at the Norwegian Foundation Dam were randomly assigned to receive either a general feedback report or an individual feedback report. The general feedback group received one report before the start of the reviews that contained general information about the previous call in which the reviewers participated. In the individual feedback group, the reviewers received two reports, one before the review period (based on the previous call) and one during the period (based on the current call). In the individual feedback group, the reviewers were presented with detailed information on their scoring compared with the review committee as a whole, both before and during the review period. The main outcomes were the proportion of agreement in the eligibility assessment and the average difference in scores between pairs of reviewers assessing the same proposal. The outcomes were measured in 2017 and after the feedback was provided in 2018.ResultsA total of 2398 paired reviews were included in the analysis. There was a significant difference between the two groups in the proportion of absolute agreement on whether the proposal was eligible for the funding programme, with the general feedback group demonstrating a higher rate of agreement. There was no difference between the two groups in terms of the average score difference. However, the agreement regarding the proposal score remained critically low for both groups.ConclusionsWe did not observe changes in proposal score agreement between 2017 and 2018 in reviewers receiving different feedback. The low levels of agreement remain a major concern in grant peer review, and research to identify contributing factors as well as the development and testing of interventions to increase agreement rates are still needed.Trial registrationThe study was preregistered at OSF.io/n4fq3 .

Project description:Individual patient data (IPD) meta-analysis is considered the "gold standard" for exploring the effectiveness of interventions in different subgroups of patients. However, obtaining IPD is time-consuming and contact with the researchers responsible for the original trials is usually required. To date, there are no studies evaluating different strategies to optimize the process for retrieval of IPD from such researchers. Our aim is to examine the impact of providing incentives to the researchers responsible for the trials eligible for a meta-analysis to submit their IPD.We updated our previously published systematic reviews for type 1 diabetes mellitus comparing long- and intermediate-acting insulin regimens (from January 2013 to June 2015) and for Alzheimer's dementia comparing cognitive enhancers (from January 2015 to May 2015). Eligible were randomized controlled trials (RCTs) fulfilling the eligibility criteria of the systematic reviews. We will randomly allocate authors of the reports of these RCTs into an intervention or control group. Those allocated to the intervention group will be contacted by email, mail, and phone, and will be asked to provide the IPD from their RCT and will be given a financial incentive. Those allocated to the control group will be contacted by email, mail, and phone, but will not receive a financial incentive. Our primary outcome will be the proportion of authors who provide the IPD. The secondary outcomes will be the time to return the dataset (defined as the period between the information request and the authors' response with the dataset), and completeness of data. We will compare the response rates in the two groups using the odds ratio and the corresponding 95 % confidence interval. We will also use binary logistic regression and cox regression analyses to examine whether different RCT characteristics, such as study size and sponsor information, influence the probability of providing IPD and the time needed to share the data.This study will determine whether a financial incentive affects response rates when seeking IPD from the original researchers. We will disseminate our findings in an open access scientific journal and present results at national and international conferences.This trial is registered in Clinical Trials.gov, ID number NCT02569411 . Date of registration 5 October 2015.

Dataset Information

Effect of revealing authors' conflicts of interests in peer review: randomized controlled trial.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure