Dataset Information

COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials.

ABSTRACT: BACKGROUND:Discrepancies between pre-specified and reported outcomes are an important and prevalent source of bias in clinical trials. COMPare (Centre for Evidence-Based Medicine Outcome Monitoring Project) monitored all trials in five leading journals for correct outcome reporting, submitted correction letters on all misreported trials in real time, and then monitored responses from editors and trialists. From the trialists' responses, we aimed to answer two related questions. First, what can trialists' responses to corrections on their own misreported trials tell us about trialists' knowledge of correct outcome reporting? Second, what can a cohort of responses to a standardised correction letter tell us about how researchers respond to systematic critical post-publication peer review? METHODS:All correspondence from trialists, published by journals in response to a correction letter from COMPare, was filed and indexed. We analysed the letters qualitatively and identified key themes in researchers' errors about correct outcome reporting, and approaches taken by researchers when their work was criticised. RESULTS:Trialists frequently expressed views that contradicted the CONSORT (Consolidated Standards of Reporting Trials) guidelines or made inaccurate statements about correct outcome reporting. Common themes were: stating that pre-specification after trial commencement is acceptable; incorrect statements about registries; incorrect statements around the handling of multiple time points; and failure to recognise the need to report changes to pre-specified outcomes in the trial report. We identified additional themes in the approaches taken by researchers when responding to critical correspondence, including the following: ad hominem criticism; arguing that trialists should be trusted, rather than follow guidelines for trial reporting; appealing to the existence of a novel category of outcomes whose results need not necessarily be reported; incorrect statements by researchers about their own paper; and statements undermining transparency infrastructure, such as trial registers. CONCLUSIONS:Researchers commonly make incorrect statements about correct trial reporting. There are recurring themes in researchers' responses when their work is criticised, some of which fall short of the scientific ideal. Research on methodological shortcomings is now common, typically in the form of retrospective cohort studies describing the overall prevalence of a problem. We argue that prospective cohort studies which additionally issue correction letters in real time on each individual flawed study-and then follow-up responses from trialists and journals-are more impactful, more informative for those consuming the studies critiqued, more informative on the causes of shortcomings in research, and a better use of research resources.

SUBMITTER: Goldacre B

PROVIDER: S-EPMC6374909 | biostudies-literature | 2019 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials.

Goldacre Ben B Drysdale Henry H Marston Cicely C Mahtani Kamal R KR Dale Aaron A Milosevic Ioan I Slade Eirion E Hartley Philip P Heneghan Carl C

Trials 20190214 1

<h4>Background</h4>Discrepancies between pre-specified and reported outcomes are an important and prevalent source of bias in clinical trials. COMPare (Centre for Evidence-Based Medicine Outcome Monitoring Project) monitored all trials in five leading journals for correct outcome reporting, submitted correction letters on all misreported trials in real time, and then monitored responses from editors and trialists. From the trialists' responses, we aimed to answer two related questions. First, wh ...[more]

PMID: 30760328

Similar Datasets

Project description:BackgroundDiscrepancies between pre-specified and reported outcomes are an important source of bias in trials. Despite legislation, guidelines and public commitments on correct reporting from journals, outcome misreporting continues to be prevalent. We aimed to document the extent of misreporting, establish whether it was possible to publish correction letters on all misreported trials as they were published, and monitor responses from editors and trialists to understand why outcome misreporting persists despite public commitments to address it.MethodsWe identified five high-impact journals endorsing Consolidated Standards of Reporting Trials (CONSORT) (New England Journal of Medicine, The Lancet, Journal of the American Medical Association, British Medical Journal, and Annals of Internal Medicine) and assessed all trials over a six-week period to identify every correctly and incorrectly reported outcome, comparing published reports against published protocols or registry entries, using CONSORT as the gold standard. A correction letter describing all discrepancies was submitted to the journal for all misreported trials, and detailed coding sheets were shared publicly. The proportion of letters published and delay to publication were assessed over 12 months of follow-up. Correspondence received from journals and authors was documented and themes were extracted.ResultsSixty-seven trials were assessed in total. Outcome reporting was poor overall and there was wide variation between journals on pre-specified primary outcomes (mean 76% correctly reported, journal range 25-96%), secondary outcomes (mean 55%, range 31-72%), and number of undeclared additional outcomes per trial (mean 5.4, range 2.9-8.3). Fifty-eight trials had discrepancies requiring a correction letter (87%, journal range 67-100%). Twenty-three letters were published (40%) with extensive variation between journals (range 0-100%). Where letters were published, there were delays (median 99 days, range 0-257 days). Twenty-nine studies had a pre-trial protocol publicly available (43%, range 0-86%). Qualitative analysis demonstrated extensive misunderstandings among journal editors about correct outcome reporting and CONSORT. Some journals did not engage positively when provided correspondence that identified misreporting; we identified possible breaches of ethics and publishing guidelines.ConclusionsAll five journals were listed as endorsing CONSORT, but all exhibited extensive breaches of this guidance, and most rejected correction letters documenting shortcomings. Readers are likely to be misled by this discrepancy. We discuss the advantages of prospective methodology research sharing all data openly and pro-actively in real time as feedback on critiqued studies. This is the first empirical study of major academic journals' willingness to publish a cohort of comparable and objective correction letters on misreported high-impact studies. Suggested improvements include changes to correspondence processes at journals, alternatives for indexed post-publication peer review, changes to CONSORT's mechanisms for enforcement, and novel strategies for research on methods and reporting.

Project description:BackgroundFundamental to the success of clinical research that involves human participants is the quality of the data that is generated. To ensure data quality, clinical trials must comply with the Good Clinical Practice guideline which recommends data monitoring. To date, the guideline is broad, requires technology for enforcement, follows strict industry standards, mostly designed for drug-registration trials and based on informal consensus. It is also unknown what challenges clinical trials and researchers face in implementing data monitoring procedures. Thus, this study aimed to describe researcher experiences with data quality monitoring in clinical trials.MethodsWe conducted semi-structured telephone interviews following a guided-phenomenological approach. Participants were recruited from the Australian and New Zealand Clinical Trials Registry and were researchers affiliated with a listed clinical study. Each transcript was analysed with inductive thematic analysis before thematic categorisation of themes from all transcripts. Primary, secondary and subthemes were categorised according to the emerging relationships.ResultsData saturation were reached after interviewing seven participants. Five primary themes, two secondary themes and 21 subthemes in relation to data quality monitoring emerged from the data. The five primary themes included: education and training, ways of working, working with technology, working with data, and working within regulatory requirements. The primary theme 'education and training' influenced the other four primary themes. While 'working with technology' influenced the 'way of working'. All other themes had reciprocal relationships. There was no relationship reported between 'working within regulatory requirements' and 'working with technology'. The researchers experienced challenges in meeting regulatory requirements, using technology and fostering working relationships for data quality monitoring.ConclusionClinical trials implemented a variety of data quality monitoring procedures tailored to their situation and study context. Standardised frameworks that are accessible to all types of clinical trials are needed with an emphasis on education and training.

Project description:BackgroundClinical trials in critical care are often resource-intense, with many unique challenges. Barriers to effective recruitment and implementation of study intervention have not been explored in a UK context.AimTo identify facilitating factors and barriers to enrolling patients into critical care clinical trials within the UK from clinician's perspectives.MethodsA qualitative interview study was undertaken on behalf of the National Institute of Health Research critical care specialty group, in which research active clinicians across different Clinical Research Networks were interviewed. A loosely structured interview schedule was used, based on themes generated from the literature associated with accessing critical care trials. Research teams (critical care doctors, research nurses, and trial coordinators) from hospitals from each Clinical Research Network were contacted to try to achieve representation across the UK.ResultsInterviews were carried out across nine UK Clinical Research Networks with a range of doctors and research nurses. All hospitals were teaching hospitals with varying research nurse numbers and allocated consultant research sessions. There were a range of six to nine ongoing clinical trials in critical care for each centre representative interviewed. Data were analysed using framework analysis, and six final themes were identified related to factors associated with: centre, unit, resources, study, clinician, and patient/family. The most commonly cited barrier to conducting clinical trials was related to resources, namely insufficient human and financial resources, leading to staff and study recruitment difficulties. Clinical uncertainty and equipoise regarding comparative merits of trials were challenging in terms of engaging critical care teams. A number of patient and family factors added complexities in terms of recruitment; however, refusal rates were generally reported as low.ConclusionFlexibility in funding and employment by research teams enables continuity of studies and staff. Innovative measures to incentivise research nurses and clinical teams can help recruit more patients into trials. Research teams are highly committed to providing cover to recruit critical care trials, and a significant effort to anticipate barriers is undertaken; these endeavours are summarised to provide guidance for other teams wishing to address any potential difficulties.

Project description:Unlabelled: Patient and public involvement (PPI) in research is increasingly required, although evidence to inform its implementation is limited.ObjectiveInform the evidence base by describing how plans for PPI were implemented within clinical trials and identifying the challenges and lessons learnt by research teams.MethodsWe compared PPI plans extracted from clinical trial grant applications (funded by the National Institute for Health Research Health Technology Assessment Programme between 2006 and 2010) with researchers' and PPI contributors' interview accounts of PPI implementation. Analysis of PPI plans and transcribed qualitative interviews drew on the Framework technique.ResultsOf 28 trials, 25 documented plans for PPI in funding applications and half described implementing PPI before applying for funding. Plans varied from minimal to extensive, although almost all anticipated multiple modes of PPI. Interview accounts indicated that PPI plans had been fully implemented in 20/25 trials and even expanded in some. Nevertheless, some researchers described PPI within their trials as tokenistic. Researchers and contributors noted that late or minimal PPI engagement diminished its value. Both groups perceived uncertainty about roles in relation to PPI, and noted contributors' lack of confidence and difficulties attending meetings. PPI contributors experienced problems in interacting with researchers and understanding technical language. Researchers reported difficulties finding 'the right' PPI contributors, and advised caution when involving investigators' current patients.ConclusionsEngaging PPI contributors early and ensuring ongoing clarity about their activities, roles and goals, is crucial to PPI's success. Funders, reviewers and regulators should recognise the value of preapplication PPI and allocate further resources to it. They should also consider whether PPI plans in grant applications match a trial's distinct needs. Monitoring and reporting PPI before, during and after trials will help the research community to optimise PPI, although the need for ongoing flexibility in implementing PPI should also be recognised.

Project description:BACKGROUND:Health and health-related data collected as part of clinical care is a foundational component of quality improvement and research. While the importance of these data is widely recognized, there are many challenges faced by researchers attempting to use such data. It is crucial to acknowledge and identify barriers to improve data sharing and access practices and ultimately optimize research capacity. OBJECTIVE:To better understand the current state, explore opportunities, and identify barriers, an environmental scan of investigators at BC Children's Hospital Research Institute (BCCHR) was conducted to elucidate current local practices around data access and usage. METHODS:The Clinical and Community Data, Analytics and Informatics group at BCCHR comprises over 40 investigators with diverse expertise and interest in data who share a common goal of facilitating data collection, usage, and access across the community. Semistructured interviews with 35 of these researchers were conducted, and data were summarized qualitatively. A total impact score, considering both frequency with which a problem occurs and the impact of the problem, was calculated for each item to prioritize and rank barriers. RESULTS:Three main themes for barriers emerged: the lengthy turnaround time before data access (18/35, 51%), inconsistent and opaque data access processes (16/35, 46%), and the inability to link data (15/35, 43%) effectively. Less frequent themes included quality and usability of data, ethics and privacy review barriers, lack of awareness of data sources, and efforts required duplicating data extraction and linkage. The two main opportunities for improvement were data access facilitation (14/32, 44%) and migration toward a single data platform (10/32, 31%). CONCLUSIONS:By identifying the current state and needs of the data community onsite, this study enables us to focus our resources on combating the challenges having the greatest impact on researchers. The current state parallels that of the national landscape. By ensuring protection of privacy while achieving efficient data access, research institutions will be able to maximize their research capacity, a crucial step towards achieving the ultimate and shared goal between all stakeholders-to better health outcomes.

Dataset Information

COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials.

Publications

COMPare: Qualitative analysis of researchers' responses to critical correspondence on a cohort of 58 misreported trials.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets