Dataset Information

A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice.

ABSTRACT: BACKGROUND:While randomised controlled trials (RCTs) provide high-quality evidence to guide practice, much routine care is not based upon available RCTs. This disconnect between evidence and practice is not sufficiently well understood. This case study explores this relationship using a novel approach. Better understanding may improve trial design, conduct, reporting and implementation, helping patients benefit from the best available evidence. METHODS:We employed a case-study approach, comprising mixed methods to examine the case of interest: the primary outcome paper of a surgical RCT (the TIME trial). Letters and editorials citing the TIME trial's primary report underwent qualitative thematic analysis, and the RCT was critically appraised using validated tools. These analyses were compared to provide insight into how the TIME trial findings were interpreted and appraised by the clinical community. RESULTS:23 letters and editorials were studied. Most authorship included at least one academic (20/23) and one surgeon (21/23). Authors identified wide-ranging issues including confounding variables or outcome selection. Clear descriptions of bias or generalisability were lacking. Structured appraisal identified risks of bias. Non-RCT evidence was less critically appraised. Authors reached varying conclusions about the trial without consistent justification. Authors discussed aspects of internal and external validity covered by appraisal tools but did not use these methodological terms in their articles. CONCLUSIONS:This novel method for examining interpretation of an RCT in the clinical community showed that published responses identified limited issues with trial design. Responses did not provide coherent rationales for accepting (or not) trial results. Findings may suggest that authors lacked skills in appraisal of RCT design and conduct. Multiple case studies with cross-case analysis of other trials are needed.

SUBMITTER: Byrne BE

PROVIDER: S-EPMC7216481 | biostudies-literature | 2020 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice.

Byrne Ben E BE Rooshenas Leila L Lambert Helen S HS Blazeby Jane M JM

BMC medical research methodology 20200512 1

<h4>Background</h4>While randomised controlled trials (RCTs) provide high-quality evidence to guide practice, much routine care is not based upon available RCTs. This disconnect between evidence and practice is not sufficiently well understood. This case study explores this relationship using a novel approach. Better understanding may improve trial design, conduct, reporting and implementation, helping patients benefit from the best available evidence.<h4>Methods</h4>We employed a case-study app ...[more]

PMID: 32398100

Similar Datasets

Project description:BackgroundWhile patient and public involvement (PPI) in clinical trials is beneficial and mandated by some funders, formal guidance on how to implement PPI is limited and challenges have been reported. We aimed to investigate how PPI is approached within a UK Clinical Trials Unit (CTU)'s portfolio of randomised controlled trials, perceived barriers to/facilitators of its successful implementation, and perspectives on the CTU's role in PPI.MethodsA mixed-methods study design, involving (1) an online survey of 26 trial managers (TMs) and (2) Interviews with Trial Management Group members and public contributors from 8 case-study trials. Quantitative survey data were summarised using descriptive statistics and interview transcripts analysed thematically. Two public contributors advised throughout and are co-authors.Results(1) 21 TMs completed the survey; (2) 19 in-depth interviews were conducted with public contributors (n=8), TMs (n=5), chief investigators (n=3), PPI coordinators (n=2) and a researcher. 15/21 TMs surveyed reported that a public contributor was on the trial team, and 5 used another PPI method. 12/21 TMs reported that public contributors were paid (range £10-50/h). 5 TMs reported that training was provided for public contributors and few staff members had received any formal PPI training. The most commonly reported tasks undertaken by public contributors were the review of participant-facing materials/study documents and advising on recruitment/retention strategies. Public contributors wanted and valued feedback on changes made due to their input, but it was not always provided. Barriers to successful PPI included recruitment challenges, group dynamics, maintaining professional boundaries, negative attitudes to PPI amongst some researchers, a lack of continuity of trial staff, and the academic environment. Successful PPI required early and explicit planning, sharing of power and ownership of the trial with public contributors, building and maintaining relationships, and joint understanding and clarity about expectations/roles. CTUs have an important role to play in supporting recruitment, signposting and coordinating PPI.ConclusionsWhile highly valuable, PPI in trials is currently variable. PPI representatives are recruited informally, may not be provided with any training and are paid inconsistently across trials. Study findings can help optimise PPI in trials and ensure researchers and public contributors are adequately supported.

Project description:IntroductionRandomised controlled trials (RCTs) provide high-quality evidence to inform practice. However, much routine care is not based on available RCT evidence. Understanding this disconnect may improve trial design, reporting and implementation. Published literature commenting on RCTs may yield relevant insights. This protocol presents a new approach examining how researchers understand, contextualise and use evidence from RCTs, through analysis of letters, editorials and discussion pieces citing individual RCTs. Surgical case studies will illustrate its ability to identify wide-ranging factors influencing application of trials evidence.Methods and analysisIn-depth study of published literature will explore written responses to RCTs. After purposefully selecting individual RCTs, we will systematically identify all citing articles covered in Web of Science and Scopus. Editorials, discussions and letters will be included. These are considered most likely to provide critiques and opinions about index RCTs. Original articles and reviews will be excluded. Clinical specialty, RCT design, outcomes and bibliographical data will be collected for RCTs and citing articles. Citing articles will be thematically analysed using the constant comparison technique to explore author understanding, contextualisation and relationship to clinical practice for the index trial. Coding will include generic issues relevant to all RCTs, such as sample size or blinding, and features specific to surgery, such as learning curve. Index trial quality will be examined using validated tools. Results will be combined to create a broad overview of the understanding and use of RCT evidence.Ethics and disseminationThis study involves secondary use of existing articles and does not require ethical approval. Pilot work will establish its feasibility and inform progression to larger scale utilisation across a broad range of RCTs. Findings will be published in a peer-reviewed journal and presented at surgical and methodological conferences. Results will guide future work on trial design to optimise implementation of results.

Project description:BackgroundRoutine data are increasingly used in randomised controlled trials evaluating healthcare interventions. They can aid participant identification, outcome assessment, and intervention delivery. Randomised implementation trials evaluate the effect of implementation strategies on implementation outcomes. Implementation strategies, such as reminders, are used to increase the uptake of evidence-based interventions into practice, while implementation outcomes, such as adoption, are key measures of the implementation process. The use of routine data in effectiveness trials has been explored; however, there are no reviews on implementation trials. We therefore aimed to describe how routine data have been used in randomised implementation trials and the design characteristics of these trials.MethodsWe searched MEDLINE (Ovid) and Cochrane Central Register of Controlled Trials from Jan 2000 to Dec 2021 and manually searched protocols from trial registers. We included implementation trials and type II and type III hybrid effectiveness-implementation trials conducted using routine data. We extracted quantitative and qualitative data and narratively synthesised findings.ResultsFrom 4206 titles, we included 80 trials, of which 22.5% targeted implementation of evidence-based clinical guidelines. Multicomponent implementation strategies were more commonly evaluated (70.0%) than single strategies. Most trials assessed adoption as the primary outcome (65.0%). The majority of trials extracted data from electronic health records (EHRs) (62.5%), and 91.3% used routine data for outcome ascertainment. Reported reasons for using routine data were increasing efficiency, assessing outcomes, reducing research burden, improving quality of care, identifying study samples, confirming findings, and assessing representativeness. Data quality, the EHR system, research governance, and external factors such as government policy could act either as facilitators or barriers.ConclusionsAdherence to guidance on designing and reporting implementation studies, and specifically to harmonise the language used in describing implementation strategies and implementation outcomes, would aid identification of studies and data extraction. Routine healthcare data are widely used for participant identification, outcome assessment and intervention delivery. Researchers should familiarise themselves with the barriers and facilitators to using routine data, and efforts could be made to improve data quality to overcome some of the barriers.RegistrationPROSPERO CRD42022292321.

Project description:BackgroundA common challenge for randomised controlled trials (RCTs) is recruiting enough participants to be adequately powered to answer the research question. Recruitment has been set as a priority research area in trials to improve recruitment and thereby reduce wasted resources in conducted trials that fail to recruit sufficiently.MethodsWe conducted a systematic mixed studies review to identify the factors associated with recruitment to RCTs in general practice. On September 8, 2020, English language studies were identified from MEDLINE, EMBASE, Cochrane Database of Systematic Reviews and CENTRAL databases for published studies. NTIS and OpenGrey were searched for grey literature, and BMC Trials was hand searched. A narrative synthesis was conducted for qualitative studies and a thematic synthesis for qualitative studies.ResultsThirty-seven studies met the inclusion criteria. These were of different study types (10 cross-sectional, 5 non-randomised studies of interventions, 2 RCTs, 10 qualitative and 10 mixed methods). The highest proportion was conducted in the UK (48%). The study quality was generally poor with 24 (65%) studies having major concerns. A complex combination of patient, practitioner or practice factors, and patient, practitioner or practice recruitment were assessed to determine the possible associations. There were more studies of patients than of practices or practitioners.ConclusionsFor practitioners and patients alike, a trial that is clinically relevant is critical in influencing participation. Competing demands are given as an important reason for declining participation. There are concerns about randomisation relating to its impact on shared decision-making and not knowing which treatment will be assigned. Patients make decisions about whether they are a candidate for the trial even when they objectively fulfil the eligibility criteria. General practice processes, such as difficulties arranging appointments, can hinder recruitment, and a strong pre-existing doctor-patient relationship can improve recruitment. For clinicians, the wish to contribute to the research enterprise itself is seldom an important reason for participating, though clinicians reported being motivated to participate when the research could improve their clinical practice. One of the few experimental findings was that opportunistic recruitment resulted in significantly faster recruitment compared to systematic recruitment. These factors have clear implications for trial design. Methodologically, recruitment research of practices and practitioners should have increased priority. Higher quality studies of recruitment are required to find out what actually works rather than what might work.Trial registrationPROSPERO CRD42018100695. Registered on 03 July 2018.

Project description:PURPOSE:The CONSORT extension for patient reported outcomes (PROs) aims to improve reporting, but guidance on the optimal integration with clinical data is lacking. This study examines in detail the reporting of PROs and clinical data from randomized controlled trials (RCTs) in gastro-intestinal cancer to inform design and reporting of combined PRO and clinical data from trials to improve the 'take home' message for clinicians to use in practice. MATERIALS AND METHODS:The case study was undertaken in gastro-intestinal cancer trials. Well-conducted RCTs reporting PROs with validated instruments were identified and categorized into those combining PRO and clinical data in a single paper, or those separating data into linked primary and supplemental papers. Qualitative methods were developed to examine reporting of the critical interpretation of the trial results (trial exegesis) in the papers in relation of the PRO and clinical outcomes and applied to each publication category. Results were used to inform recommendations for practice. RESULTS:From 1917 screened abstracts, 49 high quality RCTs were identified reported in 36 combined and 15 linked primary and supplemental papers. In-depth analysis of manuscript text identified three categories for understanding trial exegesis: where authors reported a "detailed", "general", or absent PRO rationale and integrated interpretation of clinical and PRO results. A total of 11 (30%) and 6 (16%) combined papers reported "detailed" PRO rationale and integrated interpretation of results although only 2 (14%) and 1 (7%) primary papers achieved the same standard respectively. Supplemental papers provide better information with 11 (73%) and 3 (20%) achieving "detailed" rationale and integrated interpretation of results. Supplemental papers, however, were published a median of 20 months after the primary RCT data in lower impact factor journals (median 16.8 versus 5.2). CONCLUSION:It is recommended that single papers, with detailed PRO rationale and integrated PRO and clinical data are published to optimize trial exegesis. Further work to examine whether this improves the use of PRO data to inform practice is needed.

Project description:ObjectivesThe objective of this exploratory study was to see how the Patient-Reported Apnea Questionnaire (PRAQ) may impact the daily clinical practice of sleep centres, and why it may or may not work as expected. The hypotheses were tested that this patient-reported outcome measure makes patients more aware of which of their health complaints may be related to obstructive sleep apnoea (OSA), and that it improves patient-centredness of care by shifting the focus of care away from (only) medical problems towards the individual burden of disease and quality of life.DesignMixed methods. The quantitative study (surveys, patient records) was a before-and-after study.SettingThree sleep centres in The Netherlands (secondary care).Participants27 patients and 14 healthcare professionals were interviewed. 487 patients completed surveys pre-implementation, and 377 patients completed surveys post-implementation of the PRAQ. For the health records, 125 patients were included in the pre-implementation group, and 124 other patients in the post-implementation group.InterventionsThe PRAQ was used in clinical practice for six successive months.Outcome measuresScores on individual survey items, number of patients receiving non-medical treatment, adjustment of treatment at first follow-up, compliance with treatment.ResultsPatients were generally positive about the usefulness of the PRAQ before and during the consultation, as they felt more informed. Healthcare providers did not consider the PRAQ very useful, and they reported minor impact on their consultations. The surveys and health record study did not show an impact of the PRAQ on clinical practice.ConclusionsImplementing the PRAQ may be beneficial to patients, but this study does not show much impact with regard to patient-centredness of care. New Dutch guidelines for OSA care may lead to a greater emphasis on quality of life and value of care for patients, making its integration in clinical care potentially more useful.

Project description:IntroductionAlthough the sustainability of the health workforce has been identified as essential to achieving health and wider development objectives, challenges with securing and retaining the healthcare workforce persist. In the UK, there are notable shortages across a wide range of National Health Service (NHS) staff groups, with a high staff turnover indicating retention issues in the healthcare workforce. In addition, gaps exist in understanding the root cause of individual organisation's workforce deficiencies and how their practice environment factors interact to impact workforce recruitment and retention.Methods and analysisAn exploratory mixed-methods approach will be conducted to investigate the impact of organisational practice environment factors on healthcare workforce recruitment and retention in two Integrated Care Systems (ICS) in the East of England. We will conduct an online survey of newly qualified and established nurses and allied health professionals using a questionnaire adapted from two validated instruments. Our calculation suggests a sample size of 373 participants, we will aim to surpass this in our recruitment to strengthen the statistical analyses. Multilevel linear regression models will be fitted to evaluate the association between organisational practice environmental factors and staff recruitment and retention. The qualitative interviews will explore the experiences and perspectives of staff and senior leaders to explain the survey results and any significant associations therein. Also, the interviews will explore how to strengthen the partnership between higher education institutions, Health Education England, health and care service providers, NHS nursing and allied health professional staff to enhance recruiting and retaining staff. An exploratory inductive coding and analysis will follow Braun and Clarke's recommendations to generate key themes from transcribed interview data.Ethics and disseminationEthical approval has been obtained through the University of Suffolk Research Ethics Committee (approval number: RETH(S)22/051). Findings from our work will be disseminated through publications in peer-reviewed journals; presentations at stakeholders' events, professional and academic conferences; and short reports for stakeholders, including participating ICSs.

Dataset Information

A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice.

Publications

A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets