Dataset Information

Evidence into practice: protocol for a new mixed-methods approach to explore the relationship between trials evidence and clinical practice through systematic identification and analysis of articles citing randomised controlled trials.

ABSTRACT:

Introduction

Randomised controlled trials (RCTs) provide high-quality evidence to inform practice. However, much routine care is not based on available RCT evidence. Understanding this disconnect may improve trial design, reporting and implementation. Published literature commenting on RCTs may yield relevant insights. This protocol presents a new approach examining how researchers understand, contextualise and use evidence from RCTs, through analysis of letters, editorials and discussion pieces citing individual RCTs. Surgical case studies will illustrate its ability to identify wide-ranging factors influencing application of trials evidence.

Methods and analysis

In-depth study of published literature will explore written responses to RCTs. After purposefully selecting individual RCTs, we will systematically identify all citing articles covered in Web of Science and Scopus. Editorials, discussions and letters will be included. These are considered most likely to provide critiques and opinions about index RCTs. Original articles and reviews will be excluded. Clinical specialty, RCT design, outcomes and bibliographical data will be collected for RCTs and citing articles. Citing articles will be thematically analysed using the constant comparison technique to explore author understanding, contextualisation and relationship to clinical practice for the index trial. Coding will include generic issues relevant to all RCTs, such as sample size or blinding, and features specific to surgery, such as learning curve. Index trial quality will be examined using validated tools. Results will be combined to create a broad overview of the understanding and use of RCT evidence.

Ethics and dissemination

This study involves secondary use of existing articles and does not require ethical approval. Pilot work will establish its feasibility and inform progression to larger scale utilisation across a broad range of RCTs. Findings will be published in a peer-reviewed journal and presented at surgical and methodological conferences. Results will guide future work on trial design to optimise implementation of results.

SUBMITTER: Byrne BE

PROVIDER: S-EPMC6231588 | biostudies-literature | 2018 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evidence into practice: protocol for a new mixed-methods approach to explore the relationship between trials evidence and clinical practice through systematic identification and analysis of articles citing randomised controlled trials.

Byrne Benjamin E BE Rooshenas Leila L Lambert Helen H Blazeby Jane M JM

BMJ open 20181108 11

<h4>Introduction</h4>Randomised controlled trials (RCTs) provide high-quality evidence to inform practice. However, much routine care is not based on available RCT evidence. Understanding this disconnect may improve trial design, reporting and implementation. Published literature commenting on RCTs may yield relevant insights. This protocol presents a new approach examining how researchers understand, contextualise and use evidence from RCTs, through analysis of letters, editorials and discussio ...[more]

PMID: 30413510

Similar Datasets

Project description:BackgroundThe h-index is increasingly being used as a measure of individual research achievement (IRA). More than 4876 citing articles have been published and indexed in Web of Science. The articles citing the h-index that have made the greatest contribution to scientific academics are still unknown. It is also unclear which subject categories (SCs) can be classified based on their keywords.MethodsThese 4976 citing articles have been collected from the Web of Science since 2005. SCs were classified using chord diagrams to visualize their associations of SCs and documents in 100 top-cited articles (T100hciting). In addition to chord diagrams, 6 visualizations were used to illustrate study results: choropleth maps were used to depict the geographical distribution of publications across countries, network diagrams were created by using coword analysis, box plots were created to complement the network diagrams, Sankey diagrams highlighted the 5 most important elements in each article entity, the dot plot was used for displaying T100hciting, and a radar plot was used to present the top 10 high-IRA elements of countries, institutes, departments, and authors based on category, journal impact factor, authorship, and L-index scores.ResultsA coword cluster analysis indicates that the majority of articles come from the US (918, 18%) and China (603, 12%), the top 2 SCs are h-index and bibliometric analysis, and the top 5 countries account for 55% in T100hciting, such as the US (25%), Spain (10%), Netherlands (9%), China (6%), and Belgium (5%). In T100hciting, 4 SCs are included, namely, the h-index (72%), bibliometric analysis (24%), physics & multidisciplinary (3%), and infectious diseases (1%).ConclusionA total of 7 visualizations were used to display the results in this study. Chord diagrams are suggested as a tool for future bibliographical studies to classify SCs Future bibliometrics with chord diagrams should not be limited to the topic of h-index-citing articles, as we did in this study.

Project description:BackgroundWhile patient and public involvement (PPI) in clinical trials is beneficial and mandated by some funders, formal guidance on how to implement PPI is limited and challenges have been reported. We aimed to investigate how PPI is approached within a UK Clinical Trials Unit (CTU)'s portfolio of randomised controlled trials, perceived barriers to/facilitators of its successful implementation, and perspectives on the CTU's role in PPI.MethodsA mixed-methods study design, involving (1) an online survey of 26 trial managers (TMs) and (2) Interviews with Trial Management Group members and public contributors from 8 case-study trials. Quantitative survey data were summarised using descriptive statistics and interview transcripts analysed thematically. Two public contributors advised throughout and are co-authors.Results(1) 21 TMs completed the survey; (2) 19 in-depth interviews were conducted with public contributors (n=8), TMs (n=5), chief investigators (n=3), PPI coordinators (n=2) and a researcher. 15/21 TMs surveyed reported that a public contributor was on the trial team, and 5 used another PPI method. 12/21 TMs reported that public contributors were paid (range £10-50/h). 5 TMs reported that training was provided for public contributors and few staff members had received any formal PPI training. The most commonly reported tasks undertaken by public contributors were the review of participant-facing materials/study documents and advising on recruitment/retention strategies. Public contributors wanted and valued feedback on changes made due to their input, but it was not always provided. Barriers to successful PPI included recruitment challenges, group dynamics, maintaining professional boundaries, negative attitudes to PPI amongst some researchers, a lack of continuity of trial staff, and the academic environment. Successful PPI required early and explicit planning, sharing of power and ownership of the trial with public contributors, building and maintaining relationships, and joint understanding and clarity about expectations/roles. CTUs have an important role to play in supporting recruitment, signposting and coordinating PPI.ConclusionsWhile highly valuable, PPI in trials is currently variable. PPI representatives are recruited informally, may not be provided with any training and are paid inconsistently across trials. Study findings can help optimise PPI in trials and ensure researchers and public contributors are adequately supported.

Project description:Methodologically sound mixed methods research can improve our understanding of health services by providing a more comprehensive picture of health services than either method can alone. This study describes the frequency of mixed methods in published health services research and compares the presence of methodological components indicative of rigorous approaches across mixed methods, qualitative, and quantitative articles.All empirical articles (n = 1,651) published between 2003 and 2007 from four top-ranked health services journals.All mixed methods articles (n = 47) and random samples of qualitative and quantitative articles were evaluated to identify reporting of key components indicating rigor for each method, based on accepted standards for evaluating the quality of research reports (e.g., use of p-values in quantitative reports, description of context in qualitative reports, and integration in mixed method reports). We used chi-square tests to evaluate differences between article types for each component.Mixed methods articles comprised 2.85 percent (n = 47) of empirical articles, quantitative articles 90.98 percent (n = 1,502), and qualitative articles 6.18 percent (n = 102). There was a statistically significant difference (?(2) (1) = 12.20, p = .0005, Cramer's V = 0.09, odds ratio = 1.49 [95% confidence interval = 1,27, 1.74]) in the proportion of quantitative methodological components present in mixed methods compared to quantitative papers (21.94 versus 47.07 percent, respectively) but no statistically significant difference (?(2) (1) = 0.02, p = .89, Cramer's V = 0.01) in the proportion of qualitative methodological components in mixed methods compared to qualitative papers (21.34 versus 25.47 percent, respectively).Few published health services research articles use mixed methods. The frequency of key methodological components is variable. Suggestions are provided to increase the transparency of mixed methods studies and the presence of key methodological components in published reports.

Project description:BackgroundRoutine data are increasingly used in randomised controlled trials evaluating healthcare interventions. They can aid participant identification, outcome assessment, and intervention delivery. Randomised implementation trials evaluate the effect of implementation strategies on implementation outcomes. Implementation strategies, such as reminders, are used to increase the uptake of evidence-based interventions into practice, while implementation outcomes, such as adoption, are key measures of the implementation process. The use of routine data in effectiveness trials has been explored; however, there are no reviews on implementation trials. We therefore aimed to describe how routine data have been used in randomised implementation trials and the design characteristics of these trials.MethodsWe searched MEDLINE (Ovid) and Cochrane Central Register of Controlled Trials from Jan 2000 to Dec 2021 and manually searched protocols from trial registers. We included implementation trials and type II and type III hybrid effectiveness-implementation trials conducted using routine data. We extracted quantitative and qualitative data and narratively synthesised findings.ResultsFrom 4206 titles, we included 80 trials, of which 22.5% targeted implementation of evidence-based clinical guidelines. Multicomponent implementation strategies were more commonly evaluated (70.0%) than single strategies. Most trials assessed adoption as the primary outcome (65.0%). The majority of trials extracted data from electronic health records (EHRs) (62.5%), and 91.3% used routine data for outcome ascertainment. Reported reasons for using routine data were increasing efficiency, assessing outcomes, reducing research burden, improving quality of care, identifying study samples, confirming findings, and assessing representativeness. Data quality, the EHR system, research governance, and external factors such as government policy could act either as facilitators or barriers.ConclusionsAdherence to guidance on designing and reporting implementation studies, and specifically to harmonise the language used in describing implementation strategies and implementation outcomes, would aid identification of studies and data extraction. Routine healthcare data are widely used for participant identification, outcome assessment and intervention delivery. Researchers should familiarise themselves with the barriers and facilitators to using routine data, and efforts could be made to improve data quality to overcome some of the barriers.RegistrationPROSPERO CRD42022292321.

Project description:BackgroundA common challenge for randomised controlled trials (RCTs) is recruiting enough participants to be adequately powered to answer the research question. Recruitment has been set as a priority research area in trials to improve recruitment and thereby reduce wasted resources in conducted trials that fail to recruit sufficiently.MethodsWe conducted a systematic mixed studies review to identify the factors associated with recruitment to RCTs in general practice. On September 8, 2020, English language studies were identified from MEDLINE, EMBASE, Cochrane Database of Systematic Reviews and CENTRAL databases for published studies. NTIS and OpenGrey were searched for grey literature, and BMC Trials was hand searched. A narrative synthesis was conducted for qualitative studies and a thematic synthesis for qualitative studies.ResultsThirty-seven studies met the inclusion criteria. These were of different study types (10 cross-sectional, 5 non-randomised studies of interventions, 2 RCTs, 10 qualitative and 10 mixed methods). The highest proportion was conducted in the UK (48%). The study quality was generally poor with 24 (65%) studies having major concerns. A complex combination of patient, practitioner or practice factors, and patient, practitioner or practice recruitment were assessed to determine the possible associations. There were more studies of patients than of practices or practitioners.ConclusionsFor practitioners and patients alike, a trial that is clinically relevant is critical in influencing participation. Competing demands are given as an important reason for declining participation. There are concerns about randomisation relating to its impact on shared decision-making and not knowing which treatment will be assigned. Patients make decisions about whether they are a candidate for the trial even when they objectively fulfil the eligibility criteria. General practice processes, such as difficulties arranging appointments, can hinder recruitment, and a strong pre-existing doctor-patient relationship can improve recruitment. For clinicians, the wish to contribute to the research enterprise itself is seldom an important reason for participating, though clinicians reported being motivated to participate when the research could improve their clinical practice. One of the few experimental findings was that opportunistic recruitment resulted in significantly faster recruitment compared to systematic recruitment. These factors have clear implications for trial design. Methodologically, recruitment research of practices and practitioners should have increased priority. Higher quality studies of recruitment are required to find out what actually works rather than what might work.Trial registrationPROSPERO CRD42018100695. Registered on 03 July 2018.

Project description:This protocol describes the 'development of outcome measures and suitable methodologies for dissemination and implementation approaches,' a priority for implementation research. Although many evidence-based practices (EBPs) have been developed, large knowledge gaps remain regarding how to routinely move EBPs into usual care. The lack of understanding of 'what it takes' to install EBPs has costly public health consequences, including a lack of availability of the most beneficial services, wasted efforts and resources on failed implementation attempts, and the potential for engendering reluctance to try implementing new EBPs after failed attempts.The Stages of Implementation Completion (SIC) is an eight-stage tool of implementation process and milestones, with stages spanning three implementation phases (pre-implementation, implementation, sustainability). Items delineate the date that a site completes implementation activities, yielding an assessment of duration (time to complete a stage), proportion (of stage activities completed), and a general measure of how far a site moved in the implementation process.We propose to extend the SIC to EBPs operating in child service sectors (juvenile justice, schools, substance use, child welfare). Both successful and failed implementation attempts will be scrutinized using a mixed methods design. Stage costs will be measured and examined. Both retrospective data (from previous site implementation efforts) and prospective data (from newly adopting sites) will be analyzed. The influence of pre-implementation on implementation and sustainability outcomes will be examined (Aim 1). Mixed methods procedures will focus on increasing understanding of the process of implementation failure in an effort to determine if the SIC can provide early detection of sites that are unlikely to succeed (Aim 2). Study activities will include cost mapping of SIC stages and an examination of the relationship between implementation costs and implementation performance (Aim 3).This project fills a gap in the field of implementation science by addressing the measurement gap between the implementation process and the associated costs. The goal of this project is to provide tools that will help increase the uptake of EBPs, thereby increasing the availability of services to youth and decreasing wasted resources from failed implementation efforts.

Project description:ImportanceSystematic reviews can help to justify a new randomized clinical trial (RCT), inform its design, and interpret its results in the context of prior evidence.ObjectiveTo assess trends and factors associated with citing (a marker of the use of) prior systematic reviews in RCT reports.Design, setting, and participantsThis cross-sectional study investigated 737 Cochrane reviews assessing health interventions to identify 4003 eligible RCTs, defined as those included in an updated version but not in the first version of a Cochrane review and published 2 years after the first version of the Cochrane review was published.Main outcomes and measuresThe primary outcome was the citation of prior systematic reviews, Cochrane or others, as determined by screening references of eligible RCTs. Factors that may be associated with the citation of prior systematic reviews were also examined.ResultsAmong 4003 eligible RCTs, 1241 studies (31.0%) cited Cochrane reviews, 1698 studies (42.4%) cited prior non-Cochrane reviews, and 2265 studies (56.6%) cited either type of systematic review or both; 1738 RCTs (43.4%) cited no systematic reviews. The percentage of RCTs citing prior Cochrane reviews, non-Cochrane reviews, and either or both types of review increased from 28 studies (15.3%), 46 studies (25.1%), and 65 studies (35.5%) of 183 RCTs before 2008 to 42 studies (40.8%), 65 studies (64.1%), and 73 studies (71.8%) of 102 RCTs since 2020, respectively; the annual increases were 1.9% (95% CI, 1.4%-2.3%), 3.3% (95% CI, 2.9%-3.7%), and 3.0% (95% CI, 2.5%-3.5%), respectively. The proportion of RCTs citating prior systematic reviews varied considerably across clinical specialties, ranging from 28 of 106 RCTs (26.4%) in ophthalmology to 386 of 553 RCTs (69.8%) in psychiatry (P < .001). RCTs with 100 participants or more (risk ratio [RR], 1.16; 95% CI, 1.03-1.30), nonindustry funding (RR, 1.43; 95% CI, 1.27-1.61), and authors from high-income countries (RR, 1.10; 95% CI, 1.03-1.17) were more likely to cite systematic reviews than those with fewer than 100 participants, industry funding, and authors from low- and middle-income countries, respectively. A journal requirement to cite systematic reviews was not associated with the likelihood of citing a systematic review.Conclusions and relevanceThis study found that the citation of prior systematic reviews in RCT reports improved over time, but approximately 40% of RCTs failed to do so. These findings suggest that reference to prior evidence for initiating, designing, and reporting RCTs should be further emphasized to assure clinical relevance, improve methodological quality, and facilitate interpretation of new results.

Project description:BackgroundFew studies have explored evidence-based practice (EBP) knowledge, attitudes and behaviours of health professional graduates transitioning into the workforce. This study evaluated changes in these EBP domains in physiotherapists after one year of working.MethodA mixed methods design was used. Participants completed two psychometrically-tested EBP questionnaires at two timepoints. The Evidence-Based Practice Profile questionnaire collected self-report EBP data (Terminology, Relevance, Confidence, Practice, Sympathy) and the Knowledge of Research Evidence Competencies collected objective data (Actual Knowledge). Changes were calculated using descriptive statistics (paired t-tests, 95% CI, effect sizes). Qualitative interview data collected at one timepoint were analysed using a descriptive approach and thematic analysis, to examine the lived experience of participants in the context of their first employment. The aim of the mixed methods approach was a broader and deeper understanding of participants' first year of employment and using EBP.ResultsData were analysed from 50 participants who completed both questionnaires at the two timepoints. After one year in the workforce, there was a significant decrease in participants' perceptions of Relevance (p<0.001) and Confidence with EBP (p<0.001) and non-significant decreases in the other domains. Effect sizes showed medium decreases for Relevance (0.69) and Confidence (0.57), small decreases in Terminology (0.28) and Practice (0.23), and very small decreases in Sympathy (0.08) and Actual Knowledge (0.11). Seven themes described participants experience of using EBP in their first working year.ConclusionsAfter a year in the workplace, confidence and perceptions of relevance of EBP were significantly reduced. A subtle interplay of features related to workplace culture, competing demands to develop clinical skills, internal and external motivators to use EBP and patient expectations, together with availability of resources and time, may impact early graduates' perceptions of EBP. Workplace role models who immersed themselves in evidence discussion and experience were inspiring to early graduates.

Dataset Information

Evidence into practice: protocol for a new mixed-methods approach to explore the relationship between trials evidence and clinical practice through systematic identification and analysis of articles citing randomised controlled trials.

Introduction

Methods and analysis

Ethics and dissemination

Publications

Evidence into practice: protocol for a new mixed-methods approach to explore the relationship between trials evidence and clinical practice through systematic identification and analysis of articles citing randomised controlled trials.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets