Dataset Information

The generation of consensus guidelines for carrying out process evaluations in rehabilitation research.

ABSTRACT:

Background

Although in recent years there has been a strong increase in published research on theories (e.g. realist evaluation, normalization process theory) driving and guiding process evaluations of complex interventions, there is limited guidance to help rehabilitation researchers design and carry out process evaluations. This can lead to the risk of process evaluations being unsystematic. This paper reports on the development of new consensus guidelines that address the specific challenges of conducting process evaluations alongside clinical trials of rehabilitation interventions.

Methods

A formal consensus process was carried out based on a modified nominal group technique, which comprised two phases. Phase I was informed by the findings of a systematic review, and included a nominal group meeting with an expert panel of participants to rate and discuss the proposed statements. Phase II was an in depth semi-structured telephone interviews with expert panel participants in order to further discuss the structure and contents of the revised guidelines. Frequency of rating responses to each statement was calculated and thematic analysis was carried out on all qualitative data.

Results

The guidelines for carrying out process evaluations within complex intervention rehabilitation research were produced by combining findings from Phase I and Phase II. The consensus guidelines include recommendations that are grouped in seven sections. These sections are theoretical work, design and methods, context, recruitment and retention, intervention staff, delivery of the intervention and results. These sections represent different aspects or stages of the evaluation process.

Conclusion

The consensus guidelines here presented can play a role at assisting rehabilitation researchers at the time of designing and conducting process evaluations alongside trials of complex interventions. The guidelines break new ground in terms of concepts and theory and works towards a consensus in regards to how rehabilitation researchers should go about carrying out process evaluations and how this evaluation should be linked into the proposed trials. These guidelines may be used, adapted and tested by rehabilitation researchers depending on the research stage or study design (e.g. feasibility trial, pilot trial, etc.).

SUBMITTER: Masterson-Algar P

PROVIDER: S-EPMC6311071 | biostudies-literature | 2018 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The generation of consensus guidelines for carrying out process evaluations in rehabilitation research.

Masterson-Algar P P Burton C R CR Rycroft-Malone J J

BMC medical research methodology 20181229 1

<h4>Background</h4>Although in recent years there has been a strong increase in published research on theories (e.g. realist evaluation, normalization process theory) driving and guiding process evaluations of complex interventions, there is limited guidance to help rehabilitation researchers design and carry out process evaluations. This can lead to the risk of process evaluations being unsystematic. This paper reports on the development of new consensus guidelines that address the specific cha ...[more]

PMID: 30594133

Similar Datasets

Project description:BackgroundExperimental designs for evaluating knowledge translation (KT) interventions can provide strong estimates of effectiveness but offer limited insight into how the intervention worked. Consequently, process evaluations have been used to explore the causal mechanisms at work; however, there are limited standards to guide this work. This study synthesizes current evidence of KT process evaluations to provide future methodological recommendations.MethodsPeer-reviewed search strategies were developed by a health research librarian. Studies had to be in English, published since 1996, and were not excluded based on design. Studies had to (1) be a process evaluation of a KT intervention study in primary health, (2) be a primary research study, and (3) include a licensed healthcare professional delivering or receiving the intervention. A two-step, two-person hybrid screening approach was used for study inclusion with inter-rater reliability ranging from 94 to 95%. Data on study design, data collection, theoretical influences, and approaches used to evaluate the KT intervention, analysis, and outcomes were extracted by two reviewers. Methodological quality was assessed with the Mixed Methods Appraisal Tool (MMAT).ResultsOf the 20,968 articles screened, 226 studies fit our inclusion criteria. The majority of process evaluations used qualitative forms of data collection (43.4%) and individual interviews as the predominant data collection method. 72.1% of studies evaluated barriers and/or facilitators to implementation. 59.7% of process evaluations were stand-alone evaluations. The timing of data collection varied widely with post-intervention data collection being the most frequent (46.0%). Only 38.1% of the studies were informed by theory. Furthermore, 38.9% of studies had MMAT scores of 50 or less indicating poor methodological quality.ConclusionsThere is widespread acceptance that the generalizability of quantitative trials of KT interventions would be significantly enhanced through complementary process evaluations. However, this systematic review found that process evaluations are of mixed quality and lack theoretical guidance. Most process evaluation data collection occurred post-intervention undermining the ability to evaluate the process of implementation. Strong science and methodological guidance is needed to underpin and guide the design and execution of process evaluations in KT science.RegistrationThis study is not registered with PROSPERO.

Project description:PurposeHip fracture is common in older people - with prevalence even higher for people with dementia. Research often excludes people with dementia - especially those in the more advanced stages. Therefore, the most appropriate interventions remain unknown. The main aim of this study was to gain consensus about the core considerations needed to deliver a physical intervention for people with advanced dementia who fracture their hip. Materials and Methods: An expert consensus process was undertaken, using Nominal Group Technique, to explore the key considerations when delivering rehabilitation. Data collection was undertaken in January 2023 and involved an online group discussion followed by voting and off-line rating. Qualitative content analysis and quantitative analysis of consensus scoring was undertaken. An international group of seven highly specialised physiotherapists took part.Results59 statements were agreed following the process. Content analysis was used to categorise these statements according to the International Classification of Functioning, Disability and Health. Although consensus levels were high, there was disagreement in several areas.ConclusionThe statements provide an overarching understanding of the multidisciplinary expertise that is needed to effectively deliver rehabilitation interventions to this population. People with dementia require highly skilled and trained professionals, providing holistic and person-centred approaches to deliver rehabilitation interventions.IMPLICATIONS FOR REHABILITATIONThe expert consensus provides an overarching understanding of the multidisciplinary expertise that is needed to effectively deliver rehabilitation interventions to this population.Physiotherapy - or other interventions - cannot be used in isolation.People with dementia require highly skilled and trained professionals, providing holistic and person-centred approaches to deliver rehabilitation interventions.While our focus was on hip fracture, we suggest these statements can be used for people with advanced dementia with a variety of other conditions.

Project description:BACKGROUND:Marginalized communities often attract more than their share of research. Too often, this research benefits researchers disproportionately and leaves such communities feeling exploited, misrepresented, and exhausted. The Downtown Eastside (DTES) neighborhood of Vancouver, Canada, has been the site of multiple public health epidemics related to injection drug use as well as the site of much community-led resistance and struggle that has led to the development of cutting-edge harm reduction interventions (e.g., North America's first supervised injection facility, Insite) and a strong sense of community organization. This background has made the DTES one of the most heavily researched communities in the world. Amidst ongoing experiences of unethical or disrespectful research engagement in the neighborhood, a collaboration between local academic researchers and community representatives developed to explore how we could work together to encourage more respectful, community-responsive research and discourage exploitative or disrespectful research. METHODS:We developed a series of six weekly workshops called "Research 101." These workshops brought together approximately 13 representatives from peer-based organizations in the DTES with a variety of experiences with research. Research 101 created space for community members themselves to discuss the pitfalls and potential of research in their neighborhood and to express community expectations for more ethical and respectful research. RESULTS:We summarized workshop discussions in a co-authored "Manifesto for Ethical Research in the Downtown Eastside." This document serves as a resource to empower community organizations to develop more equitable partnerships with researchers and help researchers ground their work in the principles of locally developed "community ethics." Manifesto guidelines include increased researcher transparency, community-based ethical review of projects, empowering peer researchers in meaningful roles within a research project, and taking seriously the need for reciprocity in the research exchange. CONCLUSIONS:Research 101 was a process for eliciting and presenting a local vision of "community ethics" in a heavily researched neighborhood to guide researchers and empower community organizations. Our ongoing work involves building consensus for these guidelines within the community and communicating these expectations to researchers and ethics offices at local universities. We also describe how our Research 101 process could be replicated in other heavily researched communities.

Project description:BackgroundEmbedded implementation research (IR) promotes evidence-informed policy and practices by involving decision-makers and program implementers in research activities that focus on understanding and solving existing implementation challenges. Although embedded IR has been conducted in multiple settings by different organizations, there are limited experiences of embedded IR in humanitarian settings. This study highlights some of the key challenges of conducting embedded IR in a humanitarian setting based on our experience with the Rohingya refugee population in Cox's Bazar, Bangladesh.Methods and findingsWe collected qualitative data in between January and July 2019. First, we visited Rohingya refugee camps and interviewed representatives from different humanitarian organizations. Second, we conducted interviews with researchers from BRAC University who were engaged with data collection and analysis in a broader embedded IR study on maternal, newborn, child, and adolescent health (MNCAH) program implementation challenges. Data were analyzed using a thematic analysis approach. Two researchers developed and agreed on codes and relevant themes based on the objectives of this study. The findings of this study highlight several challenges encountered while conducting embedded IR in the Rohingya emergency setting in Cox's Bazar, which may have implications for other humanitarian settings. The overall context of the camps was complex, with more than 100 organizations devoted to providing health services for approximately 1 million refugees. Despite the presence of the Bangladesh government, United Nations agencies and other international organizations played key roles in making programmatic and policy decisions for the Rohingya. Because health service delivery modalities and policies and related implementation challenges for MNCAH programs for the refugees changed rapidly, the embedded IR approach used was flexible and able to adapt to changes identified, with research questions and methods modified accordingly. Access to the camps, reaching Rohingya respondents, overcoming language barriers in order to get quality information, and the limited availability of local research collaborators were additional challenges. Working with researchers or research institutes that are familiar with the context and have experience in conducting implementation and health systems research can help with collection of quality data, identifying key stakeholders and bringing them on board to ensure the execution of the project, and ensuring utilization of the research findings. Study limitations include possible constraints in generalizing our conclusions to other humanitarian settings. Implementation research conducted in additional humanitarian settings can contribute to the evidence on this topic.ConclusionsFindings indicate that embedded IR can be done effectively in humanitarian settings if the challenges are anticipated, and appropriate strategies and in-country partners put in place to address or mitigate them, before commencing the funding or starting of the project. Understanding the context and analyzing the role of relevant stakeholders prior to conducting the research, considering a simple descriptive method appropriate to answering real-time IR questions, and working with local researchers or research institutes with specific skill sets and prior experience conducting research in humanization contexts may reduce costs and time spent, and ensure collection of quality data relevant for policy and practice.

Project description:Introduction: Growing evidence suggests that acupuncture can improve pelvic pain in women with endometriosis. The treatments used in research vary considerably. It remains unclear which treatment could be recommended for clinical practice. This research project aimed at clarifying how acupuncture could be used when treating this condition. Methods: This research comprised two phases: a systematized literature review to extract acupuncture treatment details from published research, and an e-Delphi study to gain knowledge about details as used by expert acupuncturists. Review: Four databases were searched using predefined eligibility criteria. Data were extracted based on the STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) criteria. e-Delphi: Purposeful sampling from colleagues and international experts. An open first round gathered qualitative data, analyzed with the Framework method. In rounds 2 and 3, experts rated statements to build group consensus, defined as a rating of ≥5 on a 7-point Likert scale by ≥70% of the experts. The strength of agreement was graded using the median score and interquartile range. Results from the literature review and the e-Delphi were compared using the STRICTA items. Results: The literature review (n = 29 unique studies) found a wide range of treatment details with little agreement. The e-Delphi of international experts (n = 20) resulted in agreement on 94 statements (such as key factors for effectiveness); disagreement on a further 29 (such as acupressure); and absence of consensus on 55 statements (such as the number of needle insertions). A comparison of the review and e-Delphi results found little agreement. Conclusions: Details of acupuncture treatment for endometriosis-related pelvic pain were presented. In the absence of acupuncture guidelines for this condition, the researchers of this e-Delphi recommend using the treatment details on which experts agreed as guidance for good practice. The effectiveness of these guidelines should be evaluated in future research. Study registration: Deutsches Register Klinischer Studien, DRKS00022215, June 30, 2020, retrospectively registered.

Dataset Information

The generation of consensus guidelines for carrying out process evaluations in rehabilitation research.

Background

Methods

Results

Conclusion

Publications

The generation of consensus guidelines for carrying out process evaluations in rehabilitation research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets