Dataset Information

Evaluation of the Cochrane Consumers and Communication Group's systematic review priority-setting project.

ABSTRACT:

Background

Health researchers and funders are increasingly consulting with stakeholders to set their research agendas but these activities are rarely evaluated. The Cochrane Consumers and Communication Group (CCCG) conducted a priority-setting project for systematic reviews in partnership with stakeholders (consumers/patients, health professionals, policy-makers and others). In this paper, we aim to describe our evaluation of the project's processes and outcomes.

Methods

We used a 10-element conceptual framework designed to evaluate processes (e.g. stakeholder engagement, use of explicit process) and outcomes (e.g. improved decision-making quality, stakeholder acceptance and understanding) of health priority-setting. Data sources included empirical data (feedback surveys, project documents and CCCG editorial policies) and CCCG staff reflections. Data were analysed using content analysis.

Results

The project met three and partially met two of the process elements, for example, by engaging key stakeholders throughout the project and using pre-determined and transparent methods that offered multiple and meaningful ways to contribute. The project met three and partially met two of the outcome elements. Stakeholders were satisfied with and accepted the process and an additional six Cochrane Review titles aligned with stakeholder priorities are now being conducted in partnership with stakeholders. The project has also directly influenced the editorial work of CCCG, for example, by shifting its organisational focus towards coproduction, and indirectly influenced the work of Cochrane's prioritisation and coproduction activities. Some areas were identified as having room for improvement, for example, there was low participation by people from diverse backgrounds, stakeholders could contribute to most but not all project stages, and there was no formal way for stakeholders to appeal decisions at project end. In the 3 years since its completion, the Cochrane Reviews are nearing completion but none of the reviews have been published.

Conclusion

We demonstrated that our priority-setting methods were broadly in line with best practice and the project resulted in many positive outcomes beyond just identifying the top priorities for research. Our evaluation framework and recommendations for future evaluations may be of use to priority-setting researchers planning similar activities.

SUBMITTER: Synnot A

PROVIDER: S-EPMC7465879 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPriority-setting partnerships between researchers and stakeholders (meaning consumers, health professionals and health decision-makers) may improve research relevance and value. The Cochrane Consumers and Communication Group (CCCG) publishes systematic reviews in 'health communication and participation', which includes concepts such as shared decision-making, patient-centred care and health literacy. We aimed to select and refine priority topics for systematic reviews in health communication and participation, and use these to identify five priority CCCG Cochrane Reviews.MethodsTwenty-eight participants (14 consumers, 14 health professionals/decision-makers) attended a 1-day workshop in Australia. Using large-group activities and voting, participants discussed, revised and then selected 12 priority topics from a list of 21 previously identified topics. In mixed small groups, participants refined these topics, exploring underlying problems, who they affect and potential solutions. Thematic analysis identified cross-cutting themes, in addition to key populations and potential interventions for future Cochrane Reviews. We mapped these against CCCG's existing review portfolio to identify five priority reviews.ResultsPriority topics included poor understanding and implementation of patient-centred care by health services, the fact that health information can be a low priority for health professionals, communication and coordination breakdowns in health services, and inadequate consumer involvement in health service design. The four themes underpinning the topics were culture and organisational structures, health professional attitudes and assumptions, inconsistent experiences of care, and lack of shared understanding in the sector. Key populations for future reviews were described in terms of social health characteristics (e.g. people from indigenous or culturally and linguistically diverse backgrounds, elderly people, and people experiencing socioeconomic disadvantage) more than individual health characteristics. Potential interventions included health professional education, interventions to change health service/health professional culture and attitudes, and health service policies and standards. The resulting five priority Cochrane Reviews identified were improving end-of-life care communication, patient/family involvement in patient safety, improving future doctors' communication skills, consumer engagement strategies, and promoting patient-centred care.ConclusionsStakeholders identified priority topics for systematic reviews associated with structural and cultural challenges underlying health communication and participation, and were concerned that issues of equity be addressed. Priority-setting with stakeholders presents opportunities and challenges for review producers.

Project description:Current research suggests that while patients are becoming more engaged across the health delivery spectrum, this involvement occurs most often at the pre-preparation stage to identify 'high-level' priorities in health ecosystem priority setting, and at the preparation phase for health research.The purpose of this systematic rapid review of the literature is to describe the evidence that does exist in relation to patient and public engagement priority setting in both health ecosystem and health research.HealthStar (via OVID); CINAHL; Proquest Databases; and Scholar's Portal.i) published in English; ii) published within the timeframe of 2007-Current (10 years) unless the report/article was formative in synthesizing key considerations of patient engagement in health ecosystem and health research priority setting; iii) conducted in Canada, the US, Europe, UK, Australia/New Zealand, or Scandinavian countries.i) Is the research valid, sound, and applicable?; ii) what outcomes can we potentially expect if we implement the findings from this research?; iii) will the target population (i.e., health researchers and practitioners) be able to use this research?. A summary of findings from each of the respective processes was synthesized to highlight key information that would support decision-making for researchers when determining the best priority setting process to apply for their specific patient-oriented research.Seventy articles from the UK, US, Canada, Netherlands and Australia were selected for review. Results were organized into two tiers of public and patient engagement in prioritization: Tier 1-Deliberative and Tier 2-Consultative. Highly structured patient and public engagement planning activities include the James Lind Alliance Priority Setting Partnerships (UK), Dialogue Method (Netherlands), Global Evidence Mapping (Australia), and the Deep Inclusion Method/CHoosing All Together (US).The critical study limitations include challenges in comprehensively identifying the patient engagement literature for review, bias in article selection due to the identified scope, missed information due to a more limited use of exhaustive search strategies (e.g., in-depth hand searching), and the heterogeneity of reported study findings.The four public and patient engagement priority setting processes identified were successful in setting priorities that are inclusive and objectively based, specific to the priorities of stakeholders engaged in the process. The processes were robust, strategic and aimed to promote equity in patient voices. Key limitations identified a lack of evaluation data on the success and extent in which patients were engaged. Issues pertaining to feasibility of stakeholder engagement, coordination, communication and limited resources were also considered.

Project description:BackgroundThis scoping review provides a thorough analysis of how stakeholders have so far been involved in research priority setting. The review describes, synthesizes, and evaluates research priority setting projects not only for the field of health-as previous reviews have done-but does so on a much broader scale for any research area.MethodsA comprehensive electronic literature search was conducted in the databases PubMed, Scopus, and Web of Science. Reflecting the importance of grey literature, Google Scholar and relevant websites were also screened for eligible publications. A computational approach was then used for the study selection. The final screening for inclusion was done manually.ResultsThe scoping review encompasses 731 research priority setting projects published until the end of 2020. Overall, the projects were conducted within the realm of 50 subject areas ranging from agriculture and environment over health to social work and technology. Key learnings include that nearly all priority setting projects aimed to identify research priorities for the field of health (93%), particularly for nursing and care, cancer, pediatrics, and mental, behavioral and neurodevelopmental disorders. Only 6% of the projects were not health-related and 1% identified research priorities at the interface between health and a non-health area. Over time, 30 different stakeholder groups took part in research priority setting. The stakeholders most frequently asked to identify research priorities were doctors, patients, academics/researchers, nurses, allied healthcare professionals, family members, friends, and carers. Nearly two thirds of all projects have been conducted in Europe and North America. Overall, only 9% of the projects emphasized the importance of stakeholders in their goals and rationales and actively involved them. In around a quarter of the projects, stakeholders deliberated on their research priorities throughout the entire process.ConclusionBy mapping out the complex landscape of stakeholder involvement in research priority setting, this review guides future efforts to involve stakeholders effectively, inclusively, and transparently, which in turn may increase the overall value of research for society. As a practical addition to this review, the first worldwide research priority setting database was created: https://ois.lbg.ac.at/en/project-database . The database contains all the projects analyzed for this review and is constantly updated with the latest published research priority setting projects.

Project description:BackgroundThere is an unmet need for review methods to support priority-setting, policy-making and strategic planning when a wide variety of interventions from differing disciplines may have the potential to impact a health outcome of interest. This article describes a Modular Literature Review, a novel systematic search and review method that employs systematic search strategies together with a hierarchy-based appraisal and synthesis of the resulting evidence.MethodsWe designed the Modular Review to examine the effects of 43 interventions on a health problem of global significance. Using the PICOS (Population, Intervention, Comparison, Outcome, Study design) framework, we developed a single four-module search template in which population, comparison and outcome modules were the same for each search and the intervention module was different for each of the 43 interventions. A series of literature searches were performed in five databases, followed by screening, extraction and analysis of data. "ES documents", source documents for effect size (ES) estimates, were systematically identified based on a hierarchy of evidence. The evidence was categorised according to the likely effect on the outcome and presented in a standardised format with quantitative effect estimates, meta-analyses and narrative reporting. We compared the Modular Review to other review methods in health research for its strengths and limitations.ResultsThe Modular Review method was used to review the impact of 46 antenatal interventions on four specified birth outcomes within 12 months. A total of 61,279 records were found; 35,244 were screened by title-abstract. Six thousand two hundred seventy-two full articles were reviewed against the inclusion criteria resulting in 365 eligible articles.ConclusionsThe Modular Review preserves principles that have traditionally been important to systematic reviews but can address multiple research questions simultaneously. The result is an accessible, reliable answer to the question of "what works?". Thus, it is a well-suited literature review method to support prioritisation, decisions and planning to implement an agenda for health improvement.

Project description:BackgroundPoor communication skills can potentially compromise patient care. However, as communication skills training (CST) programs are not seen as a priority to many clinical departments, there is a discernible absence of a standardised, recommended framework for these programs to be built upon. This systematic scoping review (SSR) aims to gather prevailing data on existing CSTs to identify key factors in teaching and assessing communication skills in the postgraduate medical setting.MethodsIndependent searches across seven bibliographic databases (PubMed, PsycINFO, EMBASE, ERIC, CINAHL, Scopus and Google Scholar) were carried out. Krishna's Systematic Evidence-Based Approach (SEBA) was used to guide concurrent thematic and content analysis of the data. The themes and categories identified were compared and combined where possible in keeping with this approach and then compared with the tabulated summaries of the included articles.ResultsTwenty-five thousand eight hundred ninety-four abstracts were identified, and 151 articles were included and analysed. The Split Approach revealed similar categories and themes: curriculum design, teaching methods, curriculum content, assessment methods, integration into curriculum, and facilitators and barriers to CST. Amidst a wide variety of curricula designs, efforts to develop the requisite knowledge, skills and attitudes set out by the ACGME current teaching and assessment methods in CST maybe categorised into didactic and interactive methods and assessed along Kirkpatrick's Four Levels of Learning Evaluation.ConclusionsA major flaw in existing CSTs is a lack of curriculum structure, focus and standardisation. Based upon the findings and current design principles identified in this SSR in SEBA, we forward a stepwise approach to designing CST programs. These involve 1) defining goals and learning objectives, 2) identifying target population and ideal characteristics, 3) determining curriculum structure, 4) ensuring adequate resources and mitigating barriers, 5) determining curriculum content, and 6) assessing learners and adopting quality improvement processes.

Project description:BackgroundBlack, Asian and minority ethnic communities suffer from disproportionately poorer health than the general population. This issue has been recently exemplified by the large numbers of infection rates and deaths caused by covid-19 in BAME populations. Future research has the potential to improve health outcomes for these groups. High quality research priority setting is crucial to effectively consider the needs of the most vulnerable groups of the population.ObjectiveThe purpose of this systematic review is to identify existing research priority studies conducted for BAME health and to determine the extent to which they followed good practice principles for research priority setting.MethodIncluded studies were identified by searching Medline, Cinnahl, PsychINFO, Psychology and Behavioral Sciences Collection, as well as searches in grey literature. Search terms included "research priority setting", "research prioritisation", "research agenda", "Black and minority ethnic", "ethnic group". Studies were included if they identified or elicited research priorities for BAME health and if they outlined a process of conducting a research prioritisation exercise. A checklist of Nine Common Themes of Good Practice in research priority setting was used as a methodological framework to evaluate the research priority processes of each study.ResultsOut of 1514 citations initially obtained, 17 studies were included in the final synthesis. Topic areas for their research prioritisation exercise included suicide prevention, knee surgery, mental health, preterm birth, and child obesity. Public and patient involvement was included in eleven studies. Methods of research prioritisation included workshops, Delphi techniques, surveys, focus groups and interviews. The quality of empirical evidence was diverse. None of the exercises followed all good practice principles as outlined in the checklist. Areas that were lacking in particular were: the lack of a comprehensive approach to guide the process; limited use of criteria to guide discussion around priorities; unequal or no representation from ethnic minorities, and poor evaluation of their own processes.ConclusionsResearch priority setting practices were found to mostly not follow good practice guidelines which aim to ensure rigour in priority setting activities and support the inclusion of BAME communities in establishing the research agenda. Research is unlikely to deliver useful findings that can support relevant research and positive change for BAME communities unless they fulfil areas of good practice such as inclusivity of key stakeholders' input, planning for implementation of identified priorities, criteria for deciding on priorities, and evaluation of their processes in research priority setting.

Project description:OBJECTIVE:Describe research methods used in priority-setting exercises for musculoskeletal conditions and synthesise the priorities identified. DESIGN:Scoping review. SETTING AND POPULATION:Studies that elicited the research priorities of patients/consumers, clinicians, researchers, policy-makers and/or funders for any musculoskeletal condition were included. METHODS AND ANALYSIS:We searched MEDLINE and EMBASE from inception to November 2017 and the James Lind Alliance top 10 priorities, Cochrane Priority Setting Methods Group, and Cochrane Musculoskeletal and Back Groups review priority lists. The reported methods and research topics/questions identified were extracted, and a descriptive synthesis conducted. RESULTS:Forty-nine articles fulfilled our inclusion criteria. Methodologies and stakeholders varied widely (26 included a mix of clinicians, consumers and others, 16 included only clinicians, 6 included only consumers or patients and in 1 participants were unclear). Only two (4%) reported any explicit inclusion criteria for priorities. We identified 294 broad research priorities from 37 articles and 246 specific research questions from 17 articles, although only four (24%) of the latter listed questions in an actionable format. Research priorities for osteoarthritis were identified most often (n=7), followed by rheumatoid arthritis (n=4), osteoporosis (n=4) and back pain (n=4). Nearly half of both broad and specific research priorities were focused on treatment interventions (n=116?and 111, respectively), while few were economic (n=8, 2.7% broad and n=1, 0.4% specific), implementation (n=6, 2% broad and n=4, 1.6% specific) or health services and systems research (n=15, 5.1% broad and n=9, 3.7% specific) priorities. CONCLUSIONS:While many research priority-setting studies in the musculoskeletal field have been performed, methodological limitations and lack of actionable research questions limit their usefulness. Future studies should ensure they conform to good priority-setting practice to ensure that the generated priorities are of maximum value. PROSPERO REGISTRATION NUMBER:CRD42017059250.