Dataset Information

Assessing the impact of healthcare research: A systematic review of methodological frameworks.

ABSTRACT:

Background

Increasingly, researchers need to demonstrate the impact of their research to their sponsors, funders, and fellow academics. However, the most appropriate way of measuring the impact of healthcare research is subject to debate. We aimed to identify the existing methodological frameworks used to measure healthcare research impact and to summarise the common themes and metrics in an impact matrix.

Methods and findings

Two independent investigators systematically searched the Medical Literature Analysis and Retrieval System Online (MEDLINE), the Excerpta Medica Database (EMBASE), the Cumulative Index to Nursing and Allied Health Literature (CINAHL+), the Health Management Information Consortium, and the Journal of Research Evaluation from inception until May 2017 for publications that presented a methodological framework for research impact. We then summarised the common concepts and themes across methodological frameworks and identified the metrics used to evaluate differing forms of impact. Twenty-four unique methodological frameworks were identified, addressing 5 broad categories of impact: (1) 'primary research-related impact', (2) 'influence on policy making', (3) 'health and health systems impact', (4) 'health-related and societal impact', and (5) 'broader economic impact'. These categories were subdivided into 16 common impact subgroups. Authors of the included publications proposed 80 different metrics aimed at measuring impact in these areas. The main limitation of the study was the potential exclusion of relevant articles, as a consequence of the poor indexing of the databases searched.

Conclusions

The measurement of research impact is an essential exercise to help direct the allocation of limited research resources, to maximise research benefit, and to help minimise research waste. This review provides a collective summary of existing methodological frameworks for research impact, which funders may use to inform the measurement of research impact and researchers may use to inform study design decisions aimed at maximising the short-, medium-, and long-term impact of their research.

SUBMITTER: Cruz Rivera S

PROVIDER: S-EPMC5549933 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:Psychodrama is an experiential psychotherapy in which guided role-play is used to gain insights and work on personal and interpersonal problems and possible solutions. Despite the wealth of literature describing clinical work, psychodrama intervention research is relatively scarce compared to other psychotherapies and psychological interventions. OBJECTIVE:For this reason we implemented the integrative approach to systematic review that authorizes the combination of publications with diverse methodologies and all types of participants, interventions, comparisons, and outcomes. Our aim was to produce a comprehensive summary of psychodrama intervention research in the last decade that critically evaluates methodological issues to inform future studies. METHODS:We searched four major electronic databases (PsycINFO, PubMEd, Scopus by Elsevier, and Web of Science) for peer-reviewed articles on psychodrama interventions published in English between 1 January 2007 and 31 December 2017. The quality of qualitative and mixed methods studies was assessed on the basis of pre-established guidelines, and the risk of bias was assessed for all quantitative randomized control studies, consistent with the PRISMA protocol. FINDINGS:The database search and a hand search resulted in 31 psychodrama intervention publications. Overall, these studies examined the effects of psychodrama on more than 20 different outcomes and most studies had adult clients. The next largest group was adolescents, whereas only two studies involved children. Thus psychodrama intervention research in the last decade suggests there are promising results in all methodologies, and highlights the need to enhance methodological as well as reporting quality and to theorize and examine modality-specific mechanisms that lead to therapeutic change. Recommendations to improve methodology, transparency, and specificity in reporting future psychodrama and other psychotherapy research are discussed.

Project description:Qualitative systematic reviews or qualitative evidence syntheses (QES) are increasingly recognised as a way to enhance the value of systematic reviews (SRs) of clinical trials. They can explain the mechanisms by which interventions, evaluated within trials, might achieve their effect. They can investigate differences in effects between different population groups. They can identify which outcomes are most important to patients, carers, health professionals and other stakeholders. QES can explore the impact of acceptance, feasibility, meaningfulness and implementation-related factors within a real world setting and thus contribute to the design and further refinement of future interventions. To produce valid, reliable and meaningful QES requires systematic identification of relevant qualitative evidence. Although the methodologies of QES, including methods for information retrieval, are well-documented, little empirical evidence exists to inform their conduct and reporting.This structured methodological overview examines papers on searching for qualitative research identified from the Cochrane Qualitative and Implementation Methods Group Methodology Register and from citation searches of 15 key papers.A single reviewer reviewed 1299 references. Papers reporting methodological guidance, use of innovative methodologies or empirical studies of retrieval methods were categorised under eight topical headings: overviews and methodological guidance, sampling, sources, structured questions, search procedures, search strategies and filters, supplementary strategies and standards.This structured overview presents a contemporaneous view of information retrieval for qualitative research and identifies a future research agenda. This review concludes that poor empirical evidence underpins current information practice in information retrieval of qualitative research. A trend towards improved transparency of search methods and further evaluation of key search procedures offers the prospect of rapid development of search methods.

Project description:BackgroundBuilding the capacity of allied health professionals to engage in research has been recognised as a priority due to the many benefits it brings for patients, healthcare professionals, healthcare organisations and society more broadly. There is increasing recognition of the need for a coordinated multi-strategy approach to building research capacity. The aim of this systematic review was to identify existing integrated models and frameworks which guide research capacity building for allied health professionals working in publicly funded secondary and tertiary healthcare organisations.MethodsA systematic review was undertaken searching five databases (Medline, CINAHL, Embase, AustHealth and Web of Science) using English language restrictions. Two authors independently screened and reviewed studies, extracted data and performed quality assessments using the Mixed Methods Appraisal Tool. Content and thematic analysis methods were used to code and categorise the data.ResultsA total of 8492 unique records were screened by title and abstract, of which 20 were reviewed in full-text. One quantitative study and five qualitative studies were included, each of which describing a research capacity building framework. Three interconnected and interdependent themes were identified as being essential for research capacity building, including 'supporting clinicians in research', 'working together' and 'valuing research for excellence'.ConclusionsThe findings of this systematic review have been synthesised to develop a succinct and integrated framework for research capacity building which is relevant for allied health professionals working in publicly funded secondary and tertiary healthcare organisations. This framework provides further evidence to suggest that research capacity building strategies are interlinked and interdependent and should be implemented as part of an integrated 'whole of system' approach, with commitment and support from all levels of leadership and management. Future directions for research include using behaviour change and knowledge translation theories to guide the implementation and evaluation of this new framework.Trial registrationThe protocol for this systematic review has been registered with PROSPERO. The registration number is CRD42018087476 .

Project description:Objective: To identify processes to engage stakeholders in healthcare Simulation Modeling (SM), and the impacts of this engagement on model design, model implementation, and stakeholder participants. To investigate how engagement process may lead to specific impacts. Data Sources: English-language articles on health SM engaging stakeholders in the MEDLINE, EMBASE, Scopus, Web of Science and Business Source Complete databases published from inception to February 2020. Study Design: A systematic review of the literature based on a priori protocol and reported according to PRISMA guidelines. Extraction Methods: Eligible articles were SM studies with a health outcome which engaged stakeholders in model design. Data were extracted using a data extraction form adapted to be specific for stakeholder engagement in SM studies. Data were analyzed using summary statistics, deductive and inductive content analysis, and narrative synthesis. Principal Findings: Thirty-two articles met inclusion criteria. Processes used to engage stakeholders in healthcare SM are heterogenous and often based on intuition rather than clear methodological frameworks. These processes most commonly involve stakeholders across multiple SM stages via discussion/dialogue, interviews, workshops and meetings. Key reported impacts of stakeholder engagement included improved model quality/accuracy, implementation, and stakeholder decision-making. However, for all but four studies, these reports represented author perceptions rather than formal evaluations incorporating stakeholder perspectives. Possible process enablers of impact included the use of models as "boundary objects" and structured facilitation via storytelling to promote effective communication and mutual understanding between stakeholders and modelers. Conclusions: There is a large gap in the current literature of formal evaluation of SM stakeholder engagement, and a lack of consensus about the processes required for effective SM stakeholder engagement. The adoption and clear reporting of structured engagement and process evaluation methodologies/frameworks are required to advance the field and produce evidence of impact.

Project description:BackgroundMuch research is being carried out using publicly available Twitter data in the field of public health, but the types of research questions that these data are being used to answer and the extent to which these projects require ethical oversight are not clear.ObjectiveThis review describes the current state of public health research using Twitter data in terms of methods and research questions, geographic focus, and ethical considerations including obtaining informed consent from Twitter handlers.MethodsWe implemented a systematic review, following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, of articles published between January 2006 and October 31, 2019, using Twitter data in secondary analyses for public health research, which were found using standardized search criteria on SocINDEX, PsycINFO, and PubMed. Studies were excluded when using Twitter for primary data collection, such as for study recruitment or as part of a dissemination intervention.ResultsWe identified 367 articles that met eligibility criteria. Infectious disease (n=80, 22%) and substance use (n=66, 18%) were the most common topics for these studies, and sentiment mining (n=227, 62%), surveillance (n=224, 61%), and thematic exploration (n=217, 59%) were the most common methodologies employed. Approximately one-third of articles had a global or worldwide geographic focus; another one-third focused on the United States. The majority (n=222, 60%) of articles used a native Twitter application programming interface, and a significant amount of the remainder (n=102, 28%) used a third-party application programming interface. Only one-third (n=119, 32%) of studies sought ethical approval from an institutional review board, while 17% of them (n=62) included identifying information on Twitter users or tweets and 36% of them (n=131) attempted to anonymize identifiers. Most studies (n=272, 79%) included a discussion on the validity of the measures and reliability of coding (70% for interreliability of human coding and 70% for computer algorithm checks), but less attention was paid to the sampling frame, and what underlying population the sample represented.ConclusionsTwitter data may be useful in public health research, given its access to publicly available information. However, studies should exercise greater caution in considering the data sources, accession method, and external validity of the sampling frame. Further, an ethical framework is necessary to help guide future research in this area, especially when individual, identifiable Twitter users and tweets are shared and discussed.Trial registrationPROSPERO CRD42020148170; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=148170.