Dataset Information

A refined method for theory-based evaluation of the societal impacts of research.

ABSTRACT: With high and increasing expectations for research to have social and environmental impact, there is a corresponding need for appropriate methods to demonstrate (for accountability) and analyze (for learning) whether and how research projects contribute to change processes. Evaluation is especially challenging for problem-oriented research that employs inter- and transdisciplinary approaches and intervenes in complex systems, where experimental and statistical approaches to causal inference are inappropriate. Instead, theory-based evaluation can be applied to identify and test causal processes. This paper presents a detailed explanation of the Outcome Evaluation approach applied in Belcher et al. (2019b). It draws on concepts and approaches used in theory-based program evaluation and the more limited experience of theory-based research evaluation, providing a brief overview of conceptual strengths and limitations of other methods. The paper offers step-by-step guidance on application of the Outcome Evaluation approach, detailing how to: document a theory of change; determine data needs and sources; collect data; manage and analyze data; and present findings. This approach provides a clear conceptual and analytical framework in addition to actor-specific and impact pathway analyses for more precision in the assessment of outcomes. Specifically, the Outcome Evaluation approach: •Conceptualizes research within a complex system and explicitly recognizes the role of other actors, context, and external processes;•Utilizes a detailed actor-centred theory of change (ToC) as the analytical framework; and•Explicitly tests a set of hypotheses about the relationship between the research process/outputs and outcomes.

SUBMITTER: Belcher BM

PROVIDER: S-EPMC6997623 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A refined method for theory-based evaluation of the societal impacts of research.

Belcher Brian M BM Davel Rachel R Claus Rachel R

MethodsX 20200123

With high and increasing expectations for research to have social and environmental impact, there is a corresponding need for appropriate methods to demonstrate (for accountability) and analyze (for learning) whether and how research projects contribute to change processes. Evaluation is especially challenging for problem-oriented research that employs inter- and transdisciplinary approaches and intervenes in complex systems, where experimental and statistical approaches to causal inference are ...[more]

PMID: 32025508

Similar Datasets

Project description:Ongoing debates surrounding Open Access to the scholarly literature are multifaceted and complicated by disparate and often polarised viewpoints from engaged stakeholders. At the current stage, Open Access has become such a global issue that it is critical for all involved in scholarly publishing, including policymakers, publishers, research funders, governments, learned societies, librarians, and academic communities, to be well-informed on the history, benefits, and pitfalls of Open Access. In spite of this, there is a general lack of consensus regarding the potential pros and cons of Open Access at multiple levels. This review aims to be a resource for current knowledge on the impacts of Open Access by synthesizing important research in three major areas: academic, economic and societal. While there is clearly much scope for additional research, several key trends are identified, including a broad citation advantage for researchers who publish openly, as well as additional benefits to the non-academic dissemination of their work. The economic impact of Open Access is less well-understood, although it is clear that access to the research literature is key for innovative enterprises, and a range of governmental and non-governmental services. Furthermore, Open Access has the potential to save both publishers and research funders considerable amounts of financial resources, and can provide some economic benefits to traditionally subscription-based journals. The societal impact of Open Access is strong, in particular for advancing citizen science initiatives, and leveling the playing field for researchers in developing countries. Open Access supersedes all potential alternative modes of access to the scholarly literature through enabling unrestricted re-use, and long-term stability independent of financial constraints of traditional publishers that impede knowledge sharing. However, Open Access has the potential to become unsustainable for research communities if high-cost options are allowed to continue to prevail in a widely unregulated scholarly publishing market. Open Access remains only one of the multiple challenges that the scholarly publishing system is currently facing. Yet, it provides one foundation for increasing engagement with researchers regarding ethical standards of publishing and the broader implications of 'Open Research'.

Project description:PurposeResearch-based theater uses drama to communicate research findings to audiences beyond those that typically read peer-reviewed journals. We applied research-based theater to translate qualitative research findings on the impact of the COVID-19 pandemic on different segments of U.S. society.ApproachTheater artists and public health researchers collaborated to create a collection of eight monologues from systematically sourced, peer-reviewed publications. Following three virtual performances in Spring, 2021, audience members were invited to complete a survey.Setting/participantsAudience survey respondents (n = 120) were mostly U.S.-based and were diverse in terms of age, race/ethnicity, gender, profession, and experience attending theater.MethodWe summarized closed-ended responses and explored patterns by demographic characteristics. We synthesized themes of open-ended responses with inductive coding.ResultsAudience members somewhat/strongly agreed that COVID Monologues increased their knowledge (79.4%), represented the reality of the U.S. COVID-19 epidemic (95.7%), and offered new perspectives on what people had been experiencing (87.5%). Most also agreed research-based theater is an effective means of understanding health research (93.5%) and can promote community resilience in times of public health crisis (83.2%). Mann-Whitney U tests suggested less positive reactions from demographics that were not well-represented in monologue characters (cisgender men, Hispanics). Qualitative comments suggested audience members valued monologues that offered self-reflection and validation of their own COVID-19 experiences through relatable characters as well as those that offered insight into the experiences of people different from themselves.ConclusionThis work adds to evidence that research-based theater can help build knowledge and emotional insight around a public health issue. As these elements are foundational to pro-social, preventative health behaviors, research-based theater may have a useful role in promoting collective response to public health crises like COVID-19. Our method of systematically-sourcing research for theater-based dissemination could be extended to target more specific audiences with actionable behaviors.

Dataset Information

A refined method for theory-based evaluation of the societal impacts of research.

Publications

A refined method for theory-based evaluation of the societal impacts of research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets