Dataset Information

Is research working for you? Validating a tool to examine the capacity of health organizations to use research.

ABSTRACT:

Background

'Is research working for you? A self-assessment tool and discussion guide for health services management and policy organizations', developed by the Canadian Health Services Research Foundation, is a tool that can help organizations understand their capacity to acquire, assess, adapt, and apply research. Objectives were to: determine whether the tool demonstrated response variability; describe how the tool differentiated between organizations that were known to be lower-end or higher-end research users; and describe the potential usability of the tool.

Methods

Thirty-two focus groups were conducted among four sectors of Canadian health organizations. In the first hour of the focus group, participants individually completed the tool and then derived a group consensus ranking on items. In the second hour, the facilitator asked about overall impressions of the tool, to identify insights that emerged during the review of items on the tool and to elicit comments on research utilization. Discussion data were analyzed qualitatively, and individual and consensus item scores were analyzed using descriptive and non-parametric statistics.

Results

The tool demonstrated good usability and strong response variability. Differences between higher-end and lower-end research use organizations on scores suggested that this tool has adequate discriminant validity. The group discussion based on the tool was the more useful aspect of the exercise, rather than the actual score assigned.

Conclusion

The tool can serve as a catalyst for an important discussion about research use at the organizational level; such a discussion, in and of itself, demonstrates potential as an intervention to encourage processes and supports for research translation.

SUBMITTER: Kothari A

PROVIDER: S-EPMC2727486 | biostudies-literature | 2009 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Is research working for you? Validating a tool to examine the capacity of health organizations to use research.

Kothari Anita A Edwards Nancy N Hamel Nadia N Judd Maria M

Implementation science : IS 20090723

<h4>Background</h4>'Is research working for you? A self-assessment tool and discussion guide for health services management and policy organizations', developed by the Canadian Health Services Research Foundation, is a tool that can help organizations understand their capacity to acquire, assess, adapt, and apply research. Objectives were to: determine whether the tool demonstrated response variability; describe how the tool differentiated between organizations that were known to be lower-end or ...[more]

PMID: 19627601

Similar Datasets

Project description:International institutions provide well over US$10 billion in development assistance for health (DAH) annually and between 1990 and 2014, DAH disbursements totaled $458 billion but how do they decide who gets what, and for what purpose? In this article, we explore how allocation decisions were made by the nine convening agencies of the Equitable Access Initiative. We provide clear, plain language descriptions of the complete process from resource mobilization to allocation for the nine multilateral agencies with prominent agendas in global health. Then, through a comparative analysis we illuminate the choices and strategies employed in the nine international institutions. We find that resource allocation in all reviewed institutions follow a similar pattern, which we categorized in a framework of five steps: strategy definition, resource mobilization, eligibility of countries, support type and funds allocation. All the reviewed institutions generate resource allocation decisions through well-structured and fairly complex processes. Variations in those processes seem to reflect differences in institutional principles and goals. However, these processes have serious shortcomings. Technical problems include inadequate flexibility to account for or meet country needs. Although aid effectiveness and value for money are commonly referenced, we find that neither performance nor impact is a major criterion for allocating resources. We found very little formal consideration of the incentives generated by allocation choices. Political issues include non-transparent influence on allocation processes by donors and bureaucrats, and the common practice of earmarking funds to bypass the normal allocation process entirely. Ethical deficiencies include low accountability and transparency at international institutions, and limited participation by affected citizens or their representatives. We find that recipient countries have low influence on allocation processes themselves, although within these processes they have some influence in relatively narrow areas.

Project description:BackgroundPrevious surveys of organizations that support the development of evidence-informed health policies have focused on organizations that produce clinical practice guidelines (CPGs) or undertake health technology assessments (HTAs). Only rarely have surveys focused at least in part on units that directly support the use of research evidence in developing health policy on an international, national, and state or provincial level (i.e., government support units, or GSUs) that are in some way successful or innovative or that support the use of research evidence in low- and middle-income countries (LMICs).MethodsWe drew on many people and organizations around the world, including our project reference group, to generate a list of organizations to survey. We modified a questionnaire that had been developed originally by the Appraisal of Guidelines, Research and Evaluation in Europe (AGREE) collaboration and adapted one version of the questionnaire for organizations producing CPGs and HTAs, and another for GSUs. We sent the questionnaire by email to 176 organizations and followed up periodically with non-responders by email and telephone.ResultsWe received completed questionnaires from 152 (86%) organizations. More than one-half of the organizations (and particularly HTA agencies) reported that examples from other countries were helpful in establishing their organization. A higher proportion of GSUs than CPG- or HTA-producing organizations involved target users in the selection of topics or the services undertaken. Most organizations have few (five or fewer) full-time equivalent (FTE) staff. More than four-fifths of organizations reported providing panels with or using systematic reviews. GSUs tended to use a wide variety of explicit valuation processes for the research evidence, but none with the frequency that organizations producing CPGs, HTAs, or both prioritized evidence by its quality. Between one-half and two-thirds of organizations do not collect data systematically about uptake, and roughly the same proportions do not systematically evaluate their usefulness or impact in other ways.ConclusionThe findings from our survey, the most broadly based of its kind, both extend or clarify the applicability of the messages arising from previous surveys and related documentary analyses, such as how the 'principles of evidence-based medicine dominate current guideline programs' and the importance of collaborating with other organizations. The survey also provides a description of the history, structure, processes, outputs, and perceived strengths and weaknesses of existing organizations from which those establishing or leading similar organizations can draw.

Project description:Despite significant investments in health systems research (HSR) capacity development, there is a dearth of information regarding how to assess HSR capacity. An alliance of schools of public health (SPHs) in East and Central Africa developed a tool for the self-assessment of HSR capacity with the aim of producing institutional capacity development plans.Between June and November 2011, seven SPHs across the Democratic Republic of Congo, Ethiopia, Kenya, Rwanda, Tanzania, and Uganda implemented this co-created tool. The objectives of the institutional assessments were to assess existing capacities for HSR and to develop capacity development plans to address prioritized gaps. A mixed-method approach was employed consisting of document analysis, self-assessment questionnaires, in-depth interviews, and institutional dialogues aimed at capturing individual perceptions of institutional leadership, collective HSR skills, knowledge translation, and faculty incentives to engage in HSR. Implementation strategies for the capacity assessment varied across the SPHs. This paper reports findings from semi-structured interviews with focal persons from each SPH, to reflect on the process used at each SPH to execute the institutional assessments as well as the perceived strengths and weaknesses of the assessment process.The assessment tool was robust enough to be utilized in its entirety across all seven SPHs resulting in a thorough HSR capacity assessment and a capacity development plan for each SPH. Successful implementation of the capacity assessment exercises depended on four factors: (i) support from senior leadership and collaborators, (ii) a common understanding of HSR, (iii) adequate human and financial resources for the exercise, and (iv) availability of data. Methods of extracting information from the results of the assessments, however, were tailored to the unique objectives of each SPH.This institutional HSR capacity assessment tool and the process for its utilization may be valuable for any SPH. The self-assessments, as well as interviews with external stakeholders, provided diverse sources of input and galvanized interest around HSR at multiple levels.

Project description:BackgroundPrevious efforts to produce case descriptions have typically not focused on the organizations that produce research evidence and support its use. External evaluations of such organizations have typically not been analyzed as a group to identify the lessons that have emerged across multiple evaluations. Case descriptions offer the potential for capturing the views and experiences of many individuals who are familiar with an organization, including staff, advocates, and critics.MethodsWe purposively sampled a subgroup of organizations from among those that participated in the second (interview) phase of the study and (once) from among other organizations with which we were familiar. We developed and pilot-tested a case description data collection protocol, and conducted site visits that included both interviews and documentary analyses. Themes were identified from among responses to semi-structured questions using a constant comparative method of analysis. We produced both a brief (one to two pages) written description and a video documentary for each case.ResultsWe conducted 51 interviews as part of the eight site visits. Two organizational strengths were repeatedly cited by individuals participating in the site visits: use of an evidence-based approach (which was identified as being very time-consuming) and existence of a strong relationship between researchers and policymakers (which can be challenged by conflicts of interest). Two organizational weaknesses - a lack of resources and the presence of conflicts of interest - were repeatedly cited by individuals participating in the site visits. Participants offered two main suggestions for the World Health Organization (and other international organizations and networks): 1) mobilize one or more of government support, financial resources, and the participation of both policymakers and researchers; and 2) create knowledge-related global public goods.ConclusionThe findings from our case descriptions, the first of their kind, intersect in interesting ways with the messages arising from two systematic reviews of the factors that increase the prospects for research use in policymaking. Strong relationships between researchers and policymakers bodes well given such interactions appear to increase the prospects for research use. The time-consuming nature of an evidence-based approach, on the other hand, suggests the need for more efficient production processes that are 'quick and clean enough.' Our case descriptions and accompanying video documentaries provide a rich description of organizations supporting the use of research evidence, which can be drawn upon by those establishing or leading similar organizations, particularly in low- and middle-income countries.

Project description:BackgroundOnly a small number of previous efforts to describe the experiences of organizations that produce clinical practice guidelines (CPGs), undertake health technology assessments (HTAs), or directly support the use of research evidence in developing health policy (i.e., government support units, or GSUs) have relied on interviews and then only with HTA agencies. Interviews offer the potential for capturing experiences in great depth, particularly the experiences of organizations that may be under-represented in surveys.MethodsWe purposively sampled organizations from among those who completed a questionnaire in the first phase of our three-phase study, developed and piloted a semi-structured interview guide, and conducted the interviews by telephone, audio-taped them, and took notes simultaneously. Binary or categorical responses to more structured questions were counted when possible. Themes were identified from among responses to semi-structured questions using a constant comparative method of analysis. Illustrative quotations were identified to supplement the narrative description of the themes.ResultsWe interviewed the director (or his or her nominee) in 25 organizations, of which 12 were GSUs. Using rigorous methods that are systematic and transparent (sometimes shortened to 'being evidence-based') was the most commonly cited strength among all organizations. GSUs more consistently described their close links with policymakers as a strength, whereas organizations producing CPGs, HTAs, or both had conflicting viewpoints about such close links. With few exceptions, all types of organizations tended to focus largely on weaknesses in implementation, rather than strengths. The advice offered to those trying to establish similar organizations include: 1) collaborate with other organizations; 2) establish strong links with policymakers and stakeholders; 3) be independent and manage conflicts of interest; 4) build capacity; 5) use good methods and be transparent; 6) start small and address important questions; and 7) be attentive to implementation considerations. The advice offered to the World Health Organization (WHO) and other international organizations and networks was to foster collaborations across organizations.ConclusionThe findings from our interview study, the most broadly based of its kind, extend to both CPG-producing organizations and GSUs the applicability of the messages arising from previous interview studies of HTA agencies, such as to collaborate with other organizations and to be attentive to implementation considerations. Our interview study also provides a rich description of organizations supporting the use of research evidence, which can be drawn upon by those establishing or leading similar organizations in LMICs.

Project description:Working in a police organization often involves being exposed to potentially traumatic events and stressful circumstances regardless of occupation or rank. Police mental health is a public health concern, but the mental health of civilian employees working in police organizations has been much less studied. The current study aims to compare the frequency of mental health conditions in both police officers and civilians. This was evaluated by measuring mean scores on several mental health screening tools including scales to determine symptom severity for posttraumatic stress disorder (PTSD) with the PTSD Checklist – PCL-5, depression with the Patient Health Questionnaire-9 (PHQ), general anxiety with the Generalized Anxiety Disorder 7-item scale (GAD-7), and alcohol use with the Alcohol Use Disorders Identification Test (AUDIT). The total potential population was 1,225 civilian employees and 3,714 police officers, of which 513 (10%) participated. Of these, 201 (16%) were civilians, and 312 (8%) were police officers (p<0.001). In the study population, 26% screened positive for any mental health disorder. Somewhat surprisingly, we found significantly more civilians (32.8%) than police officers (22.7%) met diagnostic criteria. We also found that civilian participants had higher mean scores in measures of PTSD, anxiety, and depression, although only for depression did this reach statistical significance. Civilians were 1.7 times more likely to screen positive for depression compared to police officers, a statistically significant difference. In contrast, police officers demonstrated statistically higher scores for alcohol use than civilians. One limitation of this study is that the data reflects responses from only a minority of the overall population and, therefore, may not accurately reflect the frequency of mental health issues in the total police organization including civilian employees. Nonetheless, the results strongly suggest that the mental health of all employees can be negatively impacted by working in a police environment, and this is important given the growing number of civilians employed within police organizations. These findings support initiatives aimed at destigmatizing mental health disorders, improving stress management, and increasing access to mental health care on an organization-wide basis, and not just limited to front-line police officers.

Dataset Information

Is research working for you? Validating a tool to examine the capacity of health organizations to use research.

Background

Methods

Results

Conclusion

Publications

Is research working for you? Validating a tool to examine the capacity of health organizations to use research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets