Dataset Information

Development and validation of an organizational readiness to change instrument focused on cultural competency.

ABSTRACT:

Objective

To validate a brief survey developed to be used in hospitals nationwide to assess organizational readiness to change to increase cultural competency.

Data sources/study setting

Analysis of primary data collected as part of a 125-item Organizational Assessment Survey conducted in the ten US hospitals participating in the Robert Wood Johnson Foundation Expecting Success program in 2005-2006.

Study design

The study utilized a cross-sectional survey.

Data collection

Surveys were distributed to participants in the ten hospitals based on job title and role within the organization (including clinicians, clinical administrators, other clinical professionals, and those in relevant nonclinical roles; respondents = 513; response rate = 31%). Missing data were deleted listwise. We computed internal consistency reliability via Cronbach's alpha and interrater agreement using the r_wg(j) index, and conducted exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to examine validity of the survey. We subsequently conducted ANOVAs to examine whether the instrument adequately distinguished between hospitals.

Principal findings

Across 408 complete responses, a scree plot generated by the EFA and a follow-up CFA indicated a 2-factor solution (RMSEA = 0.06; CFI = 0.96; GFI = 0.96; RMSR = 0.08). We identified these primary factors as two scales, a 12-item Readiness to Address Quality scale (α = 0.85; r_wg(j) = 0.93) and an 11-item Readiness to Address Disparities scale (α = 0.65; r_wg(j) = 0.89). ANOVAs suggested that these scales distinguished between hospitals (RTAQ: F[9, 428] = 3.70, P < .001; RTAD: F[9, 435] = 3.02, P = .002).

Conclusions

This survey can help identify an organization's readiness to change to increase cultural competency.

SUBMITTER: McAlearney AS

PROVIDER: S-EPMC7839647 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundOrganizational readiness is a factor known to influence the predisposition of individuals within an organization to change. Based on Weiner's organizational theory, the "Organizational Readiness for Implementing Change" (ORIC) questionnaire was developed and validated to measure organizational readiness in healthcare contexts. However, no such tools allow French-speaking organizations to measure this concept. The objectives of this study were to (1) conduct a French cultural adaptation of the ORIC questionnaire, and (2) initiate the study of its psychometric properties.Methods(1) Cross-cultural adaptation and translation processes were first conducted with the methodologies of Beaton, Vallerand and Massoubre. (2) Subsequently, internal consistency was documented by calculating Cronbach's alpha and inter-item and item-to-scale correlations. The study of construct validity was initiated with a confirmatory factor analysis.ResultsA French 10-item scale named the Réceptivité organisationnelle à l'implantation d'un changement (ROIC) was developed and pretested by 125 occupational therapists working in Quebec homecare services. Cronbach's alpha values for the 2 item subscales show satisfactory internal consistency (Commitment: α = 0.84 and Efficacy: α = 0.86). Inter-item correlations revealed that the ROIC's items are moderately related to each other while item-to-total scale correlations pinpoint items that accounts for variance and influence internal consistency. Confirmatory factor analysis allowed the initiation of a substantial documentation of ROIC's model fit with the original version (CFI = 0.89, TLI = 0.85, SRMR = 0.08, and RMSEA = 0.12).ConclusionsThe ROIC is a new theory-based and translated questionnaire that can be used to rigorously document the organizational readiness of French organizations. The ROIC has the potential to support members of different organizations in the identification of subsequent efforts for the implementation of a change.

Project description:BackgroundThe Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation.MethodsWe conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales.ResultsCronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale).ConclusionWe find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation.

Project description:UnlabelledBackgroundThere is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment.ObjectivesOur objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias.Methods and designWe will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique.DiscussionWe propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs.

Project description:Organizational readiness for change in healthcare settings is an important factor in successful implementation of new policies, programs, and practices. However, research on the topic is hindered by the absence of a brief, reliable, and valid measure. Until such a measure is developed, we cannot advance scientific knowledge about readiness or provide evidence-based guidance to organizational leaders about how to increase readiness. This article presents results of a psychometric assessment of a new measure called Organizational Readiness for Implementing Change (ORIC), which we developed based on Weiner's theory of organizational readiness for change.We conducted four studies to assess the psychometric properties of ORIC. In study one, we assessed the content adequacy of the new measure using quantitative methods. In study two, we examined the measure's factor structure and reliability in a laboratory simulation. In study three, we assessed the reliability and validity of an organization-level measure of readiness based on aggregated individual-level data from study two. In study four, we conducted a small field study utilizing the same analytic methods as in study three.Content adequacy assessment indicated that the items developed to measure change commitment and change efficacy reflected the theoretical content of these two facets of organizational readiness and distinguished the facets from hypothesized determinants of readiness. Exploratory and confirmatory factor analysis in the lab and field studies revealed two correlated factors, as expected, with good model fit and high item loadings. Reliability analysis in the lab and field studies showed high inter-item consistency for the resulting individual-level scales for change commitment and change efficacy. Inter-rater reliability and inter-rater agreement statistics supported the aggregation of individual level readiness perceptions to the organizational level of analysis.This article provides evidence in support of the ORIC measure. We believe this measure will enable testing of theories about determinants and consequences of organizational readiness and, ultimately, assist healthcare leaders to reduce the number of health organization change efforts that do not achieve desired benefits. Although ORIC shows promise, further assessment is needed to test for convergent, discriminant, and predictive validity.