Dataset Information

Multi-staged development and pilot testing of a self-assessment tool for organizational health literacy.

ABSTRACT:

Background

Until now a comprehensive, consensus-based tool that can be used by a variety of health care organizations for assessing their organizational health literacy (OHL) is not available. Therefore, we aimed to develop and test a literature- and consensus-based self-assessment tool.

Methods

The study is based on a scoping review that was previously published by the authors. For the development of the self-assessment tool, the criteria identified in the literature were synthesized with criteria gained through group discussions with representatives of different types of health care organizations (N = 27) all based in Hamburg (Germany). Consensus on the criteria was reached by conducting a Delphi process (N = 22). A review by the project's patient advisory council was included in the process. The self-assessment tool was converted into an online tool and refined by a pretest. Finally, the online survey was piloted (N = 53) and the reliability and item loadings for each scale were analyzed.

Results

In total, 77 criteria (items) characterizing a health literate health care organization were developed and grouped into five main categories (scales): (1) "easy access and navigation", (2) "integration, prioritization, and dissemination of OHL", (3) "qualification, quality management, evaluation, and needs assessment", (4) "communication with target groups", and (5) "involvement and support of target groups". The results of the online survey showed that the tool is suitable for assessing an organization's status quo on OHL. The psychometric analysis showed good to excellent internal consistency. Item analyses of the developed self-assessment tool was satisfactory.

Conclusions

We were able to define a set of 77 items to characterize OHL, which were integrated into a new, comprehensive, and consensus-based self-assessment tool to identify areas for improvement. We found evidence that the self-assessment tool, based on the identified criteria, consists of the assumed five scales. Further research should analyze the validity of the self-assessment tool on a higher detail level.

SUBMITTER: Klockmann I

PROVIDER: S-EPMC10720162 | biostudies-literature | 2023 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multi-staged development and pilot testing of a self-assessment tool for organizational health literacy.

Klockmann Izumi I Jaß Leonie L Härter Martin M von dem Knesebeck Olaf O Lüdecke Daniel D Heeg Johanna J

BMC health services research 20231213 1

<h4>Background</h4>Until now a comprehensive, consensus-based tool that can be used by a variety of health care organizations for assessing their organizational health literacy (OHL) is not available. Therefore, we aimed to develop and test a literature- and consensus-based self-assessment tool.<h4>Methods</h4>The study is based on a scoping review that was previously published by the authors. For the development of the self-assessment tool, the criteria identified in the literature were synthes ...[more]

PMID: 38093331

Similar Datasets

Project description:BackgroundThe World Health Organization describes health literacy as a critical determinant of health and driver of citizen empowerment and health equity. Several studies have shown that health literacy is associated with a range of socioeconomic factors including educational attainment, financial position and ethnicity. The complexity of the health system influences how well a person is able to engage with information and services. Health organisations can empower the populations they serve and address inequity by ensuring they are health literacy responsive. The aim of this study was to develop the Organisational Health Literacy Responsiveness self-assessment tool (Org-HLR Tool), and an assessment process to support organisations with application of the tool.MethodsA co-design workshop with health and social service professionals was undertaken to inform the structure of the tool and assessment process. Participants critiqued existing self-assessment tools and discussed the likely utility of the data they generate. A review of widely used organisational performance assessment tools informed the structure and self-assessment process. The Organisational Health Literacy Responsiveness (Org-HLR) Framework (with seven domains/24 sub-domains) provided the structure for the assessment dimensions of the tool. The performance indicators were drawn from raw data collected during development of the Org-HLR Framework.ResultsTwenty-two professionals participated in the workshop. Based on the feedback provided and a review of existing tools, a multi-stage, group-based assessment process for implementing the Org-HLR Tool was developed. The assessment process was divided into three parts; i) reflection; ii) self-rating; and iii) priority setting, each supported by a corresponding tool. The self-rating tool, consistent with the Org-HLR Framework, was divided into: External policy and funding environment; Leadership and culture; Systems, processes and policies; Access to services and programs; Community engagement and partnerships; Communication practices and standards; Workforce. Each of these had 1 to 5 sub-dimensions (24 in total), and 135 performance indicators.ConclusionsThe Org-HLR Tool and assessment process were developed to address a gap in available tools to support organisations to assess their health literacy responsiveness, and prioritise and plan their quality improvement activities. The tool is currently in the field for further utility and acceptability testing.

Project description:BackgroundThe Portuguese Physical Literacy Assessment (PPLA) is a novel tool to assess high-school students' (grade 10-12; 15-18 years) Physical Literacy (PL) in Physical Education (PE); inspired by the four domains of the Australian Physical Literacy Framework (APLF), and the Portuguese PE syllabus. This paper describes the development, content validation, and pilot testing of the PPLA-Questionnaire (PPLA-Q), one of two instruments in the PPLA, comprised of modules to assess the psychological, social, and part of the cognitive domain of PL.MethodsDevelopment was supported by previous work, analysis of the APLF, and literature review. We iteratively gathered evidence on content validity through two rounds of qualitative and quantitative expert validation (n = 11); three rounds of cognitive interviews with high-school students (n = 12); and multiple instances of expert advisor input. A pilot study in two grade 10 classes (n = 41) assessed feasibility, preliminary reliability, item difficulty and discrimination.ResultsInitial versions of the PPLA-Q gathered evidence in favor of adequate content validity at item level: most items had an Item-Content Validity Index ≥.78 and Cohen's κ ≥ .76. At module-level, S-CVI/Ave and UA were .87/.60, .98/.93 and .96/.84 for the cognitive, psychological, and social modules, respectively. Through the pilot study, we found evidence for feasibility, preliminary subscale and item reliability, difficulty, and discrimination. Items were reviewed through qualitative methods until saturation. Current PPLA-Q consists of 3 modules: cognitive (knowledge test with 10 items), psychological (46 Likert-type items) and social (43 Likert-type items).ConclusionResults of this study provide evidence for content validity, feasibility within PE setting and preliminary reliability of the PPLA-Q as an instrument to assess the psychological, social, and part of the cognitive domain of PL in grade 10 to 12 adolescents. Further validation and development are needed to establish construct validity and reliability, and study PPLA-Q's integration with the PPLA-Observation (an instrument in development to assess the remaining domains of PL) within the PPLA framework.

Project description:BackgroundFor people to be able to access, understand, and benefit from the increasing digitalization of health services, it is critical that services are provided in a way that meets the user's needs, resources, and competence.ObjectiveThe objective of the study was to develop a questionnaire that captures the 7-dimensional eHealth Literacy Framework (eHLF).MethodsDraft items were created in parallel in English and Danish. The items were generated from 450 statements collected during the conceptual development of eHLF. In all, 57 items (7 to 9 items per scale) were generated and adjusted after cognitive testing. Items were tested in 475 people recruited from settings in which the scale was intended to be used (community and health care settings) and including people with a range of chronic conditions. Measurement properties were assessed using approaches from item response theory (IRT) and classical test theory (CTT) such as confirmatory factor analysis (CFA) and reliability using composite scale reliability (CSR); potential bias due to age and sex was evaluated using differential item functioning (DIF).ResultsCFA confirmed the presence of the 7 a priori dimensions of eHLF. Following item analysis, a 35-item 7-scale questionnaire was constructed, covering (1) using technology to process health information (5 items, CSR=.84), (2) understanding of health concepts and language (5 items, CSR=.75), (3) ability to actively engage with digital services (5 items, CSR=.86), (4) feel safe and in control (5 items, CSR=.87), (5) motivated to engage with digital services (5 items, CSR=.84), (6) access to digital services that work (6 items, CSR=.77), and (7) digital services that suit individual needs (4 items, CSR=.85). A 7-factor CFA model, using small-variance priors for cross-loadings and residual correlations, had a satisfactory fit (posterior productive P value: .27, 95% CI for the difference between the observed and replicated chi-square values: -63.7 to 133.8). The CFA showed that all items loaded strongly on their respective factors. The IRT analysis showed that no items were found to have disordered thresholds. For most scales, discriminant validity was acceptable; however, 2 pairs of dimensions were highly correlated; dimensions 1 and 5 (r=.95), and dimensions 6 and 7 (r=.96). All dimensions were retained because of strong content differentiation and potential causal relationships between these dimensions. There is no evidence of DIF.ConclusionsThe eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool based on a well-defined a priori eHLF framework with robust properties. It has satisfactory evidence of construct validity and reliable measurement across a broad range of concepts (using both CTT and IRT traditions) in various groups. It is designed to be used to understand and evaluate people's interaction with digital health services.

Dataset Information

Multi-staged development and pilot testing of a self-assessment tool for organizational health literacy.

Background

Methods

Results

Conclusions

Publications

Multi-staged development and pilot testing of a self-assessment tool for organizational health literacy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets