Dataset Information

QuikLitE, a Framework for Quick Literacy Evaluation in Medicine: Development and Validation.

ABSTRACT:

Background

A plethora of health literacy instruments was developed over the decades. They usually start with experts curating passages of text or word lists, followed by psychometric validation and revision based on test results obtained from a sample population. This process is costly and it is difficult to customize for new usage scenarios.

Objective

This study aimed to develop and evaluate a framework for dynamically creating test instruments that can provide a focused assessment of patients' health literacy.

Methods

A health literacy framework and scoring method were extended from the vocabulary knowledge test to accommodate a wide range of item difficulties and various degrees of uncertainty in the participant's answer. Web-based tests from Amazon Mechanical Turk users were used to assess reliability and validity.

Results

Parallel forms of our tests showed high reliability (correlation=.78; 95% CI 0.69-0.85). Validity measured as correlation with an electronic health record comprehension instrument was higher (.47-.61 among 3 groups) than 2 existing tools (Short Assessment of Health Literacy-English, .38-.43; Short Test of Functional Health Literacy in Adults, .34-.46). Our framework is able to distinguish higher literacy levels that are often not measured by other instruments. It is also flexible, allowing customizations to the test the designer's focus on a particular interest in a subject matter or domain. The framework is among the fastest health literacy instrument to administer.

Conclusions

We proposed a valid and highly reliable framework to dynamically create health literacy instruments, alleviating the need to repeat a time-consuming process when a new use scenario arises. This framework can be customized to a specific need on demand and can measure skills beyond the basic level.

SUBMITTER: Zheng J

PROVIDER: S-EPMC6406229 | biostudies-literature | 2019 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

QuikLitE, a Framework for Quick Literacy Evaluation in Medicine: Development and Validation.

Zheng Jiaping J Yu Hong H

Journal of medical Internet research 20190222 2

<h4>Background</h4>A plethora of health literacy instruments was developed over the decades. They usually start with experts curating passages of text or word lists, followed by psychometric validation and revision based on test results obtained from a sample population. This process is costly and it is difficult to customize for new usage scenarios.<h4>Objective</h4>This study aimed to develop and evaluate a framework for dynamically creating test instruments that can provide a focused assessme ...[more]

PMID: 30794206

Similar Datasets

Project description:BackgroundMobile devices have greatly facilitated the use of digital health resources, particularly during the COVID-19 pandemic. Mobile health (mHealth) has become a common and important way to monitor and improve health conditions for people from different social classes. The ability to utilize mHealth affects its effectiveness; therefore, the widespread application of mHealth technologies calls for an instrument that can accurately measure health literacy in the era of mobile media.ObjectiveWe aimed to (1) identify the components of mHealth literacy for ordinary users and (2) develop a systematic scale for appropriately measuring individuals' self-perceived mHealth literacy through a problem-based framework.MethodsWe conducted an exploratory study involving in-depth interviews and observations (15 participants) in January 2020 and used exploratory factor analysis and confirmatory factor analysis to identify the components of mHealth literacy and develop an item pool. In February 2020, we conducted a pilot survey with 148 participants to explore the factor structures of items identified during the exploratory study. Subsequently, 2 surveys were administrated using quota sampling. The first survey (conducted in Guangdong, China) collected 552 responses during March 2020; we assessed composite reliability, convergent validity, and discriminant validity. The second survey (conducted in China nationwide) collected 433 responses during October 2021; we assessed criterion-related validity using structural equation modeling.ResultsWe identified 78 items during the exploratory study. The final scale-the Problem-Based mHealth Literacy Scale-consists of 33 items that reflect 8 domains of mHealth literacy. The first web-based survey suggested that mHealth literacy consists of 8 factors (ie, subscales), namely, mHealth desire, mobile phone operational skills, acquiring mHealth information, acquiring mHealth services, understanding of medical terms, mobile-based patient-doctor communication, evaluating mHealth information, and mHealth decision-making. These factors were found to be reliable (composite reliability >0.7), with good convergent validity (average variance extracted >0.5) and discriminant validity (square root of average variance extracted are greater than the correlation coefficients between factors). The findings also revealed that these 8 factors should be grouped under a second-order factor model (χ2/df=2.701; comparative fit index 0.921; root mean square error of approximation 0.056; target coefficient 0.831). The second survey revealed that mHealth use had a significant impact (β=0.43, P<.001) on mHealth literacy and that mHealth literacy had a significant impact (β=0.23, P<.001) on health prevention behavior.ConclusionsThis study revealed the distinctiveness of mHealth literacy by placing mHealth needs, the ability to understand medical terms, and the skills in patient-doctor interactions in the foreground. The Problem-Based mHealth Literacy Scale is a useful instrument for comprehensively measuring individuals' mHealth literacy and extends the concept of health literacy to the context of mobile communication.

Project description:BACKGROUND:There is extensive literature on the methodology of evaluation research and the development and evaluation of complex interventions but little guidance on the formative stages before evaluation and how to work with partner organizations that wish to have their provision evaluated. It is important to be able to identify suitable projects for evaluation from a range of provision and describe the steps required, often with academic institutions working in partnership with external organizations, in order to set up an evaluation. However, research evaluating programs or interventions rarely discusses these stages. OBJECTIVE:This study aimed to extend work on evaluability assessment and pre-evaluation planning by proposing an 8-Step Scoping Framework to enable the appraisal of multiple programs in order to identify interventions suitable for evaluation. We aimed to add to the literature on evaluability assessment and more recent evaluation guidance by describing the processes involved in working with partner organizations. METHODS:This paper documents the steps required to identify multiple complex interventions suitable for process and outcome evaluation. The steps were developed using an iterative approach by working alongside staff in a local government organization, to build an evidence base to demonstrate which interventions improve children's outcomes. The process of identifying suitable programs for evaluation, thereby establishing the pre-evaluation steps, was tested using all Flying Start provision. RESULTS:The 8-Step Scoping Framework was described using the example of the local government organization Flying Start to illustrate how each step contributes to finding projects suitable for process and outcome evaluation: (1) formulating overarching key questions that encompass all programs offered by an organization, (2) gaining an in-depth understanding of the work and provision of an organization and engaging staff, (3) completing a data template per project/program offered, (4) assessing the robustness/validity of data across all programs, (5) deciding on projects suitable for evaluation and those requiring additional data, (6) negotiating with chosen project leads, both within and outside the organization, (7) developing individual project evaluation protocols, and (8) applying for ethical approval from the university and partner organization. CONCLUSIONS:This paper describes the processes involved in identifying suitable projects for evaluation. It adds to the existing literature on the assessment of specific programs suitable for evaluation and guidance for conducting evaluations by establishing the formative steps required to identify suitable programs from a range of provision. This scoping framework particularly relates to academic partners and organizations tasked with delivering evidence-based services designed to meet local needs. The steps identified have been described in the context of early years provision but can be applied to a range of community-based evaluations, or more generally, to cases where an academic partner is working with external stakeholders to identify projects suitable for academic evaluation.

Project description:ObjectiveValidity refers to the extent to which evidence and theory support the adequacy and appropriateness of inferences based on score interpretations. The health sector is lacking a theoretically-driven framework for the development, testing and use of health assessments. This study used the Standards for Educational and Psychological Testing framework of five sources of validity evidence to assess the types of evidence reported for health literacy assessments, and to identify studies that referred to a theoretical validity testing framework.MethodsA systematic descriptive literature review investigated methods and results in health literacy assessment development, application and validity testing studies. Electronic searches were conducted in EBSCOhost, Embase, Open Access Theses and Dissertations and ProQuest Dissertations. Data were coded to the Standards' five sources of validity evidence, and for reference to a validity testing framework.ResultsCoding on 46 studies resulted in 195 instances of validity evidence across the five sources. Only nine studies directly or indirectly referenced a validity testing framework. Evidence based on relations to other variables is most frequently reported.ConclusionsThe health and health equity of individuals and populations are increasingly dependent on decisions based on data collected through health assessments. An evidence-based theoretical framework provides structure and coherence to existing evidence and stipulates where further evidence is required to evaluate the extent to which data are valid for an intended purpose. This review demonstrates the use of the Standards' theoretical validity testing framework to evaluate sources of evidence reported for health literacy assessments. Findings indicate that theoretical validity testing frameworks are rarely used to collate and evaluate evidence in validation practice for health literacy assessments. Use of the Standards' theoretical validity testing framework would improve evaluation of the evidence for inferences derived from health assessment data on which public health and health equity decisions are based.

Project description:BACKGROUND:For people to be able to access, understand, and benefit from the increasing digitalization of health services, it is critical that services are provided in a way that meets the user's needs, resources, and competence. OBJECTIVE:The objective of the study was to develop a questionnaire that captures the 7-dimensional eHealth Literacy Framework (eHLF). METHODS:Draft items were created in parallel in English and Danish. The items were generated from 450 statements collected during the conceptual development of eHLF. In all, 57 items (7 to 9 items per scale) were generated and adjusted after cognitive testing. Items were tested in 475 people recruited from settings in which the scale was intended to be used (community and health care settings) and including people with a range of chronic conditions. Measurement properties were assessed using approaches from item response theory (IRT) and classical test theory (CTT) such as confirmatory factor analysis (CFA) and reliability using composite scale reliability (CSR); potential bias due to age and sex was evaluated using differential item functioning (DIF). RESULTS:CFA confirmed the presence of the 7 a priori dimensions of eHLF. Following item analysis, a 35-item 7-scale questionnaire was constructed, covering (1) using technology to process health information (5 items, CSR=.84), (2) understanding of health concepts and language (5 items, CSR=.75), (3) ability to actively engage with digital services (5 items, CSR=.86), (4) feel safe and in control (5 items, CSR=.87), (5) motivated to engage with digital services (5 items, CSR=.84), (6) access to digital services that work (6 items, CSR=.77), and (7) digital services that suit individual needs (4 items, CSR=.85). A 7-factor CFA model, using small-variance priors for cross-loadings and residual correlations, had a satisfactory fit (posterior productive P value: .27, 95% CI for the difference between the observed and replicated chi-square values: -63.7 to 133.8). The CFA showed that all items loaded strongly on their respective factors. The IRT analysis showed that no items were found to have disordered thresholds. For most scales, discriminant validity was acceptable; however, 2 pairs of dimensions were highly correlated; dimensions 1 and 5 (r=.95), and dimensions 6 and 7 (r=.96). All dimensions were retained because of strong content differentiation and potential causal relationships between these dimensions. There is no evidence of DIF. CONCLUSIONS:The eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool based on a well-defined a priori eHLF framework with robust properties. It has satisfactory evidence of construct validity and reliable measurement across a broad range of concepts (using both CTT and IRT traditions) in various groups. It is designed to be used to understand and evaluate people's interaction with digital health services.

Dataset Information

QuikLitE, a Framework for Quick Literacy Evaluation in Medicine: Development and Validation.

Background

Objective

Methods

Results

Conclusions

Publications

QuikLitE, a Framework for Quick Literacy Evaluation in Medicine: Development and Validation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets