Dataset Information

MAPPinfo, mapping quality of health information: study protocol for a validation study of an assessment instrument.

ABSTRACT: INTRODUCTION:Health information is a prerequisite of informed decision-making. Criteria for development, content and presentation have recently been published in a corresponding guideline. Within a systematic search, 27 relevant checklists were identified, none of them, however, complying with the guideline or providing reasonably operationalised measurement items. Therefore, a draft of a checklist with 19 criteria was drafted. The current study aims at developing and validating this measure of quality. METHODS AND ANALYSIS:The validation design consists of five single studies to be conducted at the University of Halle-Wittenberg/Germany and Graz/Austria. (1) Achieving content validity through expert reviews of the first draft, (2) achieving feasibility using 'think aloud' in piloting with untrained users, (3) pretesting the instrument applied to health information materials without use of secondary sources: determining inter-rater reliability and criterion validity, (4) determining construct validity using information on proceedings and methods in the development process provided by the developers and (5) determining divergent validity in comparison with the Ensuring Quality Information for Patients (EQUIP) (expanded) Scale. The substudies will use varying samples of experts, students and developers and will apply the instrument to materials of various domains. Sample sizes will be adjusted to the particular research designs and questions. Analyses will employ qualitative methods, such as content analyses and discourse within the expert panel, and correlation-based methods both for determining inter-rater reliability and validity. ETHICS AND DISSEMINATION:The project is approved by the ethics committee of the Martin Luther University Halle-Wittenberg (approval number: 2019 115). Results will be published, and the instrument made accessible on public health platforms. It is meant to become a certification standard. MAPPinfo can be used as a screening instrument without training or secondary sources. Although developed in the German language, the instrument will be applicable also in other languages. TRIAL REGISTRATION NUMBER:AsPredected22546; date of registration: 24 July 2019. PROTOCOL VERSION:July 2020.

SUBMITTER: Kasper J

PROVIDER: S-EPMC7640523 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

MAPPinfo, mapping quality of health information: study protocol for a validation study of an assessment instrument.

Kasper Jürgen J Lühnen Julia J Hinneburg Jana J Siebenhofer Andrea A Posch Nicole N Berger-Höger Birte B Grafe Alexander A Keppler Jan J Steckelberg A A

BMJ open 20201103 11

<h4>Introduction</h4>Health information is a prerequisite of informed decision-making. Criteria for development, content and presentation have recently been published in a corresponding guideline. Within a systematic search, 27 relevant checklists were identified, none of them, however, complying with the guideline or providing reasonably operationalised measurement items. Therefore, a draft of a checklist with 19 criteria was drafted. The current study aims at developing and validating this mea ...[more]

PMID: 33148762

Similar Datasets

Project description:BackgroundChatGPT-4 is the latest release of a novel artificial intelligence (AI) chatbot able to answer freely formulated and complex questions. In the near future, ChatGPT could become the new standard for health care professionals and patients to access medical information. However, little is known about the quality of medical information provided by the AI.ObjectiveWe aimed to assess the reliability of medical information provided by ChatGPT.MethodsMedical information provided by ChatGPT-4 on the 5 hepato-pancreatico-biliary (HPB) conditions with the highest global disease burden was measured with the Ensuring Quality Information for Patients (EQIP) tool. The EQIP tool is used to measure the quality of internet-available information and consists of 36 items that are divided into 3 subsections. In addition, 5 guideline recommendations per analyzed condition were rephrased as questions and input to ChatGPT, and agreement between the guidelines and the AI answer was measured by 2 authors independently. All queries were repeated 3 times to measure the internal consistency of ChatGPT.ResultsFive conditions were identified (gallstone disease, pancreatitis, liver cirrhosis, pancreatic cancer, and hepatocellular carcinoma). The median EQIP score across all conditions was 16 (IQR 14.5-18) for the total of 36 items. Divided by subsection, median scores for content, identification, and structure data were 10 (IQR 9.5-12.5), 1 (IQR 1-1), and 4 (IQR 4-5), respectively. Agreement between guideline recommendations and answers provided by ChatGPT was 60% (15/25). Interrater agreement as measured by the Fleiss κ was 0.78 (P<.001), indicating substantial agreement. Internal consistency of the answers provided by ChatGPT was 100%.ConclusionsChatGPT provides medical information of comparable quality to available static internet information. Although currently of limited quality, large language models could become the future standard for patients and health care professionals to gather medical information.

Project description:BackgroundDigital health technologies (DHTs), such as electronic health records and prescribing systems, are transforming health care delivery around the world. The quality of information in DHTs is key to the quality and safety of care. We developed a novel clinical information quality (CLIQ) framework to assess the quality of clinical information in DHTs.ObjectiveThis study explored clinicians' perspectives on the relevance, definition, and assessment of information quality dimensions in the CLIQ framework.MethodsWe used a systematic and iterative eDelphi approach to engage clinicians who had information governance roles or personal interest in information governance; the clinicians were recruited through purposive and snowball sampling techniques. Data were collected using semistructured online questionnaires until consensus was reached on the information quality dimensions in the CLIQ framework. Responses on the relevance of the dimensions were summarized to inform decisions on retention of the dimensions according to prespecified rules. Thematic analysis of the free-text responses was used to revise definitions and the assessment of dimensions.ResultsThirty-five clinicians from 10 countries participated in the study, which was concluded after the second round. Consensus was reached on all dimensions and categories in the CLIQ framework: informativeness (accuracy, completeness, interpretability, plausibility, provenance, and relevance), availability (accessibility, portability, security, and timeliness), and usability (conformance, consistency, and maintainability). A new dimension, searchability, was introduced in the availability category to account for the ease of finding needed information in the DHTs. Certain dimensions were renamed, and some definitions were rephrased to improve clarity.ConclusionsThe CLIQ framework reached a high expert consensus and clarity of language relating to the information quality dimensions. The framework can be used by health care managers and institutions as a pragmatic tool for identifying and forestalling information quality problems that could compromise patient safety and quality of care.International registered report identifier (irrid)RR2-10.1136/bmjopen-2021-057430.

Project description:BackgroundAs the Internet becomes the number one destination for obtaining health-related information, there is an increasing need to identify health Web pages that convey an accurate and current view of medical knowledge. In response, the research community has created multicriteria instruments for reliably assessing online medical information quality. One such instrument is DISCERN, which measures health Web page quality by assessing an array of features. In order to scale up use of the instrument, there is interest in automating the quality evaluation process by building machine learning (ML)-based DISCERN Web page classifiers.ObjectiveThe paper addresses 2 key issues that are essential before constructing automated DISCERN classifiers: (1) generation of a robust DISCERN training corpus useful for training classification algorithms, and (2) assessment of the usefulness of the current DISCERN scoring schema as a metric for evaluating the performance of these algorithms.MethodsUsing DISCERN, 272 Web pages discussing treatment options in breast cancer, arthritis, and depression were evaluated and rated by trained coders. First, different consensus models were compared to obtain a robust aggregated rating among the coders, suitable for a DISCERN ML training corpus. Second, a new DISCERN scoring criterion was proposed (features-based score) as an ML performance metric that is more reflective of the score distribution across different DISCERN quality criteria.ResultsFirst, we found that a probabilistic consensus model applied to the DISCERN instrument was robust against noise (random ratings) and superior to other approaches for building a training corpus. Second, we found that the established DISCERN scoring schema (overall score) is ill-suited to measure ML performance for automated classifiers.ConclusionUse of a probabilistic consensus model is advantageous for building a training corpus for the DISCERN instrument, and use of a features-based score is an appropriate ML metric for automated DISCERN classifiers.AvailabilityThe code for the probabilistic consensus model is available at https://bitbucket.org/A_2/em_dawid/ .

Project description:BackgroundHand hygiene is a crucial and cost-effective method to prevent health care-associated infections, and in 2009, the World Health Organization (WHO) issued guidelines to encourage and standardize hand hygiene procedures. However, a common challenge in health care settings is low adherence, leading to low handwashing quality. Recent advances in machine learning and wearable sensing have made it possible to accurately measure handwashing quality for the purposes of training, feedback, or accreditation.ObjectiveWe measured the accuracy of a sensor armband (Myo armband) in detecting the steps and duration of the WHO procedures for handwashing and handrubbing.MethodsWe recruited 20 participants (10 females; mean age 26.5 years, SD 3.3). In a semistructured environment, we collected armband data (acceleration, gyroscope, orientation, and surface electromyography data) and video data from each participant during 15 handrub and 15 handwash sessions. We evaluated the detection accuracy for different armband placements, sensor configurations, user-dependent vs user-independent models, and the use of bootstrapping.ResultsUsing a single armband, the accuracy was 96% (SD 0.01) for the user-dependent model and 82% (SD 0.08) for the user-independent model. This increased when using two armbands to 97% (SD 0.01) and 91% (SD 0.04), respectively. Performance increased when the armband was placed on the forearm (user dependent: 97%, SD 0.01; and user independent: 91%, SD 0.04) and decreased when placed on the arm (user dependent: 96%, SD 0.01; and user independent: 80%, SD 0.06). In terms of bootstrapping, user-dependent models can achieve more than 80% accuracy after six training sessions and 90% with 16 sessions. Finally, we found that the combination of accelerometer and gyroscope minimizes power consumption and cost while maximizing performance.ConclusionsA sensor armband can be used to measure hand hygiene quality relatively accurately, in terms of both handwashing and handrubbing. The performance is acceptable using a single armband worn in the upper arm but can substantially improve by placing the armband on the forearm or by using two armbands.

Project description:BackgroundConcern regarding the reliability and accuracy of the health-related information provided by online newspaper articles has increased. Numerous criteria and items have been proposed and published regarding the quality assessment of online information, but there is no standard quality assessment tool available for online newspapers.ObjectiveThis study aimed to develop the Health Information Quality Assessment Tool (HIQUAL) for online newspaper articles.MethodsWe reviewed previous health information quality assessment tools and related studies and accordingly developed and customized new criteria. The interrater agreement for the new assessment tool was assessed for 3 newspaper articles on different subjects (colorectal cancer, obesity genetic testing, and hypertension diagnostic criteria) using the Fleiss κ and Gwet agreement coefficient. To compare the quality scores generated by each pair of tools, convergent validity was measured using the Kendall τ ranked correlation.ResultsOverall, the HIQUAL for newspaper articles comprised 10 items across 5 domains: reliability, usefulness, understandability, sufficiency, and transparency. The interrater agreement for the article on colorectal cancer was in the moderate to substantial range (Fleiss κ=0.48, SE 0.11; Gwet agreement coefficient=0.74, SE 0.13), while for the article introducing obesity genetic testing it was in the substantial range, with values of 0.63 (SE 0.28) and 0.86 (SE 0.10) for the two measures, respectively. There was relatively low agreement for the article on hypertension diagnostic criteria at 0.20 (SE 0.10) and 0.75 (SE 0.13), respectively. Validity of the correlation assessed with the Kendall τ showed good correlation between tools (HIQUAL vs DISCERN=0.72, HIQUAL vs QUEST [Quality Evaluation Scoring Tool]=0.69).ConclusionsWe developed a new assessment tool to evaluate the quality of health information in online newspaper articles, to help consumers discern accurate sources of health information. The HIQUAL can help increase the accuracy and quality of online health information in Korea.

Project description:BACKGROUND:Patient-reported outcome (PRO) measures describe natural history, manage disease, and measure the effects of interventions in trials. Patients themselves increasingly use Web-based PRO tools to track their progress, share their data, and even self-experiment. However, existing PROs have limitations such as being: designed for paper (not screens), long and burdensome, negatively framed, under onerous licensing restrictions, either too generic or too specific. OBJECTIVE:This study aimed to develop and validate the core items of a modular, patient-centric, PRO system (Thrive) that could measure health status across a range of chronic conditions with minimal burden. METHODS:Thrive was developed in 4 phases, largely consistent with Food and Drug Administration guidance regarding PRO development. First, preliminary core items (common across multiple conditions: core Thrive items) were developed through literature review, analysis of approximately 20 existing PROs on PatientsLikeMe, and feedback from psychometric and content experts. Second, 2 rounds of cognitive interviews were iteratively conducted with patients (N=14) to obtain feedback on the preliminary items. Third, core Thrive items were administered electronically along with comparator measures, including 20-item Short-Form General Health Survey (SF)-20 and Patient Health Questionnaire (PHQ)-9, to a large sample (N=2002) of adults with chronic diseases through the PatientsLikeMe platform. On the basis of theoretical and empirical rationale, items were revised or removed. Fourth, the revised core Thrive items were administered to another sample of patients (N=704) with generic and condition-specific comparator measures. A psychometric evaluation, which included both modern and classical test theory approaches, was conducted on these items, and several more items were removed. RESULTS:Cognitive interviews helped to remove confusing or redundant items. Empirical testing of subscales revealed good internal consistency (Cronbach alpha=.712-.879), test-retest reliability (absolute intraclass correlations=.749-.912), and convergent validity with legacy PRO scales (eg, Pearson r=.5-.75 between Thrive subscales and PHQ-9 total). The finalized instrument consists of a 19-item core including 5 multi-item subscales: Core symptoms, Abilities, Mobility, Sleep, and Thriving. Results provide evidence of construct (content, convergent) validity, high levels of test-retest and internal consistency reliability, and the ability to detect change over time. The items did not exhibit bias based on gender or age, and the items generally functioned similarly across conditions. These results support the use of Thrive Core items across diverse chronic patient populations. CONCLUSIONS:Thrive appears to be a useful approach for capturing important domains for patients with chronic conditions. This core set serves as a foundation to begin developing modular condition-specific versions in the near future. Cross-walking against traditional PROs from the PatientsLikeMe platform is underway, in addition to clinical validation and comparison with biomarkers. Thrive is licensed under Creative Commons Attribution ShareAlike 4.0.

Project description:BackgroundBladder health is an understudied state and difficult to measure due to lack of valid and reliable instruments. While condition specific questionnaires assess presence, severity and degree of bother from lower urinary tract symptoms, the absence of symptoms is insufficient to assume bladder health. This study describes the methodology used to validate a novel bladder health instrument to measure the spectrum of bladder health from very healthy to very unhealthy in population based and clinical research.MethodsThree samples of women are being recruited: a sample from a nationally representative general population and two locally recruited clinical center samples-women with a targeted range of symptom severity and type, and a postpartum group. The general population sample includes 694 women, 18 years or older, randomly selected from a US Postal delivery sequence file. Participants are randomly assigned to electronic or paper versions of the bladder health instrument along with a battery of criterion questionnaires and a demographic survey; followed by a retest or a two-day voiding symptom diary. A total of 354 women around 7 clinical centers are being recruited across a spectrum of self-reported symptoms and randomized to mode of completion. They complete the two-day voiding symptom diary as well as a one-day frequency volume diary prior to an in-person evaluation with a standardized cough stress test, non-invasive urine flowmetry, chemical urine analysis and post void residual measurement. Independent judge ratings of bladder health are obtained by interview with a qualified health care provider. A total of 154 postpartum women recruited around 6 of the centers are completing similar assessments within 6-12 weeks postpartum. Dimensional validity will be evaluated using factor analysis and principal components analysis with varimax rotation, and internal consistency with Cronbach's alpha. Criterion validity will be assessed using multitrait-multimethod matrix including correlations across multiple data sources and multiple types of measures.DiscussionWe aim to validate a bladder health instrument to measure the degree of bladder health within the general population and among women (including postpartum) recruited from local clinical centers. Trial registration NCT04016298 Posted July 11, 2019 ( https://www.clinicaltrials.gov/ct2/show/NCT04016298?cond=bladder+health&draw=2&rank=1 ).

Project description:IntroductionAdverse events related to drug prescriptions are the main patient safety issue in primary care; however there is a lack of validated instruments for assessing the quality of prescription writing, which covers the prescriber, the patient and the drug information.ObjectiveTo develop and validate the QualiPresc instrument to assess and monitor the quality of drug prescriptions in primary care, accompanied by a self-instruction direction, with the goal of filling the gap in validated instruments to assess the quality of prescription writing.MethodologyA validation study conducted in a municipality in Northeastern Brazil, based on prescriptions prepared in January 2021 by physicians assigned to 18 Basic Health Units and filed in 6 distribution/dispensing units. Four steps were covered: 1) Analysis of content validity of each indicator (relevance and adequacy); 2) Analysis of reliability via intra and inter-rater agreement of each indicator; 3) Analysis of the utility of each indicator; 4) Construction and analysis of the reliability of a weighted composite indicator based on effectiveness and safety scores for each indicator.ResultsTwenty-nine potential indicators were listed, but only 13 were approved for validity, reliability and usefulness. Twelve indicators were excluded because of validity (<90% validity index) and four because they were not useful in the context of the study. Three weighted composite indicators were tested, but only one was approved for reliability and usefulness. The validated instrument therefore contains 13 indicators and 1 weighted composite indicator.ConclusionThis study demonstrates the validity, reliability and usefulness of QualiPresc for the evaluation of prescription writing in the context of primary care. Application to contexts such as secondary care and tertiary care requires cross-cultural adaptation and new content validity. Educators, managers and health care professionals can access QualiPresc online, free of charge, to assess performance and provide feedback involving drug prescribers.

Project description:BackgroundThis paper describes the development of the Chinese Quality of Life Instrument (ChQOL) which is a self-report health status instrument. Chinese Medicine relies very much on asking subjective feelings of patients in the process of diagnosis and monitoring of treatment. For thousands of years, Chinese Medicine practitioners have accumulated a good wealth of experiences in asking questions about health of their patients based on the concept of health in Chinese Medicine. These experiences were then transformed into questions for the ChQOL. It is believed that ChQOL can contribute to the existing Patient Report Outcome measures. This paper outlines the concept of health and disease in Traditional Chinese Medicine, the building of the conceptual framework of the ChQOL, the steps of drafting, selecting and validating the items, and the psychometric properties of the ChQOL.MethodsThe development of the ChQOL was based on the concept of health in Traditional Chinese Medicine with a theory driven approach. Based on the results of literature review, the research team developed an initial model of health which encompassed the concept of health in TCM. An expert panel was then invited to comment and give suggestions for improvement of the initial model. According to their suggestions, the model was refined and a set of initial items for the ChQOL was drafted. The refined model, together with the key domains, facets and initial items of the ChQOL were then mailed to a sample of about 100 Chinese medicine practitioners throughout Mainland China for their comments and advice. A revised set of items were developed for linguistic testing by a convenience sample consisting of both healthy people and people who attended Chinese Medicine treatment. After that, an item pool was developed for field-testing. Field test was conducted on a convenience sample of healthy and patient subjects to determine the construct validity and psychometric properties of the ChQOL.ResultsConstruct validity was established by various methods, i.e. the internal consistency in all facets and domains were good; the correlation between facets to domain, and domains to overall ChQOL correlation were high; confirmatory factor analysis showed that the structure fitness of all facets, domain and overall structure were good with CFI > 0.9. Test-retest reliability was also good, especially in the domain scores with ICC value ranging from 0.83 to 0.90. No ceiling or floor effect was noted which indicated that ChQOL can be applied to subjects with a wide range of health status. Most facet scores, domain scores and the overall CHQOL scores were able to discriminate groups of subjects with known differences in health status. The ChQOL had mild positive convergence with the other generic health related QOL measures, i.e. the WHOQOL-100 and the SF-36, with moderate correlations.ConclusionIn conclusion, the study indicated that the ChQOL is conceptually valid with satisfactory psychometric properties. It can provide additional information on health and QOL on top of the existing generic health related QOL measures. Furthermore, it forms basis for further testing and applications in clinical trials.

Dataset Information

MAPPinfo, mapping quality of health information: study protocol for a validation study of an assessment instrument.

Publications

MAPPinfo, mapping quality of health information: study protocol for a validation study of an assessment instrument.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets