Dataset Information

A Tool That Assesses the Evidence, Transparency, and Usability of Online Health Information: Development and Reliability Assessment.

ABSTRACT:

Background

The internet is commonly used by older adults to obtain health information and this trend has markedly increased in the past decade. However, studies illustrate that much of the available online health information is not informed by good quality evidence, developed in a transparent way, or easy to use. Furthermore, studies highlight that the general public lacks the skills necessary to distinguish between online products that are credible and trustworthy and those that are not. A number of tools have been developed to assess the evidence, transparency, and usability of online health information; however, many have not been assessed for reliability or ease of use.

Objective

The first objective of this study was to determine if a tool assessing the evidence, transparency, and usability of online health information exists that is easy and quick to use and has good reliability. No such tool was identified, so the second objective was to develop such a tool and assess it for reliability when used to assess online health information on topics of relevant to optimal aging.

Methods

An electronic database search was conducted between 2002 and 2012 to identify published papers describing tools that assessed the evidence, transparency, and usability of online health information. Papers were retained if the tool described was assessed for reliability, assessed the quality of evidence used to create online health information, and was quick and easy to use. When no one tool met expectations, a new instrument was developed and tested for reliability. Reliability between two raters was assessed using the intraclass correlation coefficient (ICC) for each item at two time points. SPSS Statistics 22 software was used for statistical analyses and a one-way random effects model was used to report the results. The overall ICC was assessed for the instrument as a whole in July 2015. The threshold for retaining items was ICC>0.60 (ie, "good" reliability).

Results

All tools identified that evaluated online health information were either too complex, took a long time to complete, had poor reliability, or had not undergone reliability assessment. A new instrument was developed and assessed for reliability in April 2014. Three items had an ICC<0.60 (ie, "good" reliability). One of these items was removed ("minimal scrolling") and two were retained but reworded for clarity. Four new items were added that assessed the level of research evidence that informed the online health information and the tool was retested in July 2015. The total ICC score showed excellent agreement with both single measures (ICC=0.988; CI 0.982-0.992) and average measures (ICC=0.994; CI 0.991-0.996).

Conclusions

The results of this study suggest that this new tool is reliable for assessing the evidence, transparency, and usability of online health information that is relevant to optimal aging.

SUBMITTER: Dobbins M

PROVIDER: S-EPMC6715399 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Tool That Assesses the Evidence, Transparency, and Usability of Online Health Information: Development and Reliability Assessment.

Dobbins Maureen M Watson Susannah S Read Kristin K Graham Kelly K Yousefi Nooraie Reza R Levinson Anthony J AJ

JMIR aging 20180507 1

<h4>Background</h4>The internet is commonly used by older adults to obtain health information and this trend has markedly increased in the past decade. However, studies illustrate that much of the available online health information is not informed by good quality evidence, developed in a transparent way, or easy to use. Furthermore, studies highlight that the general public lacks the skills necessary to distinguish between online products that are credible and trustworthy and those that are not ...[more]

PMID: 31518240

Similar Datasets

Project description:BackgroundConcern regarding the reliability and accuracy of the health-related information provided by online newspaper articles has increased. Numerous criteria and items have been proposed and published regarding the quality assessment of online information, but there is no standard quality assessment tool available for online newspapers.ObjectiveThis study aimed to develop the Health Information Quality Assessment Tool (HIQUAL) for online newspaper articles.MethodsWe reviewed previous health information quality assessment tools and related studies and accordingly developed and customized new criteria. The interrater agreement for the new assessment tool was assessed for 3 newspaper articles on different subjects (colorectal cancer, obesity genetic testing, and hypertension diagnostic criteria) using the Fleiss κ and Gwet agreement coefficient. To compare the quality scores generated by each pair of tools, convergent validity was measured using the Kendall τ ranked correlation.ResultsOverall, the HIQUAL for newspaper articles comprised 10 items across 5 domains: reliability, usefulness, understandability, sufficiency, and transparency. The interrater agreement for the article on colorectal cancer was in the moderate to substantial range (Fleiss κ=0.48, SE 0.11; Gwet agreement coefficient=0.74, SE 0.13), while for the article introducing obesity genetic testing it was in the substantial range, with values of 0.63 (SE 0.28) and 0.86 (SE 0.10) for the two measures, respectively. There was relatively low agreement for the article on hypertension diagnostic criteria at 0.20 (SE 0.10) and 0.75 (SE 0.13), respectively. Validity of the correlation assessed with the Kendall τ showed good correlation between tools (HIQUAL vs DISCERN=0.72, HIQUAL vs QUEST [Quality Evaluation Scoring Tool]=0.69).ConclusionsWe developed a new assessment tool to evaluate the quality of health information in online newspaper articles, to help consumers discern accurate sources of health information. The HIQUAL can help increase the accuracy and quality of online health information in Korea.

Project description:BACKGROUND:Smartphone apps are increasingly used for diabetes self-management because of their ubiquity and ability to help users to personalize health care management. The number of diabetes apps has proliferated in recent years, but only a small subset of apps that pose a higher risk are regulated by governmental agencies. The transparency and reliability of information sources are unclear for apps that provide health care advice and are not regulated by governmental agencies. OBJECTIVE:This study aimed to assess the transparency and reliability of information disseminated via diabetes apps against 8 criteria adapted from the Health On the Net code of conduct (HONcode) principles. METHODS:English-language diabetes-related terms were searched on a market explorer (42matters) on June 12, 2018. Apps with medication and blood glucose management features were downloaded and evaluated against the App-HONcode criteria adapted from the 8 HONcode principles: authoritative, complementarity, privacy, attribution, justifiability, transparency, financial disclosure, and advertising policy. Apps were profiled by operating platforms (ie, Android and iOS) and the number of downloads (ie, Android only: ?100,000 downloads and <100,000 downloads). RESULTS:A total of 143 apps (81 Android and 62 iOS) were downloaded and assessed against the adapted App-HONcode criteria. Most of the apps on the Android and iOS platforms fulfilled between 2 and 6 criteria, but few (20/143, 14.0%) apps mentioned the qualifications of individuals who contributed to app development. Less than half (59/143, 39.2%) of the apps disclaimed that the information provided or app functions do not replace the advice of the health care provider. A higher proportion of iOS apps fulfilled 5 or more App-HONcode criteria compared with Android apps. However, Android apps were more likely to have the developer's email listed on the app store (Android: 75/81, 98%; and iOS: 52/62, 84%; P=.005) compared with iOS apps. Of the Android apps assessed, a significantly higher proportion of highly downloaded apps had a privacy and confidentiality clause (high downloads: 15/17, 88%; and low downloads: 33/64, 52%; P=.006) and were more likely to discuss their financial sources (high downloads: 14/17, 82%; and low downloads: 32/64, 50%; P=.03) compared with apps with a low number of downloads. CONCLUSIONS:Gaps in the disclosure of the developer's qualification, funding source, and the complementary role of the app in disease management were identified. App stores, developers, and medical providers should collaborate to close these gaps and provide more transparency and reliability to app users. Future work can further examine the consent-seeking process for data collection, data management policies, the appropriateness of advertising content, and clarity of privacy clause of these apps.

Project description:BackgroundMedication is the most common intervention in health care, and the number of online consumer information systems within the pharmaceutical sector is increasing. However, online consumer information systems can be a barrier for users, imposing information asymmetries between stakeholders.ObjectiveThe objective of this study was to quantify and compare the usability of an online consumer medication information system (OCMIS) against a reference implementation based on an interoperable information model for patients, physicians, and pharmacists.MethodsQuantitative and qualitative data were acquired from patients, physicians, and pharmacists in this online usability study. We administered 3 use cases and a post hoc questionnaire per user. Quantitative usability data including effectiveness (task success), efficiency (task time), and user satisfaction (system usability scale [SUS]) was complemented by qualitative and demographic data. Users evaluated 6 existing systems and 1 reference implementation of an OCMIS.ResultsA total of 137 patients, 81 physicians, and 68 pharmacists participated in this study. Task success varied from 84% to 92% in patients, 66% to 100% in physicians, and 50% to 91% in pharmacists. Task completion time decreased over the course of the study for all but 2 OCMIS within the patient group. Due to an assumed nonnormal distribution of SUS scores, within-group comparison was done using the Kruskal-Wallis test. Patients showed differences in SUS scores (P=.02) and task time (P=.03), while physicians did not have significant differences in SUS scores (P=.83) and task time (P=.72). For pharmacists, a significant difference in SUS scores (P<.001) and task time (P=.007) was detected.ConclusionsThe vendor-neutral reference implementation based on an interoperable information model was proven to be a promising approach that was not inferior to existing solutions for patients and physicians. For pharmacists, it exceeded user satisfaction scores compared to other OCMIS. This data-driven approach based on an interoperable information model enables the development of more user-tailored features to increase usability. This fosters data democratization and empowers stakeholders within the pharmaceutical sector.

Project description:BackgroundCommon information facilities do not always provide the quality information needed to answer questions on health or health-related issues, such as Occupational Safety and Health (OSH) matters. Barriers may be the accessibility, quantity and readability of information. Online Question & Answer (Q&A) network tools, which link questioners directly to experts can overcome some of these barriers. When designing and testing online tools, assessing the usability and applicability is essential. Therefore, the purpose of this study is to assess the usability and applicability of a new online Q&A network tool for answers on OSH questions.MethodsWe applied a cross-sectional usability test design. Eight occupational health experts and twelve potential questioners from the working population (workers) were purposively selected to include a variety of computer- and internet-experiences. During the test, participants were first observed while executing eight tasks that entailed important features of the tool. In addition, they were interviewed. Through task observations and interviews we assessed applicability, usability (effectiveness, efficiency and satisfaction) and facilitators and barriers in use.ResultsMost features were usable, though several could be improved. Most tasks were executed effectively. Some tasks, for example searching stored questions in categories, were not executed efficiently and participants were less satisfied with the corresponding features. Participants' recommendations led to improvements. The tool was found mostly applicable for additional information, to observe new OSH trends and to improve contact between OSH experts and workers. Hosting and support by a trustworthy professional organization, effective implementation campaigns, timely answering and anonymity were seen as important use requirements.ConclusionsThis network tool is a promising new strategy for offering company workers high quality information to answer OSH questions. Q&A network tools can be an addition to existing information facilities in the field of OSH, but also to other healthcare fields struggling with how to answer questions from people in practice with high quality information. In the near future, we will focus on the use of the tool and its effects on information and knowledge dissemination.

Project description:BackgroundThe German Biobank Alliance (GBA) aims to establish a cross-site biobank network. For this endeavor, the so-called Sample Locator, a federated search tool for biospecimens and related data, has been developed, forming the heart of its information technology (IT) infrastructure.ObjectiveTo ensure the sustainable use of such a tool, we included researchers as participants in an end user-based usability evaluation.MethodsTo develop a prototype ready for evaluation, we needed input from GBA IT experts. Thus, we conducted a 2-day workshop with 8 GBA IT team members. The focus was on the respective steps of a user-centered design process. With the acquired knowledge, the participants designed low-fidelity mock-ups. The main ideas of these mock-ups were discussed, extracted, and summarized into a comprehensive prototype using Microsoft PowerPoint. Furthermore, we created a questionnaire concerning the usability of the prototype, including the System Usability Scale (SUS), questions on negative and positive aspects, and typical tasks to be fulfilled with the tool. Subsequently, the prototype was pretested on the basis of this questionnaire with researchers who have a biobank background. Based on this preliminary work, the usability analysis was ultimately carried out with researchers and the results were evaluated.ResultsAltogether, 27 researchers familiar with sample requests evaluated the prototype. The analysis of the feedback certified a good usability, given that the Sample Locator prototype was seen as intuitive and user-friendly by 74% (20/27) of the participants. The total SUS score by the 25 persons that completed the questionnaire was 80.4, indicating good system usability. Still, the evaluation provided useful advice on optimization potential (eg, offering a help function).ConclusionsThe findings of this usability analysis indicate that the considerations regarding a user-friendly application that have been made in the development process so far strongly coincide with the perception of the study participants. Nevertheless, it was important to engage prospective end users to ensure that the previous development is going in the desired direction and that the Sample Locator will be used in the future. The user comments and suggestions for improvement will be considered in upcoming iterations for refinement.

Dataset Information

A Tool That Assesses the Evidence, Transparency, and Usability of Online Health Information: Development and Reliability Assessment.

Background

Objective

Methods

Results

Conclusions

Publications

A Tool That Assesses the Evidence, Transparency, and Usability of Online Health Information: Development and Reliability Assessment.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets