Dataset Information

Consumer Health Search on the Web: Study of Web Page Understandability and Its Integration in Ranking Algorithms.

ABSTRACT: BACKGROUND:Understandability plays a key role in ensuring that people accessing health information are capable of gaining insights that can assist them with their health concerns and choices. The access to unclear or misleading information has been shown to negatively impact the health decisions of the general public. OBJECTIVE:The aim of this study was to investigate methods to estimate the understandability of health Web pages and use these to improve the retrieval of information for people seeking health advice on the Web. METHODS:Our investigation considered methods to automatically estimate the understandability of health information in Web pages, and it provided a thorough evaluation of these methods using human assessments as well as an analysis of preprocessing factors affecting understandability estimations and associated pitfalls. Furthermore, lessons learned for estimating Web page understandability were applied to the construction of retrieval methods, with specific attention to retrieving information understandable by the general public. RESULTS:We found that machine learning techniques were more suitable to estimate health Web page understandability than traditional readability formulae, which are often used as guidelines and benchmark by health information providers on the Web (larger difference found for Pearson correlation of .602 using gradient boosting regressor compared with .438 using Simple Measure of Gobbledygook Index with the Conference and Labs of the Evaluation Forum eHealth 2015 collection). CONCLUSIONS:The findings reported in this paper are important for specialized search services tailored to support the general public in seeking health advice on the Web, as they document and empirically validate state-of-the-art techniques and settings for this domain application.

SUBMITTER: Palotti J

PROVIDER: S-EPMC6372940 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundAs the Internet becomes the number one destination for obtaining health-related information, there is an increasing need to identify health Web pages that convey an accurate and current view of medical knowledge. In response, the research community has created multicriteria instruments for reliably assessing online medical information quality. One such instrument is DISCERN, which measures health Web page quality by assessing an array of features. In order to scale up use of the instrument, there is interest in automating the quality evaluation process by building machine learning (ML)-based DISCERN Web page classifiers.ObjectiveThe paper addresses 2 key issues that are essential before constructing automated DISCERN classifiers: (1) generation of a robust DISCERN training corpus useful for training classification algorithms, and (2) assessment of the usefulness of the current DISCERN scoring schema as a metric for evaluating the performance of these algorithms.MethodsUsing DISCERN, 272 Web pages discussing treatment options in breast cancer, arthritis, and depression were evaluated and rated by trained coders. First, different consensus models were compared to obtain a robust aggregated rating among the coders, suitable for a DISCERN ML training corpus. Second, a new DISCERN scoring criterion was proposed (features-based score) as an ML performance metric that is more reflective of the score distribution across different DISCERN quality criteria.ResultsFirst, we found that a probabilistic consensus model applied to the DISCERN instrument was robust against noise (random ratings) and superior to other approaches for building a training corpus. Second, we found that the established DISCERN scoring schema (overall score) is ill-suited to measure ML performance for automated classifiers.ConclusionUse of a probabilistic consensus model is advantageous for building a training corpus for the DISCERN instrument, and use of a features-based score is an appropriate ML metric for automated DISCERN classifiers.AvailabilityThe code for the probabilistic consensus model is available at https://bitbucket.org/A_2/em_dawid/ .

Project description:BACKGROUND:Three main changes were implemented in the Australian National Cervical Screening Program (NCSP) in December 2017: an increase in the recommended age to start screening, extended screening intervals, and change from the Papanicolaou (Pap) test to primary human papillomavirus screening (cervical screening test). The internet is a readily accessible source of information to explain the reasons for these changes to the public. It is important that web-based health information about changes to national screening programs is accessible and understandable for the general population. OBJECTIVE:This study aimed to evaluate Australian web-based resources that provide information about the changes to the cervical screening program. METHODS:The term cervical screening was searched in 3 search engines. The first 10 relevant results across the first 3 pages of each search engine were selected. Overall, 2 authors independently evaluated each website for readability (Flesch Reading Ease [FRE], Flesch-Kincaid Grade Level, and Simple Measure of Gobbledygook [SMOG] index), quality of information (Patient Education Materials Assessment Tool [PEMAT] for printable materials), credibility (Journal of the American Medical Association [JAMA] benchmark criteria and presence of Health on the Net Foundation code of conduct [HONcode] certification), website design, and usability with 5 simulation questions to assess the relevance of information. A descriptive analysis was conducted for the readability measures, PEMAT, and the JAMA benchmark criteria. RESULTS:Of the 49 websites identified in the search, 15 were eligible for inclusion. The consumer-focused websites were classed as fairly difficult to read (mean FRE score 51.8, SD 13.3). The highest FRE score (easiest to read) was 70.4 (Cancer Council Australia Cervical Screening Consumer Site), and the lowest FRE score (most difficult to read) was 33.0 (NCSP Clinical Guidelines). A total of 9 consumer-focused websites and 4 health care provider-focused websites met the recommended threshold (sixth to eighth grade; SMOG index) for readability. The mean PEMAT understandability scores were 87.7% (SD 6.0%) for consumer-focused websites and 64.9% (SD 13.8%) for health care provider-focused websites. The mean actionability scores were 58.1% (SD 19.1%) for consumer-focused websites and 36.7% (SD 11.0%) for health care provider-focused websites. Moreover, 9 consumer-focused and 3 health care provider-focused websites scored above 70% for understandability, and 2 consumer-focused websites had an actionability score above 70%. A total of 3 websites met all 4 of the JAMA benchmark criteria, and 2 websites displayed the HONcode. CONCLUSIONS:It is important for women to have access to information that is at an appropriate reading level to better understand the implications of the changes to the cervical screening program. These findings can help health care providers direct their patients toward websites that provide information on cervical screening that is written at accessible reading levels and has high understandability.

Project description:BACKGROUND:Health consumers are often targeted for their involvement in health research including randomized controlled trials, focus groups, interviews, and surveys. However, as reported by many studies, recruitment and engagement of consumers in academic research remains challenging. In addition, there is scarce literature describing what consumers look for and want to achieve by participating in research. OBJECTIVE:Understanding and responding to the needs of consumers is crucial to the success of health research projects. In this study, we aim to understand consumers' needs and investigate the opportunities for addressing these needs with Web-based technologies, particularly in the use of Web-based research registers and social networking sites (SNSs). METHODS:We undertook a qualitative approach, interviewing both consumer and medical researchers in this study. With the help from an Australian-based organization supporting people with musculoskeletal conditions, we successfully interviewed 23 consumers and 10 researchers. All interviews were transcribed and analyzed with thematic analysis methodology. Data collection was stopped after the data themes reached saturation. RESULTS:We found that consumers perceive research as a learning opportunity and, therefore, expect high research transparency and regular updates. They also consider the sources of the information about research projects, the trust between consumers and researchers, and the mobility of consumers before participating in any research. Researchers need to be aware of such needs when designing a campaign for recruitment for their studies. On the other hand, researchers have attempted to establish a rapport with consumer participants, design research for consumers' needs, and use technologies to reach out to consumers. A systematic approach to integrating a variety of technologies is needed. CONCLUSIONS:On the basis of the feedback from both consumers and researchers, we propose 3 future directions to use Web-based technologies for addressing consumers' needs and engaging with consumers in health research: (1) researchers can make use of consumer registers and Web-based research portals, (2) SNSs and new media should be frequently used as an aid, and (3) new technologies should be adopted to remotely collect data and reduce administrative work for obtaining consumers' consent.

Dataset Information

Consumer Health Search on the Web: Study of Web Page Understandability and Its Integration in Ranking Algorithms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets