Dataset Information

Automatic Identification of Information Quality Metrics in Health News Stories.

ABSTRACT: Objective: Many online and printed media publish health news of questionable trustworthiness and it may be difficult for laypersons to determine the information quality of such articles. The purpose of this work was to propose a methodology for the automatic assessment of the quality of health-related news stories using natural language processing and machine learning. Materials and Methods: We used a database from the website HealthNewsReview.org that aims to improve the public dialogue about health care. HealthNewsReview.org developed a set of criteria to critically analyze health care interventions' claims. In this work, we attempt to automate the evaluation process by identifying the indicators of those criteria using natural language processing-based machine learning on a corpus of more than 1,300 news stories. We explored features ranging from simple n-grams to more advanced linguistic features and optimized the feature selection for each task. Additionally, we experimented with the use of pre-trained natural language model BERT. Results: For some criteria, such as mention of costs, benefits, harms, and "disease-mongering," the evaluation results were promising with an F₁ measure reaching 81.94%, while for others the results were less satisfactory due to the dataset size, the need of external knowledge, or the subjectivity in the evaluation process. Conclusion: These used criteria are more challenging than those addressed by previous work, and our aim was to investigate how much more difficult the machine learning task was, and how and why it varied between criteria. For some criteria, the obtained results were promising; however, automated evaluation of the other criteria may not yet replace the manual evaluation process where human experts interpret text senses and make use of external knowledge in their assessment.

SUBMITTER: Al-Jefri M

PROVIDER: S-EPMC7775604 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Automatic Identification of Information Quality Metrics in Health News Stories.

Al-Jefri Majed M Evans Roger R Lee Joon J Ghezzi Pietro P

Frontiers in public health 20201218

<b>Objective:</b> Many online and printed media publish health news of questionable trustworthiness and it may be difficult for laypersons to determine the information quality of such articles. The purpose of this work was to propose a methodology for the automatic assessment of the quality of health-related news stories using natural language processing and machine learning. <b>Materials and Methods:</b> We used a database from the website HealthNewsReview.org that aims to improve the public di ...[more]

PMID: 33392124

Similar Datasets

Project description:BackgroundThe news media is a key source for health and medical information, and relies to a large degree on material from press releases (PR). Medical universities are key players in the dissemination of PRs. This study aims to 1) explore the relation between the quality of press releases (PRs) from medical universities and their corresponding news stories (NSs) and 2) to identify the likelihood that specific scientific and interest-raising measures appear or are omitted in PRs and NSs.Methods and findingsIn this retrospective study using quantitative content analysis, PRs (n = 507) from 21 medical universities in Germany, the Netherlands, Sweden, the USA and the UK were retrieved. Of all PRs, 33% had media coverage, resulting in 496 NSs. With two codebooks, 18 scientific (e.g. reporting the study design of the study correctly) and 7 interest-raising measures (e.g. words like 'ground-breaking') were evaluated in the PRs and NSs. For all measures the percentage of presence in NSs and PRs was calculated, together with a Mean PR Influence Factor. Quality of PRs and NSs was defined as a score, based on 12 of the 18 scientific measures. Mean (SD) NS quality score was 6.5 (1.7) which was significantly lower than the PR score of 8.0 (1.5). The two quality scores were significantly correlated. Quality measures that were frequently omitted included reporting important study limitations (present in 21% of PRs, 21% of NSs), funding (59% of PRs, 7% of NSs) and conflicts of interest (16% of PRs, 3% of NSs). We did not evaluate the quality of the scientific papers (SPs), and can therefore not determine if the quality of PRs and NSs is associated with the quality of SPs.ConclusionsThis large study of medical university press releases and corresponding news stories showed that important measures of a scientific study such as funding and study limitations were omitted to a very large extent. The lay public and health personnel as well as policy makers, politicians and other decision makers may be misled by incomplete and partly inaccurate representations of scientific studies which could negatively affect important health-related behaviours and decisions.

Project description:Previous research on news perception has been dominated by a cognitively oriented perspective on reception processes, whereas emotions have been widely neglected. Consequently, it has remained open which features of a news story might elicit affective responses and hence modulate news perception, shifting the focus to the emotional potential of the narrative. According to the affective-disposition theory, the experience of suspense is the striving force of immersion in fictional dramas. Thereby, a positive affective disposition toward the protagonist of a story and a high likelihood of a bad ending should increase suspense that, in turn, should positively influence reading appreciation and lingering interest in the story. We investigated whether suspense and its determinants also play such a key role in the context of news stories. Study 1 (n = 263) successfully replicated results of an earlier study, whereas Studies 2 (n = 255) and 3 (n = 599) challenged the generalizability of some effects related to manipulated characteristics of a news story. In contrast, correlational relationships between perceived news characteristics and news evaluation were relatively stable. In particular, participants' liking of the protagonist and the perceived likelihood of a good ending were positively associated with suspense, reading appreciation, and lingering interest. This result indicates a preference for happy endings and contradicts the notion that likely negative outcomes are beneficial for suspense and the enjoyment of news stories, as postulated by the affective-disposition theory in the context of fictional dramas. Moreover, experienced suspense reliably mediated the correlations between, on the one hand, participants' liking of the protagonist and the perceived likelihood of a good ending and, on the other hand, reading appreciation and lingering interest. The news story's personal relevance was less influential than expected. Further, we observed a large absence of interaction effects, indicating that central characteristics of a news story can be independently varied to a large degree. In a nutshell, we may conclude that suspense significantly mediates the correlation between perceived news characteristics and the enjoyment of news stories, whereas manipulations of news characteristics do not necessarily influence the enjoyment of narratives as desired.

Project description:Journalists' health and science reporting aid the public's direct access to research through the inclusion of hyperlinks leading to original studies in peer-reviewed journals. While this effort supports the US-government mandate that research be made widely available, little is known about what research journalists share with the public. This cross-sectional exploratory study characterises US-government-funded research on cancer that appeared most frequently in news coverage and how that coverage varied by cancer type, disease incidence and mortality rates. The subject of analysis was 11436 research articles (published in 2016) on cancer funded by the US government and 642 news stories mentioning at least one of these articles. Based on Altmetric data, researchers identified articles via PubMed and characterised each based on the news media attention received online. Only 1.88% (n = 213) of research articles mentioning US government-funded cancer research included at least one mention in an online news publication. This is in contrast to previous research that found 16.8% (n = 1925) of articles received mention by online mass media publications. Of the 13 most common cancers in the US, 12 were the subject of at least one news mention; only urinary and bladder cancer received no mention. Traditional news sources included significantly more mentions of research on common cancers than digital native news sources. However, a general discrepancy exists between cancers prominent in news sources and those with the highest mortality rate. For instance, lung cancer accounted for the most deaths annually, while melanoma led to 56% less annual deaths; however, journalists cited research regarding these cancers nearly equally. Additionally, breast cancer received the greatest coverage per estimated annual death, while pancreatic cancer received the least coverage per death. Findings demonstrated a continued misalignment between prevalent cancers and cancers mentioned in online news media. Additionally, cancer control and prevention received less coverage from journalists than other cancer continuum stages, highlighting a continued underrepresentation of prevention-focused research. Results revealed a need for further scholarship regarding the role of journalists in research dissemination.

Project description:Background:Coverage of e-cigarettes in the news media may shape public perceptions about them but little is known about such news content. This content analysis characterized discussion of e-cigarettes in leading print and online US news sources in 2015. Methods:We searched Access World News and Factiva databases for e-cigarette-related news articles appearing in the top 30 circulating newspapers, 4 newswires, and 4 online news sources in the United States in 2015 (n = 295). Coders identified the presence of various e-cigarette topics (e.g. regulation), and benefit and risk statements. Results:Nearly half of articles (45.1%) focused primarily on e-cigarette policy/regulatory issues, although e-cigarette prevalence (21.0%) and health effects (21.7%) were common main topics. Concerns about youth were frequently mentioned, including the rise in youth e-cigarette use (45.4%), gateway to smoking potential (33.9%) and appeal of flavors (22.4%). Youth e-cigarette prevalence was more frequently mentioned than adult prevalence in articles discussing FDA regulation (61% vs. 13.5%, respectively). News articles more frequently discussed potential e-cigarette risks or concerns (80%) than benefits (45.4%), such as smoking harm-reduction. Quoted physicians, researchers, and government representatives were more likely to refer to e-cigarette risks than benefits. Conclusions:In 2015, rising rates of e-cigarette use among youth and policy strategies to address e-cigarettes dominated US e-cigarette news stories, leading up to their FDA regulation in 2016. Statements about e-cigarettes' potential risks were frequently attributed to trusted sources such as physicians, and outnumbered claims about their harm-reduction benefits. Such coverage may impact e-cigarette risk perceptions, use intentions and policy support. Implications:In the year leading up to the FDA's Deeming Rule, concerns about youth use or potential use were frequently discussed in e-cigarette news. News articles more frequently discussed potential e-cigarette risks/concerns compared to potential harm-reduction benefits relative to tobacco cigarettes. While such coverage may inform the public about potential e-cigarette risks, they may also contribute to increasing perceptions that e-cigarettes are as harmful as tobacco cigarettes. E-cigarette risk and benefit statements were most frequently made by or attributed to researchers and physicians in articles, which is significant given that they may be particularly trusted sources of e-cigarette risk information.

Project description:Today's health care environment encourages health care consumers to take an active role in managing their health. As digital natives, young educated adults do much of their health information management through the Internet and consider it a valid source of health advice. However, the quality of information on health websites is highly variable and dynamic. Little is known about the understandings and perceptions that young educated adults have garnered on the quality of information on health websites used for health care-related purposes.To fill this gap, the aim of this study was to develop a conceptual framework of health website information quality with quality dimensions (ie, criteria) and associated quality drivers (ie, attributes) specified in the context of young educated adults' use of health websites for health care-related purposes. This aim was achieved by (1) identifying information quality dimensions of health websites from the perspective of young educated adults; (2) identifying the importance ratings of these quality dimensions; and (3) constructing a framework of health website information quality with quality dimensions and associated drivers specified in the context of young educated adults' use of health websites for health care-related purposes.The study employed both qualitative and quantitative methods. Methods included semistructured group interviews and an individual quality assessment exercise grounded in visiting various websites and responding to Likert scale questions regarding the importance ratings of information quality dimensions and open-ended questions with specifying website quality drivers. Study participants included junior and senior undergraduate and graduate students in business, allied health, and public health majors. Qualitative, open-coding procedures were used to develop the conceptual framework reflecting the participants' means of assessing information quality on health websites.Five dimensions of information quality for health websites were identified: Completeness of information, Understandability of information, Relevance of information, Depth of information, and Accuracy of information. Completeness of information and Understandability of information were rated as the two most important quality dimensions by the study participants. Results indicated that these five information quality dimensions for health websites were supported by the following main driver themes: Content, Design, Links, Consumer resources, Search functionality, Supporting references, User focus, Content FAQ, Open access, Policy statements, and Site performance.This study contributes to the literature by developing a health website information quality conceptual framework with quality dimensions and associated drivers specified for a young educated adult population. The detailed quality drivers supporting the corresponding quality dimensions provide a rich picture of young educated adults' perceptions on health website information quality. This framework can be used to guide the development of health websites, as well as the foundation for a means to evaluate health information from existing health websites with young educated adults as the target audience.

Project description:BackgroundCorrelation metrics are widely utilized in genomics analysis and often implemented with little regard to assumptions of normality, homoscedasticity, and independence of values. This is especially true when comparing values between replicated sequencing experiments that probe chromatin accessibility, such as assays for transposase-accessible chromatin via sequencing (ATAC-seq). Such data can possess several regions across the human genome with little to no sequencing depth and are thus non-normal with a large portion of zero values. Despite distributed use in the epigenomics field, few studies have evaluated and benchmarked how correlation and association statistics behave across ATAC-seq experiments with known differences or the effects of removing specific outliers from the data. Here, we developed a computational simulation of ATAC-seq data to elucidate the behavior of correlation statistics and to compare their accuracy under set conditions of reproducibility.ResultsUsing these simulations, we monitored the behavior of several correlation statistics, including the Pearson's R and Spearman's [Formula: see text] coefficients as well as Kendall's [Formula: see text] and Top-Down correlation. We also test the behavior of association measures, including the coefficient of determination R[Formula: see text], Kendall's W, and normalized mutual information. Our experiments reveal an insensitivity of most statistics, including Spearman's [Formula: see text], Kendall's [Formula: see text], and Kendall's W, to increasing differences between simulated ATAC-seq replicates. The removal of co-zeros (regions lacking mapped sequenced reads) between simulated experiments greatly improves the estimates of correlation and association. After removing co-zeros, the R[Formula: see text] coefficient and normalized mutual information display the best performance, having a closer one-to-one relationship with the known portion of shared, enhanced loci between simulated replicates. When comparing values between experimental ATAC-seq data using a random forest model, mutual information best predicts ATAC-seq replicate relationships.ConclusionsCollectively, this study demonstrates how measures of correlation and association can behave in epigenomics experiments. We provide improved strategies for quantifying relationships in these increasingly prevalent and important chromatin accessibility assays.

Dataset Information

Automatic Identification of Information Quality Metrics in Health News Stories.

Publications

Automatic Identification of Information Quality Metrics in Health News Stories.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets