Dataset Information

Towards Identifying and Reducing the Bias of Disease Information Extracted from Search Engine Data.

ABSTRACT: The estimation of disease prevalence in online search engine data (e.g., Google Flu Trends (GFT)) has received a considerable amount of scholarly and public attention in recent years. While the utility of search engine data for disease surveillance has been demonstrated, the scientific community still seeks ways to identify and reduce biases that are embedded in search engine data. The primary goal of this study is to explore new ways of improving the accuracy of disease prevalence estimations by combining traditional disease data with search engine data. A novel method, Biased Sentinel Hospital-based Area Disease Estimation (B-SHADE), is introduced to reduce search engine data bias from a geographical perspective. To monitor search trends on Hand, Foot and Mouth Disease (HFMD) in Guangdong Province, China, we tested our approach by selecting 11 keywords from the Baidu index platform, a Chinese big data analyst similar to GFT. The correlation between the number of real cases and the composite index was 0.8. After decomposing the composite index at the city level, we found that only 10 cities presented a correlation of close to 0.8 or higher. These cities were found to be more stable with respect to search volume, and they were selected as sample cities in order to estimate the search volume of the entire province. After the estimation, the correlation improved from 0.8 to 0.864. After fitting the revised search volume with historical cases, the mean absolute error was 11.19% lower than it was when the original search volume and historical cases were combined. To our knowledge, this is the first study to reduce search engine data bias levels through the use of rigorous spatial sampling strategies.

SUBMITTER: Huang DC

PROVIDER: S-EPMC4894584 | biostudies-literature | 2016 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Towards Identifying and Reducing the Bias of Disease Information Extracted from Search Engine Data.

Huang Da-Cang DC Wang Jin-Feng JF Huang Ji-Xia JX Sui Daniel Z DZ Zhang Hong-Yan HY Hu Mao-Gui MG Xu Cheng-Dong CD

PLoS computational biology 20160606 6

The estimation of disease prevalence in online search engine data (e.g., Google Flu Trends (GFT)) has received a considerable amount of scholarly and public attention in recent years. While the utility of search engine data for disease surveillance has been demonstrated, the scientific community still seeks ways to identify and reduce biases that are embedded in search engine data. The primary goal of this study is to explore new ways of improving the accuracy of disease prevalence estimations b ...[more]

PMID: 27271698

Similar Datasets

Project description:BackgroundAnxiety disorders are the most prevalent mental disorders globally, with a substantial impact on quality of life. The prevalence of anxiety disorders has increased substantially following the COVID-19 pandemic, and it is likely to be further affected by a global economic recession. Understanding anxiety themes and how they change over time and across countries is crucial for preventive and treatment strategies.ObjectiveThe aim of this study was to track the trends in anxiety themes between 2004 and 2020 in the 50 most populous countries with high volumes of internet search data. This study extends previous research by using a novel search-based methodology and including a longer time span and more countries at different income levels.MethodsWe used a crowdsourced questionnaire, alongside Bing search query data and Google Trends search volume data, to identify themes associated with anxiety disorders across 50 countries from 2004 to 2020. We analyzed themes and their mutual interactions and investigated the associations between countries' socioeconomic attributes and anxiety themes using time-series linear models. This study was approved by the Microsoft Research Institutional Review Board.ResultsQuery volume for anxiety themes was highly stable in countries from 2004 to 2019 (Spearman r=0.89) and moderately correlated with geography (r=0.49 in 2019). Anxiety themes were predominantly long-term and personal, with "having kids," "pregnancy," and "job" the most voluminous themes in most countries and years. In 2020, "COVID-19" became a dominant theme in 27 countries. Countries with a constant volume of anxiety themes over time had lower fragile state indexes (P=.007) and higher individualism (P=.003). An increase in the volume of the most searched anxiety themes was associated with a reduction in the volume of the remaining themes in 13 countries and an increase in 17 countries, and these 30 countries had a lower prevalence of mental disorders (P<.001) than the countries where no correlations were found.ConclusionsInternet search data could be a potential source for predicting the country-level prevalence of anxiety disorders, especially in understudied populations or when an in-person survey is not viable.

Dataset Information

Towards Identifying and Reducing the Bias of Disease Information Extracted from Search Engine Data.

Publications

Towards Identifying and Reducing the Bias of Disease Information Extracted from Search Engine Data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets