Dataset Information

Using search engine big data for predicting new HIV diagnoses.

ABSTRACT:

Background

A large and growing body of "big data" is generated by internet search engines, such as Google. Because people often search for information about public health and medical issues, researchers may be able to use search engine data to monitor and predict public health problems, such as HIV. We sought to assess the feasibility of using Google search data to analyze and predict new HIV diagnoses cases in the United States.

Methods and findings

From 2007 to 2014, we collected search volume data on HIV-related Google search keywords across the United States. State-level new HIV diagnoses data were collected from the Centers for Disease Control and Prevention (CDC) and AIDSVu.org. We developed a negative binomial model to predict HIV cases using a subset of significant predictor keywords identified by LASSO. The Google search data were combined with state-level HIV case reports provided by the CDC. We use historical data to train the model and predict new HIV diagnoses from 2011 to 2014, with an average R2 value of 0.99 between predicted versus actual cases, and average root-mean-square error (RMSE) of 108.75.

Conclusions

Results indicate that Google Trends is a feasible tool to predict new cases of HIV at the state level. We discuss the implications of integrating visualization maps and tools based on these models into public health and HIV monitoring and surveillance.

SUBMITTER: Young SD

PROVIDER: S-EPMC6042696 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using search engine big data for predicting new HIV diagnoses.

Young Sean D SD Zhang Qingpeng Q

PloS one 20180712 7

<h4>Background</h4>A large and growing body of "big data" is generated by internet search engines, such as Google. Because people often search for information about public health and medical issues, researchers may be able to use search engine data to monitor and predict public health problems, such as HIV. We sought to assess the feasibility of using Google search data to analyze and predict new HIV diagnoses cases in the United States.<h4>Methods and findings</h4>From 2007 to 2014, we collecte ...[more]

PMID: 30001360

Similar Datasets

Project description:BackgroundGiven the ongoing COVID-19 pandemic situation, accurate predictions could greatly help in the health resource management for future waves. However, as a new entity, COVID-19's disease dynamics seemed difficult to predict. External factors, such as internet search data, need to be included in the models to increase their accuracy. However, it remains unclear whether incorporating online search volumes into models leads to better predictive performances for long-term prediction.ObjectiveThe aim of this study was to analyze whether search engine query data are important variables that should be included in the models predicting new daily COVID-19 cases and deaths in short- and long-term periods.MethodsWe used country-level case-related data, NAVER search volumes, and mobility data obtained from Google and Apple for the period of January 20, 2020, to July 31, 2021, in South Korea. Data were aggregated into four subsets: 3, 6, 12, and 18 months after the first case was reported. The first 80% of the data in all subsets were used as the training set, and the remaining data served as the test set. Generalized linear models (GLMs) with normal, Poisson, and negative binomial distribution were developed, along with linear regression (LR) models with lasso, adaptive lasso, and elastic net regularization. Root mean square error values were defined as a loss function and were used to assess the performance of the models. All analyses and visualizations were conducted in SAS Studio, which is part of the SAS OnDemand for Academics.ResultsGLMs with different types of distribution functions may have been beneficial in predicting new daily COVID-19 cases and deaths in the early stages of the outbreak. Over longer periods, as the distribution of cases and deaths became more normally distributed, LR models with regularization may have outperformed the GLMs. This study also found that models performed better when predicting new daily deaths compared to new daily cases. In addition, an evaluation of feature effects in the models showed that NAVER search volumes were useful variables in predicting new daily COVID-19 cases, particularly in the first 6 months of the outbreak. Searches related to logistical needs, particularly for "thermometer" and "mask strap," showed higher feature effects in that period. For longer prediction periods, NAVER search volumes were still found to constitute an important variable, although with a lower feature effect. This finding suggests that search term use should be considered to maintain the predictive performance of models.ConclusionsNAVER search volumes were important variables in short- and long-term prediction, with higher feature effects for predicting new daily COVID-19 cases in the first 6 months of the outbreak. Similar results were also found for death predictions.

Project description:BackgroundTobacco smoking is an important public health issue and a core indicator of public health policy worldwide. However, global pandemics and natural disasters have prevented surveys from being conducted.ObjectiveThe purpose of this study was to predict smoking prevalence by prefecture and sex in Japan using Internet search trends.MethodsThis study used the infodemiology approach. The outcome variable was smoking prevalence by prefecture, obtained from national surveys. The predictor variables were the search volumes on Yahoo! Japan Search. We collected the search volumes for queries related to terms from the thesaurus of the Japanese medical article database Ichu-shi. Predictor variables were converted to per capita values and standardized as z scores. For smoking prevalence, the values for 2016 and 2019 were used, and for search volume, the values for the April 1 to March 31 fiscal year (FY) 1 year prior to the survey (ie, FY 2015 and FY 2018) were used. Partial correlation coefficients, adjusted for data year, were calculated between smoking prevalence and search volume, and a regression analysis using a generalized linear mixed model with random effects was conducted for each prefecture. Several models were tested, including a model that included all search queries, a variable reduction method, and one that excluded cigarette product names. The best model was selected with the Akaike information criterion corrected (AICC) for small sample size and the Bayesian information criterion (BIC). We compared the predicted and actual smoking prevalence in 2016 and 2019 based on the best model and predicted the smoking prevalence in 2022.ResultsThe partial correlation coefficients for men showed that 9 search queries had significant correlations with smoking prevalence, including cigarette (r=-0.417, P<.001), cigar in kanji (r=-0.412, P<.001), and cigar in katakana (r=-0.399, P<.001). For women, five search queries had significant correlations, including vape (r=0.335, P=.001), quitting smoking (r=0.288, P=.005), and cigar (r=0.286, P=.006). The models with all search queries were the best models for both AICC and BIC scores. Scatter plots of actual and estimated smoking prevalence in 2016 and 2019 confirmed a relatively high degree of agreement. The average estimated smoking prevalence in 2022 in the 47 prefectures for the total sample was 23.492% (95% CI 21.617%-25.367%), showing an increasing trend, with an average of 29.024% (95% CI 27.218%-30.830%) for men and 8.793% (95% CI 7.531%-10.054%) for women.ConclusionsThis study suggests that the search volume of tobacco-related queries in internet search engines can predict smoking prevalence by prefecture and sex in Japan. These findings will enable the development of low-cost, timely, and crisis-resistant health indicators that will enable the evaluation of health measures and contribute to improved public health.

Project description:BackgroundAnxiety disorders are the most prevalent mental disorders globally, with a substantial impact on quality of life. The prevalence of anxiety disorders has increased substantially following the COVID-19 pandemic, and it is likely to be further affected by a global economic recession. Understanding anxiety themes and how they change over time and across countries is crucial for preventive and treatment strategies.ObjectiveThe aim of this study was to track the trends in anxiety themes between 2004 and 2020 in the 50 most populous countries with high volumes of internet search data. This study extends previous research by using a novel search-based methodology and including a longer time span and more countries at different income levels.MethodsWe used a crowdsourced questionnaire, alongside Bing search query data and Google Trends search volume data, to identify themes associated with anxiety disorders across 50 countries from 2004 to 2020. We analyzed themes and their mutual interactions and investigated the associations between countries' socioeconomic attributes and anxiety themes using time-series linear models. This study was approved by the Microsoft Research Institutional Review Board.ResultsQuery volume for anxiety themes was highly stable in countries from 2004 to 2019 (Spearman r=0.89) and moderately correlated with geography (r=0.49 in 2019). Anxiety themes were predominantly long-term and personal, with "having kids," "pregnancy," and "job" the most voluminous themes in most countries and years. In 2020, "COVID-19" became a dominant theme in 27 countries. Countries with a constant volume of anxiety themes over time had lower fragile state indexes (P=.007) and higher individualism (P=.003). An increase in the volume of the most searched anxiety themes was associated with a reduction in the volume of the remaining themes in 13 countries and an increase in 17 countries, and these 30 countries had a lower prevalence of mental disorders (P<.001) than the countries where no correlations were found.ConclusionsInternet search data could be a potential source for predicting the country-level prevalence of anxiety disorders, especially in understudied populations or when an in-person survey is not viable.

Project description:ObjectivesHand, foot, and mouth disease (HFMD) has caused a substantial burden in China, especially in Guangdong Province. Based on the enhanced surveillance system, we aimed to explore whether the addition of temperate and search engine query data improves the risk prediction of HFMD.DesignEcological study.Setting and participantsInformation on the confirmed cases of HFMD, climate parameters and search engine query logs was collected. A total of 1.36 million HFMD cases were identified from the surveillance system during 2011-2014. Analyses were conducted at aggregate level and no confidential information was involved.Outcome measuresA seasonal autoregressive integrated moving average (ARIMA) model with external variables (ARIMAX) was used to predict the HFMD incidence from 2011 to 2014, taking into account temperature and search engine query data (Baidu Index, BDI). Statistics of goodness-of-fit and precision of prediction were used to compare models (1) based on surveillance data only, and with the addition of (2) temperature, (3) BDI, and (4) both temperature and BDI.ResultsA high correlation between HFMD incidence and BDI (r=0.794, p<0.001) or temperature (r=0.657, p<0.001) was observed using both time series plot and correlation matrix. A linear effect of BDI (without lag) and non-linear effect of temperature (1 week lag) on HFMD incidence were found in a distributed lag non-linear model. Compared with the model based on surveillance data only, the ARIMAX model including BDI reached the best goodness-of-fit with an Akaike information criterion (AIC) value of -345.332, whereas the model including both BDI and temperature had the most accurate prediction in terms of the mean absolute percentage error (MAPE) of 101.745%.ConclusionsAn ARIMAX model incorporating search engine query data significantly improved the prediction of HFMD. Further studies are warranted to examine whether including search engine query data also improves the prediction of other infectious diseases in other settings.

Dataset Information

Using search engine big data for predicting new HIV diagnoses.

Background

Methods and findings

Conclusions

Publications

Using search engine big data for predicting new HIV diagnoses.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets