Using prediction polling to harness collective intelligence for disease forecasting
Ontology highlight
ABSTRACT: Background
The global spread of COVID-19 has shown that reliable forecasting of public health related outcomes is important but lacking. Methods
We report the results of the first large-scale, long-term experiment in crowd-forecasting of infectious-disease outbreaks, where a total of 562 volunteer participants competed over 15 months to make forecasts on 61 questions with a total of 217 possible answers regarding 19 diseases. Results
Consistent with the “wisdom of crowds” phenomenon, we found that crowd forecasts aggregated using best-practice adaptive algorithms are well-calibrated, accurate, timely, and outperform all individual forecasters. Conclusions
Crowd forecasting efforts in public health may be a useful addition to traditional disease surveillance, modeling, and other approaches to evidence-based decision making for infectious disease outbreaks. Supplementary Information
The online version contains supplementary material available at 10.1186/s12889-021-12083-y.
SUBMITTER: Sell T
PROVIDER: S-EPMC8605461 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Short-term wind speed forecasting for Colonia Eulacio, Soriano Department, Uruguay, is performed by applying an artificial neural network (ANN) technique to the hourly time series representative of the site. To train the ANN and validate the technique, data for one year are collected by one tower, with anemometers installed at heights of 101.8, 81.8, 25.7, and 10.0 m. Different ANN configurations are applied for each site and height; then, a quantitative analysis is conducted, and the statistical results are evaluated to select the configuration that best predicts the real data. This method has lower computational costs than other techniques, such as numerical modelling. For integrating wind power into existing grid systems, accurate short-term wind speed forecasting is fundamental. Therefore, the proposed short-term wind speed forecasting method is an important scientific contribution for reliable large-scale wind power forecasting and integration in Uruguay. The results of the short-term wind speed forecasting showed good accuracy at all the anemometer heights tested, suggesting that the method is a powerful tool that can help the Administración Nacional de Usinas y Transmissiones Eléctricas manage the national energy supply.
| S-EPMC6517329 | biostudies-other
Project description:Collective intelligence is the ability of a group to perform more effectively than any individual alone. Diversity among group members is a key condition for the emergence of collective intelligence, but maintaining diversity is challenging in the face of social pressure to imitate one's peers. Through an evolutionary game-theoretic model of collective prediction, we investigate the role that incentives may play in maintaining useful diversity. We show that market-based incentive systems produce herding effects, reduce information available to the group, and restrain collective intelligence. Therefore, we propose an incentive scheme that rewards accurate minority predictions and show that this produces optimal diversity and collective predictive accuracy. We conclude that real world systems should reward those who have shown accuracy when the majority opinion has been in error.
| S-EPMC5441831 | biostudies-literature
Project description:BACKGROUND:Innovative ways of planning and conducting research have emerged recently, based on the concept of collective intelligence. Collective intelligence is defined as shared intelligence emerging when people are mobilized within or outside an organization to work on a specific task that could result in more innovative outcomes than those when individuals work alone. Crowdsourcing is defined as "the act of taking a job traditionally performed by a designated agent and outsourcing it to an undefined, generally large group of people in the form of an open call." OBJECTIVE:This qualitative study aimed to identify the barriers to mobilizing collective intelligence and ways to overcome these barriers and provide good practice advice for planning and conducting collective intelligence projects across different research disciplines. METHODS:We conducted a multinational online open-ended question survey and semistructured audio-recorded interviews with a purposive sample of researchers who had experience in running collective intelligence projects. The questionnaires had an interactive component, enabling respondents to rate and comment on the advice of their fellow respondents. Data were analyzed thematically, drawing on the framework method. RESULTS:A total of 82 respondents from various research fields participated in the survey (n=65) or interview (n=17). The main barriers identified were the lack of evidence-based guidelines for implementing collective intelligence, complexity in recruiting and engaging the community, and difficulties in disseminating the results of collective intelligence projects. We drew on respondents' experience to provide tips and good practice advice for governance, planning, and conducting collective intelligence projects. Respondents particularly suggested establishing a diverse coordination team to plan and manage collective intelligence projects and setting up common rules of governance for participants in projects. In project planning, respondents provided advice on identifying research problems that could be answered by collective intelligence and identifying communities of participants. They shared tips on preparing the task and interface and organizing communication activities to recruit and engage participants. CONCLUSIONS:Mobilizing collective intelligence through crowdsourcing is an innovative method to increase research efficiency, although there are several barriers to its implementation. We present good practice advice from researchers with experience of collective intelligence across different disciplines to overcome barriers to mobilizing collective intelligence.
| S-EPMC6632103 | biostudies-literature
Project description:Collective intelligence (CI) is critical to solving many scientific, business, and other problems, but groups often fail to achieve it. Here, we analyze data on group performance from 22 studies, including 5,279 individuals in 1,356 groups. Our results support the conclusion that a robust CI factor characterizes a group's ability to work together across a diverse set of tasks. We further show that CI is predicted by the proportion of women in the group, mediated by average social perceptiveness of group members, and that it predicts performance on various out-of-sample criterion tasks. We also find that, overall, group collaboration process is more important in predicting CI than the skill of individual members.
| S-EPMC8166150 | biostudies-literature
Project description:While collective intelligence (CI) is a powerful approach to increase decision accuracy, few attempts have been made to unlock its potential in medical decision-making. Here we investigated the performance of three well-known collective intelligence rules ("majority", "quorum", and "weighted quorum") when applied to mammography screening. For any particular mammogram, these rules aggregate the independent assessments of multiple radiologists into a single decision (recall the patient for additional workup or not). We found that, compared to single radiologists, any of these CI-rules both increases true positives (i.e., recalls of patients with cancer) and decreases false positives (i.e., recalls of patients without cancer), thereby overcoming one of the fundamental limitations to decision accuracy that individual radiologists face. Importantly, we find that all CI-rules systematically outperform even the best-performing individual radiologist in the respective group. Our findings demonstrate that CI can be employed to improve mammography screening; similarly, CI may have the potential to improve medical decision-making in a much wider range of contexts, including many areas of diagnostic imaging and, more generally, diagnostic decisions that are based on the subjective interpretation of evidence.
| S-EPMC4534443 | biostudies-literature
Project description:This article investigates cybersecurity (and risk) in the context of ‘technological singularity’ from artificial intelligence. The investigation constructs multiple risk forecasts that are synthesised in a new framework for counteracting risks from artificial intelligence (AI) itself. In other words, the research in this article is not just concerned with securing a system, but also analysing how the system responds when (internal and external) failure(s) and compromise(s) occur. This is an important methodological principle because not all systems can be secured, and totally securing a system is not feasible. Thus, we need to construct algorithms that will enable systems to continue operating even when parts of the system have been compromised. Furthermore, the article forecasts emerging cyber-risks from the integration of AI in cybersecurity. Based on the forecasts, the article is concentrated on creating synergies between the existing literature, the data sources identified in the survey, and forecasts. The forecasts are used to increase the feasibility of the overall research and enable the development of novel methodologies that uses AI to defend from cyber risks. The methodology is focused on addressing the risk of AI attacks, as well as to forecast the value of AI in defence and in the prevention of AI rogue devices acting independently. Supplementary Information The online version contains supplementary material available at 10.1007/s12530-022-09431-7.
| S-EPMC9166151 | biostudies-literature
Project description:Accurate forecasting of tourist demand is important to both business practitioners and government policymakers. In the past decade of rapid development of deep learning, many artificial intelligence methods or deep learning models have been built to improve prediction accuracy. But data-driven end-to-end deep network models usually require large data sets to support. For tourism forecasting, the sample is insufficient and many models are difficult to apply. In this article, we propose a novel hybrid model GM-LSTM, which combines the advantages of gray models and neural networks to achieve self-adaptive prediction with small samples. Specifically, the overall trend of tourism demand is captured by a first-order gray model and the non-linear residual fluctuation is characterized using a long short-term memory (LSTM) network with a rolling mechanism. The model is validated through a case study of up to 38 years of data on annual international tourist arrivals in Xi'an, China. The proposed GM-LSTM model achieved a predicted MAPE value of 11.88%, outperforming other time series models. The results indicate that our proposed hybrid model is accurate and efficient.
| S-EPMC10703057 | biostudies-literature
Project description:The exploration-exploitation dilemma is a recurrent adaptive problem for humans as well as non-human animals. Given a fixed time/energy budget, every individual faces a fundamental trade-off between exploring for better resources and exploiting known resources to optimize overall performance under uncertainty. Colonies of eusocial insects are known to solve this dilemma successfully via evolved coordination mechanisms that function at the collective level. For humans and other non-eusocial species, however, this dilemma operates within individuals as well as between individuals, because group members may be motivated to take excessive advantage of others' exploratory findings through social learning. Thus, even though social learning can reduce collective exploration costs, the emergence of disproportionate "information scroungers" may severely undermine its potential benefits. We investigated experimentally whether social learning opportunities might improve the performance of human participants working on a "multi-armed bandit" problem in groups, where they could learn about each other's past choice behaviors. Results showed that, even though information scroungers emerged frequently in groups, social learning opportunities reduced total group exploration time while increasing harvesting from better options, and consequentially improved collective performance. Surprisingly, enriching social information by allowing participants to observe others' evaluations of chosen options (e.g., Amazon's 5-star rating system) in addition to choice-frequency information had a detrimental impact on performance compared to the simpler situation with only the choice-frequency information. These results indicate that humans groups can handle the fundamental "dual exploration-exploitation dilemmas" successfully, and that social learning about simple choice-frequencies can help produce collective intelligence.
| S-EPMC3995913 | biostudies-literature
Project description:People influence each other when they interact to solve problems. Such social influence introduces both benefits (higher average solution quality due to exploitation of existing answers through social learning) and costs (lower maximum solution quality due to a reduction in individual exploration for novel answers) relative to independent problem solving. In contrast to prior work, which has focused on how the presence and network structure of social influence affect performance, here we investigate the effects of time. We show that when social influence is intermittent it provides the benefits of constant social influence without the costs. Human subjects solved the canonical traveling salesperson problem in groups of three, randomized into treatments with constant social influence, intermittent social influence, or no social influence. Groups in the intermittent social-influence treatment found the optimum solution frequently (like groups without influence) but had a high mean performance (like groups with constant influence); they learned from each other, while maintaining a high level of exploration. Solutions improved most on rounds with social influence after a period of separation. We also show that storing subjects' best solutions so that they could be reloaded and possibly modified in subsequent rounds-a ubiquitous feature of personal productivity software-is similar to constant social influence: It increases mean performance but decreases exploration.
| S-EPMC6126746 | biostudies-literature
Project description:Lyme disease is the most common tick-borne disease in the Northern Hemisphere. Existing estimates of Lyme disease spread are delayed a year or more. We introduce Lymelight-a new method for monitoring the incidence of Lyme disease in real-time. We use a machine-learned classifier of web search sessions to estimate the number of individuals who search for possible Lyme disease symptoms in a given geographical area for two years, 2014 and 2015. We evaluate Lymelight using the official case count data from CDC and find a 92% correlation (p < 0.001) at county level. Importantly, using web search data allows us not only to assess the incidence of the disease, but also to examine the appropriateness of treatments subsequently searched for by the users. Public health implications of our work include monitoring the spread of vector-borne diseases in a timely and scalable manner, complementing existing approaches through real-time detection, which can enable more timely interventions. Our analysis of treatment searches may also help reduce misdiagnosis of the disease.
| S-EPMC7000681 | biostudies-literature