Dataset Information

A survey of human judgement and quantitative forecasting methods.

ABSTRACT: This paper's top-level goal is to provide an overview of research conducted in the many academic domains concerned with forecasting. By providing a summary encompassing these domains, this survey connects them, establishing a common ground for future discussions. To this end, we survey literature on human judgement and quantitative forecasting as well as hybrid methods that involve both humans and algorithmic approaches. The survey starts with key search terms that identified more than 280 publications in the fields of computer science, operations research, risk analysis, decision science, psychology and forecasting. Results show an almost 10-fold increase in the application-focused forecasting literature between the 1990s and the current decade, with a clear rise of quantitative, data-driven forecasting models. Comparative studies of quantitative methods and human judgement show that (1) neither method is universally superior, and (2) the better method varies as a function of factors such as availability, quality, extent and format of data, suggesting that (3) the two approaches can complement each other to yield more accurate and resilient models. We also identify four research thrusts in the human/machine-forecasting literature: (i) the choice of the appropriate quantitative model, (ii) the nature of the interaction between quantitative models and human judgement, (iii) the training and incentivization of human forecasters, and (iv) the combination of multiple forecasts (both algorithmic and human) into one. This review surveys current research in all four areas and argues that future research in the field of human/machine forecasting needs to consider all of them when investigating predictive performance. We also address some of the ethical dilemmas that might arise due to the combination of quantitative models with human judgement.

SUBMITTER: Zellner M

PROVIDER: S-EPMC8074796 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundStrengthening the national health information system is one of Ethiopia's priority transformation agendas. A well-trained and competent workforce is the essential ingredient to a strong health information system. However, this workforce has neither been quantified nor characterized well, and there is no roadmap of required human resources to enhance the national health information system.ObjectiveWe aimed to determine the current state of the health information system workforce and to forecast the human resources needed for the health information system by 2030.MethodsWe conducted a survey to estimate the current number of individuals employed in the health information system unit and the turnover rate. Document review and key-informant interviews were used to collect current human resources and available health information system position data from 110 institutions, including the Ministry of Health, federal agencies, regional health bureaus, zonal health departments, district health offices, and health facilities. The Delphi technique was used to forecast human resources required for the health information system in the next ten years: 3 rounds of workshops with experts from the Ministry of Health, universities, agencies, and regional health bureaus were held. In the first expert meeting, we set criteria, which was followed by expert suggestions and feedback.ResultsAs of April 2020, there were 10,344 health information system professionals working in the governmental health system. Nearly 95% (20/21) of district health offices and 86.7% (26/30) of health centers reported that the current number of health information system positions was inadequate. In the period from June 2015 to June 2019, health information technicians had high turnover (48/244, 19.7%) at all levels of the health system. In the next ten years, we estimate that 50,656 health information system professionals will be needed to effectively implement the Ethiopia's national health information system.ConclusionsCurrent health information system-related staffing levels were found to be inadequate. To meet the estimated need of 50,656 multidisciplinary health information system professionals by 2030, the Ministry of Health and regional health bureaus, in collaboration with partners and academic institutions, need to work on retaining existing and training additional health information system professionals.

Project description:ObjectiveTo illustrate the benefits of deploying individual headache forecasting models using continuous updating with Bayesian methods.BackgroundThe ability to reliably forecast headache attacks within an individual over time would enhance the study of attacks and allow preemptive treatment. However, deploying a suitable forecasting model in a clinical setting will likely involve several unique challenges related to heterogeneity in the predictor weights, limited or sparse data, and the need for a quick "warm-up." The use of Bayesian methods offers solutions to each of these specific challenges.MethodsThis was a post hoc analysis of a cohort study of individuals with episodic migraine attacks. Individuals completed daily diaries that allowed the estimation of several forecasting models, each using different types of ancillary information incorporated into formal prior probability distributions. An in silico analysis was conducted that mimicked the deployment of these models in a clinical-like setting where the parameters of the models were continuously updated and evaluated each day using root mean square error (RMSE).ResultsIndividuals (N = 95) were followed for 50 days and contributed 3359 days of nonmissing diary data. During the observation period, there were 1293/3359 (38.5%) days with a headache attack. Self-reported baseline headache frequency was associated with the corresponding predicted probability of future attacks, r = 0.15-0.39. At Day 25, the correlation between baseline information and predicted attack likelihood was r = 0.29 (95% CI: 0.09-0.47). Additionally, the use of prior probability distributions for model parameters improved the model fit, especially early in the deployment of the models (e.g., Day 5 RMSE 0.45 vs. 0.43). Models using informative prior probability distributions outperformed the models estimated without this information during the first 42 days of observation, although performance became more similar as more data were collected.ConclusionsThis analysis demonstrates the value of Bayesian methods in using additional available information to improve forecasting model performance, especially early in the deployment of a forecasting model. To obtain the full value of such models or to apply any model in clinical settings, a model with adequate discrimination and calibration will be needed.

Dataset Information

A survey of human judgement and quantitative forecasting methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets