Dataset Information

Exploring optimal control of epidemic spread using reinforcement learning.

ABSTRACT: Pandemic defines the global outbreak of a disease having a high transmission rate. The impact of a pandemic situation can be lessened by restricting the movement of the mass. However, one of its concomitant circumstances is an economic crisis. In this article, we demonstrate what actions an agent (trained using reinforcement learning) may take in different possible scenarios of a pandemic depending on the spread of disease and economic factors. To train the agent, we design a virtual pandemic scenario closely related to the present COVID-19 crisis. Then, we apply reinforcement learning, a branch of artificial intelligence, that deals with how an individual (human/machine) should interact on an environment (real/virtual) to achieve the cherished goal. Finally, we demonstrate what optimal actions the agent perform to reduce the spread of disease while considering the economic factors. In our experiment, we let the agent find an optimal solution without providing any prior knowledge. After training, we observed that the agent places a long length lockdown to reduce the first surge of a disease. Furthermore, the agent places a combination of cyclic lockdowns and short length lockdowns to halt the resurgence of the disease. Analyzing the agent's performed actions, we discover that the agent decides movement restrictions not only based on the number of the infectious population but also considering the reproduction rate of the disease. The estimation and policy of the agent may improve the human-strategy of placing lockdown so that an economic crisis may be avoided while mitigating an infectious disease.

SUBMITTER: Ohi AQ

PROVIDER: S-EPMC7744528 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Exploring optimal control of epidemic spread using reinforcement learning.

Ohi Abu Quwsar AQ Mridha M F MF Monowar Muhammad Mostafa MM Hamid Md Abdul MA

Scientific reports 20201216 1

Pandemic defines the global outbreak of a disease having a high transmission rate. The impact of a pandemic situation can be lessened by restricting the movement of the mass. However, one of its concomitant circumstances is an economic crisis. In this article, we demonstrate what actions an agent (trained using reinforcement learning) may take in different possible scenarios of a pandemic depending on the spread of disease and economic factors. To train the agent, we design a virtual pandemic sc ...[more]

PMID: 33328551

Similar Datasets

Project description:The 'Ending the HIV Epidemic (EHE)' national plan aims to reduce annual HIV incidence in the United States from 38,000 in 2015 to 9300 by 2025 and 3300 by 2030. Diagnosis and treatment are two most effective interventions, and thus, identifying corresponding optimal combinations of testing and retention-in-care rates would help inform implementation of relevant programs. Considering the dynamic and stochastic complexity of the disease and the time dynamics of decision-making, solving for optimal combinations using commonly used methods of parametric optimization or exhaustive evaluation of pre-selected options are infeasible. Reinforcement learning (RL), an artificial intelligence method, is ideal; however, training RL algorithms and ensuring convergence to optimality are computationally challenging for large-scale stochastic problems. We evaluate its feasibility in the context of the EHE goal. We trained an RL algorithm to identify a 'sequence' of combinations of HIV-testing and retention-in-care rates at 5-year intervals over 2015-2070 that optimally leads towards HIV elimination. We defined optimality as a sequence that maximizes quality-adjusted-life-years lived and minimizes HIV-testing and care-and-treatment costs. We show that solving for testing and retention-in-care rates through appropriate reformulation using proxy decision-metrics overcomes the computational challenges of RL. We used a stochastic agent-based simulation to train the RL algorithm. As there is variability in support-programs needed to address barriers to care-access, we evaluated the sensitivity of optimal decisions to three cost-functions. The model suggests to scale-up retention-in-care programs to achieve and maintain high annual retention-rates while initiating with a high testing-frequency but relaxing it over a 10-year period as incidence decreases. Results were mainly robust to the uncertainty in costs. However, testing and retention-in-care alone did not achieve the 2030 EHE targets, suggesting the need for additional interventions. The results from the model demonstrated convergence. RL is suitable for evaluating phased public health decisions for infectious disease control.

Project description:Coronavirus disease 2019 (COVID-19) is a disease caused by severe acute respiratory syndrome coronavirus 2 (SARS CoV-2). It was declared on March 11, 2020, by the World Health Organization as pandemic disease. The disease has neither approved medicine nor vaccine and has made governments and scholars search for drastic measures in combating the pandemic. Regrettably, the spread of the virus and mortality due to COVID-19 has continued to increase daily. Hence, it is imperative to control the spread of the disease particularly using nonpharmacological strategies such as quarantine, isolation, and public health education. This work studied the effect of these different control strategies as time-dependent interventions using mathematical modeling and optimal control approach to ascertain their contributions in the dynamic transmission of COVID-19. The model was proven to have an invariant region and was well-posed. The basic reproduction number and effective reproduction numbers were computed with and without interventions, respectively, and were used to carry out the sensitivity analysis that identified the critical parameters contributing to the spread of COVID-19. The optimal control analysis was carried out using the Pontryagin's maximum principle to figure out the optimal strategy necessary to curtail the disease. The findings of the optimal control analysis and numerical simulations revealed that time-dependent interventions reduced the number of exposed and infected individuals compared to time-independent interventions. These interventions were time-bound and best implemented within the first 100 days of the outbreak. Again, the combined implementation of only two of these interventions produced a good result in reducing infection in the population. While, the combined implementation of all three interventions performed better, even though zero infection was not achieved in the population. This implied that multiple interventions need to be deployed early in order to reduce the virus to the barest minimum.

Dataset Information

Exploring optimal control of epidemic spread using reinforcement learning.

Publications

Exploring optimal control of epidemic spread using reinforcement learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure