Dataset Information

A reinforcement learning model to inform optimal decision paths for HIV elimination

ABSTRACT: The ‘Ending the HIV Epidemic (EHE)’ national plan aims to reduce annual HIV incidence in the United States from 38,000 in 2015 to 9,300 by 2025 and 3,300 by 2030. Diagnosis and treatment are two most effective interventions, and thus, identifying corresponding optimal combinations of testing and retention-in-care rates would help inform implementation of relevant programs. Considering the dynamic and stochastic complexity of the disease and the time dynamics of decision-making, solving for optimal combinations using commonly used methods of parametric optimization or exhaustive evaluation of pre-selected options are infeasible. Reinforcement learning (RL), an artificial intelligence method, is ideal; however, training RL algorithms and ensuring convergence to optimality are computationally challenging for large-scale stochastic problems. We evaluate its feasibility in the context of the EHE goal. We trained an RL algorithm to identify a ‘sequence’ of combinations of HIV-testing and retention-in-care rates at 5-year intervals over 2015–2070 that optimally leads towards HIV elimination. We defined optimality as a sequence that maximizes quality-adjusted-life-years lived and minimizes HIV-testing and care-and-treatment costs. We show that solving for testing and retention-in-care rates through appropriate reformulation using proxy decision-metrics overcomes the computational challenges of RL. We used a stochastic agent-based simulation to train the RL algorithm. As there is variability in support-programs needed to address barriers to care-access, we evaluated the sensitivity of optimal decisions to three cost-functions. The model suggests to scale-up retention-in-care programs to achieve and maintain high annual retention-rates while initiating with a high testing-frequency but relaxing it over a 10-year period as incidence decreases. Results were mainly robust to the uncertainty in costs. However, testing and retention-in-care alone did not achieve the 2030 EHE targets, suggesting the need for additional interventions. The results from the model demonstrated convergence. RL is suitable for evaluating phased public health decisions for infectious disease control.

SUBMITTER: Khatami S

PROVIDER: S-EPMC8613448 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundSurgical patients incur preventable harm from cognitive and judgment errors made under time constraints and uncertainty regarding patients' diagnoses and predicted response to treatment. Decision analysis and techniques of reinforcement learning theoretically can mitigate these challenges but are poorly understood and rarely used clinically. This review seeks to promote an understanding of decision analysis and reinforcement learning by describing their use in the context of surgical decision-making.MethodsCochrane, EMBASE, and PubMed databases were searched from their inception to June 2019. Included were 41 articles about cognitive and diagnostic errors, decision-making, decision analysis, and machine-learning. The articles were assimilated into relevant categories according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines.ResultsRequirements for time-consuming manual data entry and crude representations of individual patients and clinical context compromise many traditional decision-support tools. Decision analysis methods for calculating probability thresholds can inform population-based recommendations that jointly consider risks, benefits, costs, and patient values but lack precision for individual patient-centered decisions. Reinforcement learning, a machine-learning method that mimics human learning, can use a large set of patient-specific input data to identify actions yielding the greatest probability of achieving a goal. This methodology follows a sequence of events with uncertain conditions, offering potential advantages for personalized, patient-centered decision-making. Clinical application would require secure integration of multiple data sources and attention to ethical considerations regarding liability for errors and individual patient preferences.ConclusionTraditional decision-support tools are ill-equipped to accommodate time constraints and uncertainty regarding diagnoses and the predicted response to treatment, both of which often impair surgical decision-making. Decision analysis and reinforcement learning have the potential to play complementary roles in delivering high-value surgical care through sound judgment and optimal decision-making.

Dataset Information

A reinforcement learning model to inform optimal decision paths for HIV elimination

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets