Stochastic dynamic resource allocation for HIV prevention and treatment: An approximate dynamic programming approach

• Published in: Computers & industrial engineering Vol. 118; pp. 423 - 439
• Main Authors: Coşgun, Özlem, Esra Büyüktahtakın, İ.
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01-04-2018
• Subjects: Approximate dynamic programming; Disease transmission uncertainty; HIV epidemic disease; Interventions; Markov decision process (MDP); Resource allocation; Stochastic dynamic programming; Markov decision process (MDP); Interventions; HIV epidemic disease; Resource allocation; Stochastic dynamic programming; Disease transmission uncertainty; Approximate dynamic programming
• ISSN: 0360-8352; 1879-0550
• DOI: 10.1016/j.cie.2018.01.018

•A stochastic approximate DP (ADP) approach to optimal resource allocation for intervening HIV.•Minimize the number of HIV-infected, people with AIDS, and deaths over a multi-year horizon.•Non-stationary Markov Chain disease model is applied to black men who have sex with men.•ADP allocates the budget dynamically as the uncertainty in disease transmission is revealed.•ADP approach provides significant health benefits and cost savings in HIV intervention.
Human immunodeficiency virus (HIV) is a key global health concern, with 33 million people living with HIV worldwide and 2.7 million new infections occurring annually. To prevent the spread of this widely prevalent epidemic disease, prevention and treatment intervention strategies urgently need to be implemented. The goal of this study is to propose stochastic dynamic programming (SDP) and approximate dynamic programming (ADP) algorithms that will optimally allocate the limited intervention budget among the HIV disease compartments and determine the best set of interventions that should be applied to each disease compartment, while minimizing the number of HIV-infected and people diagnosed with acquired immune deficiency syndrome (AIDS) as well as related deaths over a multi-year planning horizon. A compartmental model is constructed and formulated as a nonstationary Markov decision process (MDP) in order to capture the progression of the disease among the highest risk group—African American/black men who have sex with men (BMSM). In order to alleviate the computational difficulties arising from the exponentially growing state space in the SDP, we propose ADP algorithms that determine the approximately optimal budget allocation policies over six years. Our results suggest a greater allocation of the limited budget to prevention measures rather than treatment interventions, such as antiretroviral therapy (ART). As opposed to traditional policies that allocate the budget only once at the beginning of the time horizon, the ADP model suggests using a dynamic proportional budget strategy, allocating the budget dynamically over a multi-period planning period as the uncertainty in disease transmission is revealed. Results show that our ADP approach provides significant increases in health benefits and cost savings in HIV prevention and intervention.