Wednesday, July 12, 2023 - 12:00 in ZiF
Primal-dual regression approach for Markov decision processes with general state and action spaces
A talk in the SPDEs, optimal control and mean field games series by
John Schoenmakers from Berlin
| Abstract: |
We develop a regression-based primal-dual martingale approach for solving discrete
time,finite horizon MDPs with state and action spaces that are general in the
sense that they may be finite or infinite (but regular enough) subsets of Euclidean
space. As a result, our method allows for the construction of tight upper and
lower biased approximations of the value functions and provides tight approximations
to the optimal policy. In particular, we prove error bounds for the estimated
duality gap featuring polynomial dependence on the time horizon and sublinear
dependence of the stochastic part of the error on the cardinality/dimension of
the state and action spaces. From a computational point of view, the proposed
method is efficient since, in contrast to the usual duality-based methods for
optimal control problems in the literature, the Monte Carlo procedures involved
here do not require nested simulations. Joint work with D. Belomestny.
|
Back
