March 2022 CDT in Maths of Random Systems Workshop

Jonathan Tam: Markov decision processes with observation costs
Christoph Reisinger, Jonathan Tam
We present a framework for a controlled Markov chain where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies therefore involve the choice of observation times as well as the subsequent control values. We show that the corresponding value function satisfies a dynamic programming principle, which leads to a system of quasi-variational inequalities (QVIs). Next, we give an extension where the model parameters are not known a priori but are inferred from the costly observations by Bayesian updates. We then prove a comparison principle for a larger class of QVIs, which implies uniqueness of solutions to our proposed problem. We utilise penalty methods to obtain arbitrarily accurate solutions. Finally, we perform numerical experiments on three applications which illustrate our framework.

Preprint at https://arxiv.org/abs/2201.07908

Remy Messadene: Signature asymptotics, empirical processes, and optimal transport

My research interests lie at the intersection of rough path theory in machine learning, reinforcement learning, stochastic control theory and finance.

Professor Julien Berestycki, Associate Professor of Probability. Department of Statistics, University of Oxford

My research is in probability theory and focuses essentially on models and situations which involve tree-like structures and branching phenomena. Examples include coalescent processes, branching processes, continuous random trees, branching random walks… These models are not only endowed with a remarkably rich mathematical structure that connects them to many area of mathematics, but they also occur naturally in physical sciences, in population genetics and in biology. Questions that arise in these fields are a major motivation of my work.