Safe and Sample-Efficient Reinforcement Learning Algorithms for Factored Environments

Safe and Sample-Efficient Reinforcement Learning Algorithms for Factored Environments

Thiago D. Simão

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Doctoral Consortium. Pages 6460-6461. https://doi.org/10.24963/ijcai.2019/919

Reinforcement Learning (RL) deals with problems that can be modeled as a Markov Decision Process (MDP) where the transition function is unknown. In situations where an arbitrary policy pi is already in execution and the experiences with the environment were recorded in a batch D, an RL algorithm can use D to compute a new policy pi'. However, the policy computed by traditional RL algorithms might have worse performance compared to pi. Our goal is to develop safe RL algorithms, where the agent has a high confidence that the performance of pi' is better than the performance of pi given D. To develop sample-efficient and safe RL algorithms we combine ideas from exploration strategies in RL with a safe policy improvement method.
Keywords:
Machine Learning: Reinforcement Learning
Planning and Scheduling: Planning under Uncertainty
Machine Learning: Learning Graphical Models
Machine Learning: Probabilistic Machine Learning