Inverse Reinforcement Learning for Team Sports: Valuing Actions and Players

Inverse Reinforcement Learning for Team Sports: Valuing Actions and Players

Yudong Luo, Oliver Schulte, Pascal Poupart

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Main track. Pages 3356-3363. https://doi.org/10.24963/ijcai.2020/464

A major task of sports analytics is to rank players based on the impact of their actions. Recent methods have applied reinforcement learning (RL) to assess the value of actions from a learned action value or Q-function. A fundamental challenge for estimating action values is that explicit reward signals (goals) are very sparse in many team sports, such as ice hockey and soccer. This paper combines Q-function learning with inverse reinforcement learning (IRL) to provide a novel player ranking method. We treat professional play as expert demonstrations for learning an implicit reward function. Our method alternates single-agent IRL to learn a reward function for multiple agents; we provide a theoretical justification for this procedure. Knowledge transfer is used to combine learned rewards and observed rewards from goals. Empirical evaluation, based on 4.5M play-by-play events in the National Hockey League (NHL), indicates that player ranking using the learned rewards achieves high correlations with standard success measures and temporal consistency throughout a season.
Keywords:
Machine Learning Applications: Applications of Reinforcement Learning
Machine Learning: Reinforcement Learning
Agent-based and Multi-agent Systems: Other