Social Motivation for Modelling Other Agents under Partial Observability in Decentralised Training

Social Motivation for Modelling Other Agents under Partial Observability in Decentralised Training

Dung Nguyen, Hung Le, Kien Do, Svetha Venkatesh, Truyen Tran

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence
Main Track. Pages 4082-4090. https://doi.org/10.24963/ijcai.2023/454

Understanding other agents is a key challenge in constructing artificial social agents. Current works focus on centralised training, wherein agents are allowed to know all the information about others and the environmental state during training. In contrast, this work studies decentralised training, wherein agents must learn the model of other agents in order to cooperate with them under partially-observable conditions, even during training, i.e. learning agents are myopic. The intrinsic motivation for artificial agents is modelled on the concept of human social motivation that entices humans to meet and understand each other, especially when experiencing a utility loss. Our intrinsic motivation encourages agents to stay near each other to obtain better observations and construct a model of others. They do so when their model of other agents is poor, or the overall task performance is bad during the learning phase. This simple but effective method facilitates the processes of modelling others, resulting in an improvement of the performance in cooperative tasks significantly. Our experiments demonstrate that the socially-motivated agent can model others better and promote cooperation across different tasks.
Keywords:
Machine Learning: ML: Deep reinforcement learning
Agent-based and Multi-agent Systems: MAS: Other