Crowd Learning: Improving Online Decision Making Using Crowdsourced Data

Crowd Learning: Improving Online Decision Making Using Crowdsourced Data

Yang Liu, Mingyan Liu

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 317-323. https://doi.org/10.24963/ijcai.2017/45

We analyze an online learning problem that arises in crowdsourcing systems for users facing crowdsourced data: a user at each discrete time step t can choose K out of a total of N options (bandits), and receives randomly generated rewards dependent on user-specific and option-specific statistics unknown to the user. Each user aims to maximize her expected total rewards over a certain time horizon through a sequence of exploration and exploitation steps. Different from the typical regret/bandit learning setting, in this case a user may also exploit crowdsourced information to augment her learning process, i.e., other users' choices or rewards from using these options. We consider two scenarios, one in which only their choices are shared, and the other in which users share full information including their choices and subsequent rewards. In both cases we derive bounds on the weak regret, the difference between the user's expected total reward and the reward from a user-specific best single-action policy; and show how they improve over their individual-learning counterpart. We also evaluate the performance of our algorithms using simulated data as well as the real-world movie ratings dataset MovieLens.
Keywords:
Agent-based and Multi-agent Systems: Multi-agent Learning
Machine Learning: Online Learning
Multidisciplinary Topics and Applications: Multidisciplinary Topics and Applications