Fair and Explainable Dynamic Engagement of Crowd Workers

1. [AI HLEG, 2019] AI HLEG. Ethics guidelines for trustwor- thy AI. Technical report, High-Level Expert Group on Ar- tificial Intelligence, the European Commission, 2019.
2. [Broekens and Brinkman, 2013] Joost Broekens and Willem-Paul Brinkman. Affectbutton: A method for reliable and valid affective self-report. International Journal of Human-Computer Studies, 71(6):641-667, 2013.
3. [Chen et al., 2016] Guo Chen, Amy K. Glasmeier, Min Zhang, and Yang Shao. Urbanization and income inequal- ity in post-reform china: A causal analysis based on time series data. PLoS One, 11(7):e0158826, 2016.
4. [Doan et al., 2011] Anhai Doan, Raghu Ramakrishnan, and Alon Y. Halevy. Crowdsourcing systems on the world- wide web. Communications of the ACM, 54(4):86-96, 2011.
5. [Fan and Toni, 2015] Xiuyi Fan and Francesca Toni. On computing explanations in argumentation. In AAAI, pages 1496-1502, 2015.
6. [Jain et al., 1998] Raj Jain, Dah-Ming Chiu, and W. Hawe. A quantitative measure of fairness and discrimination for resource allocation in shared computer systems. CoRR, cs.NI/9809099, 1998.
7. [Li et al., 2009] Boyang Li, Han Yu, Zhiqi Shen, and Chun- yan Miao. Evolutionary organizational search. In AAMAS, pages 1329-1330, 2009.
8. [Li et al., 2015] Boyang Li, Han Yu, Zhiqi Shen, Lizhen Cui, and Victor R. Lesser. An evolutionary framework for multi-agent organizations. In WI-IAT, pages 35-38, 2015.
9. [Lin et al., 2015] Han Lin, Jinghua Hou, Han Yu, Zhiqi Shen, and Chunyan Miao. An agent-based game platform for exercising people's prospective memory. In WI-IAT, pages 235-236, 2015.
10. [Neely, 2010] Michael J. Neely. Stochastic Network Opti- mization with Application to Communication and Queue- ing Systems. Morgan and Claypool Publishers, 2010.
11. [Pan et al., 2016] Zhengxiang Pan, Han Yu, Chunyan Miao, and Cyril Leung. Efficient collaborative crowdsourcing. In AAAI, pages 4248-4249, 2016.
12. [Shen et al., 2011] Zhiqi Shen, Han Yu, Chunyan Miao, and Jianshu Weng. Trust-based web service selection in vir- tual communities. Web Intelligence and Agent Systems, 9(3):227-238, 2011.
13. [Sreedharan et al., 2017] Sarath Sreedharan, Tathagata Chakraborti, and Subbarao Kambhampati. Balancing explicability and explanation in human-aware planning. Technical report, AAAI Technical Report FS-17-01, 2017.
14. [Xie and Zhou, 2014] Yu Xie and Xiang Zhou. Income in- equality in today's China. Proceedings of the National A- cademy of Sciences USA, 111(19):6928-6933, 2014.
15. [Yang et al., 2019] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, 10(2):12:1-12:19, 2019.
16. [Yang, 2013] X. Jin Yang. China's rapid urbanization. Sci- ence, 342(6156):310, 2013.
17. [Yu et al., 2007] Han Yu, Zhiqi Shen, and Chunyan Miao. Intelligent software agent design tool using goal net methodology. In IAT, pages 43-46, 2007.
18. [Yu et al., 2011] Han Yu, Zhiqi Shen, Chunyan Miao, and Ah-Hwee Tan. A simple curious agent to help people be curious. In AAMAS, pages 1159-1160, 2011.
19. [Yu et al., 2013] Han Yu, Chunyan Miao, Bo An, Cyril Le- ung, and Victor R. Lesser. A reputation management mod- el for resource constrained trustee agents. In IJCAI, pages 418-424, 2013.
20. [Yu et al., 2015] Han Yu, Chunyan Miao, Zhiqi Shen, Cyril Leung, Yiqiang Chen, and Qiang Yang. Efficient task sub- delegation for crowdsourcing. In AAAI, pages 1305-1311, 2015.
21. [Yu et al., 2016] Han Yu, Chunyan Miao, Cyril Leung, Y- iqiang Chen, Simon Fauvel, Victor R. Lesser, and Qiang Yang. Mitigating herding in hierarchical crowdsourcing networks. Scientific Reports, 6(4):doi:10.1038/s41598- 016-0011-6, 2016. DOI
22. [Yu et al., 2017a] Han Yu, Chunyan Miao, Yiqiang Chen, Simon Fauvel, Xiaoming Li, and Victor R. Less- er. Algorithmic management for improving collec- tive productivity in crowdsourcing. Scientific Reports, 7(12541):doi:10.1038/s41598-017-12757-x, 2017. DOI
23. [Yu et al., 2017b] Han Yu, Zhiqi Shen, Simon Fauvel, and Lizhen Cui. Efficient scheduling in crowdsourcing based on workers' mood. In ICA, pages 121-126, 2017.
24. [Yu et al., 2018] Han Yu, Zhiqi Shen, Chunyan Miao, Cyril Leung, Victor R. Lesser, and Qiang Yang. Building ethic- s into artificial intelligence. In IJCAI, pages 5527-5533, 2018. IJCAI
25. [Yu et al., 2019a] Han Yu, Chunyan Miao, Yongqing Zheng, Lizhen Cui, Simon Fauvel, and Cyril Leung. Ethically aligned opportunistic scheduling for productive laziness. In AIES, 2019.
26. [Yu et al., 2019b] Han Yu, ZhiqiShen, Lizhen Cui, Yongqing Zheng, and Victor R. Lesser. Ethically aligned sacrifice coordination to enhance social welfare. In AAMAS, pages 2300-2302, 2019.
27. [Zheng et al., 2018] Yongqing Zheng, Han Yu, Lizhen Cui, Chunyan Miao, Cyril Leung, and Qiang Yang. SmartHS: An AI platform for improving government service provi- sion. In IAAI, pages 7704-7711, 2018.

References were extracted automatically from the PDF and may contain errors.

Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.

1 Introduction

As a result of rural-urban migration in China [Yang, 2013], there exist a large number of people working on an ad-hoc basis in large Chinese cities. This group of people are considered to be vulnerable [Chen et al., 2016]. Many of them look for ad-hoc work through untrustworthy agents, thus exposing them to high risk of exploitation and swindle [Xie and Zhou, 2014]. On the other hand, many brick-and-mortar businesses such as franchised department stores experience large fluctuations in terms of demand for manpower under different situations (e.g. peak periods of customer influx, periods of inventory restocking). Is there a solution that can address the aforementioned societal challenge while empowering this market sector to satisfy manpower needs more efficiently?

In this paper, we outline AlgoCrowd - an artificial intelligence (AI)-empowered algorithmic crowdsourcing [Doan et al., 2011] platform, which aims to address the aforementioned challenge. It is designed to provide optimized task-worker matching recommendations to workers' personal work management mobile apps which are starting to become popular in China (e.g. the Zhiyouren app http://www.zyrwork.com/) to support dynamic engagement of the large pool of ad-hoc workers. The AI Engine of this platform is a data-driven real-time multi-agent organization [Yu et al., 2007; 2011; Li et al., 2009; 2015; Lin et al., 2015] approach extended from [Pan et al., 2016;

Yu et al., 2013; 2015; 2016; 2017b; 2017a; Zheng et al., 2018]. In order to comply with emerging best practices for building ethical AI [Yu et al., 2018; 2019a; 2019b; AI HLEG, 2019], the algorithm focuses on fair treatment of workers and explainability for human oversight.

2 The AlgoCrowd Platform

2.1 SystemArchitecture

Figure 1 illustrates the AlgoCrowd system architecture. It communicates with a personal work management mobile app through database profiles for tasks and workers. Key information such as task rewards, completion criteria, task types, worker capability levels for different types of tasks, productivity, availability, and sensitivity to price changes can be estimated based on task proposers' input and track records about the workers in the system. Such information is supplied to the AI engine which contains the proposed task-worker matching optimization approach. The resulting task allocation plan produced by the AI engine at any given point in time is fed back to the engaged workers through their personal work management mobile apps. The outcomes of the tasks can be monitored through sensors for well-defined tasks or manual assessment by for less well-defined tasks.

2.2 The AI Engine

In order to treat workers fairly, we aim to ensure that workers of similar capability and productivity earn equitable incomes in the long run. Such an objective can be translated into minimizing the regret by any worker when his income is compared to similar peers. The AI engine adopts queueing system concepts to model the dynamics of a worker's regret Yi(t) as Yi(t + 1) = max[0, Yi(t) + ¯ v(t - 1) - vi(t)], where vi(t) is the income from a task allocated to worker i in the current round t, and ¯ v(t - 1) is the average income received by i's similar peers during (t - 1).

Following the Lyapunov optimization technique [Neely, 2010], we derive the distribution of regret among workers at t as the l2-norm L(t) = 1 2 ∑Ni=1 Y 2 i (t). Thus, the fluctuation in workers' incomes over time can be expressed as △ = 1 T ∑T -1 t=0 [L(t +1) -L(t)]. By minimizing △, the algorithm provides two dimensions of fairness to workers: 1) workers with similar capability and productivity receive equitable incomes from tasks in any given round of task allocation, and 2) the fluctuation in a worker's regret over time remains small. At the same time, the collective utility of the business engaging the workers, which can be modelled as vi(t)qi,j(t)pi(t) where qi,j(t) and pi(t) are respectively worker i's capability level for tasks of type j and his productivity, shall be maximized. Following techniques developed in [Yu et al., 2016; 2017a], we can derive the joint objective function as maximizing 1 T ∑T -1 t=0 ∑Ni=1 vi(t)[ρqi,j(t)pi(t)+Yi(t)], subject to ∑Mj=1 1j→i ∈ (0, 1) and qi,j(t) ⩾ θj(t), ∀i, j, t. Here, ρ > 0 is a control variable for a manager to signal the AI engine how to trade off between generating higher revenue and fair treatment of workers; and θj(t) is the minimum capability level required for a worker to be eligible for a type of tasks.

The optimization can be solved efficiently through the index ranking approached developed in [Yu et al., 2016; 2017a]. Workers are ranked in descending order of their [ρqi,j(t)pi(t) + Yi(t)] indices, while tasks are ranked in descending order of their "reward-times-waiting time" values. Then, high-ranking tasks are assigned to high-ranking workers until no more tasks or no more suitable workers can be found. The algorithm has O(N logN) time complexity if mergesort is adopted, which supports large-scale operations.

The result from this solution algorithm is a tensor of fine granularity mapping workers' capability, productivity, regret, system managers' preference, and task rewards and waiting time into a task allocation plan. Explanations can be generated through argumentation techniques [Fan and Toni, 2015] with emotional connotations generated based on the AffectButton [Broekens and Brinkman, 2013] shown to the user. Since situations may change with time, the explanations also offer suggestions on better alternatives if the user can wait for more suitable workers to become available in order to balance explicability with explanations [Sreedharan et al., 2017].

2.3 The Prototype System

Figure 2 shows the demonstration system of AlgoCrowd. The system offers two modes of operations. Firstly, a manager can manually initiate the process of loading worker information and new task information from the database system, and initiating the task allocation algorithm of the AI engine. Visualization facilities are provided to illustrate the distribution of workers' capability (i.e. their reputation values [Shen et al., 2011]) as well as productivity. Once the task allocation operation is complete, results are visualized as a tree-view in the central panel of the system user interface. Users can expand the it to view key statistics related to the optimization algorithm. They can also click on any given task to trigger the explanation function to view the rationale behind that particular recommendation generated by the AI engine.

Secondly, the system provides an "auto-pilot" mode of operation in which it autonomously queries worker and task databases at preset intervals to update the latest information concerning worker and task status for a given business, and performs the task allocation operation. Summary information for each round of task allocation including Jain's Fairness Index [Jain et al., 1998] and the expected collective utility achieved are plotted automatically for users to view.

3 Discussions and FutureWork

The AlgoCrowd platform enables businesses to dynamically engage workers in need of flexible ad-hoc employment. Compared to existing systems, it offers efficient and explainable AI task allocation optimization designed to emphasize on fair treatment of workers, whiling reducing managers' workload to find suitable works for tasks. It offers a promising solution to a significant societal problem in China as a result of large-scale rural-urban migration, while empowering traditional businesses to efficiently satisfy their manpower needs.

One key aspect for the success of such a system is the accurate prediction of new workers' performance. Although each business can use their locally stored worker data, the resulting fragmented machine learning models may not be able to provide satisfactory performance. Here, we see opportunities for federated learning [Yang et al., 2019] for building a powerful aggregated model from locally stored worker data while complying with privacy protection regulations.

Acknowledgements

This research is supported, in part, by the Nanyang Assistant Professorship (NAP) and the Joint NTU-WeBank Research Centre of Eco-Intelligent Applications (THEIA), Nanyang Technological University, Singapore.

Machine-readable versions of this paper, for tools and AI agents:

• Markdown — metadata, abstract, full text and references
• JSON — metadata and structured references
• BibTeX