Responsible Autonomy

Dignum, Virginia

doi:10.24963/ijcai.2017/655

Responsible Autonomy

Download BibTeX Cite

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. AI and autonomy track. Pages 4698-4704.

https://doi.org/10.24963/ijcai.2017/655

Keywords

Social: Challenges
Philosophical: Concepts
Technical: Models

Abstract

As intelligent systems are increasingly making decisions that directly affect society, perhaps the most important upcoming research direction in AI is to rethink the ethical implications of their actions. Means are needed to integrate moral, societal and legal values with technological developments in AI, both during the design process as well as part of the deliberation algorithms employed by these systems. In this paper, we describe leading ethics theories and propose alternative ways to ensure ethical behavior by artificial systems.

Given that ethics are dependent on the socio-cultural context and are often only implicit in deliberation processes, methodologies are needed to elicit the values held by designers and stakeholders, and to make these explicit leading to better understanding and trust on artificial autonomous systems.

[Aldewereld et al., 2010] Huib Aldewereld, S. Álvarez- Napagao, F. Dignum, and J. Vázquez-Salceda. Making norms concrete. In AAMAS'10, pages 807-814, 2010.
[Allen et al., 2005] C. Allen, I. Smit, and W. Wallach. Ar- tificial morality: Top-down, bottom-up, and hybrid ap- proaches. Ethics and Information Technology, 7(3):149- 155, 2005.
[Binmore, 2005] Ken Binmore. Natural justice. Oxford Uni- versity Press, 2005.
[Björklund, 2003] Fredrik Björklund. Differences in the jus- tification of choices in moral dilemmas: Effects of gen- der, time pressure and dilemma seriousness. Scandinavian Journal of Psychology, 44(5):459-466, 2003.
[Boella et al., 2006] Guido Boella, L. Van Der Torre, and H. Verhagen. Introduction to normative multiagent sys- tems. Computational & Mathematical Organization The- ory, 12(2-3):71-79, 2006.
[Bonnefon et al., 2016] Jean-Franc ¸ois Bonnefon, Azim Shariff, and Iyad Rahwan. The social dilemma of autonomous vehicles. Science, 352(6293):1573-1576, 2016.
[Bryson and Kime, 2011] Joanna Bryson and Philip Kime. Just an artifact: why machines are perceived as moral agents. In IJCAI'11, pages 1641-1646, 2011. IJCAI
[Conci et al., 2012] Mario Conci, Virginia Dignum, Dirk Heylen, and Jonas Beskow. Roadmap for casa (computer as social actors) technologies. final report EIT ICT labs activity RIHA 12124, 2012.
[Conte et al., 1998] Rosaria Conte, C. Castelfranchi, and F. Dignum. Autonomous norm acceptance. In Interna- tional Workshop on Agent Theories, Architectures, and Languages, pages 99-112. Springer, 1998.
[Cranefield et al., 2017] Stephen Cranefield, M. Winikoff, V. Dignum, and F. Dignum. No pizza for you: Value-based plan selection in bdi agents, 2017. IJCAI
[Floridi and Sanders, 2004] Luciano Floridi and J. Sanders. On the morality of artificial agents. Minds and machines, 14(3):349-379, 2004.
[Foot, 1967] Philippa Foot. The problem of abortion and the doctrine of double effect. 1967.
[Friedman et al., 2006] Batya Friedman, Peter Kahn, and Alan Borning. Value sensitive design and information systems. Advances in Management Information Systems, 6:348 -372, 2006.
[Gigerenzer, 2010] Gerd Gigerenzer. Moral satisficing: Re- thinking moral behavior as bounded rationality. Topics in cognitive science, 2(3):528-554, 2010.
[Graham et al., 2011] Jesse Graham, Brian Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter Ditto. Map- ping the moral domain. Journal of personality and social psychology, 101(2):366, 2011.
[Hofstede and Hofstede, 2001] Gert Hofstede and G. Hofst- ede. Culture's consequences: Comparing values, behav- iors, institutions and organizations. Sage, 2001.
[Lay et al., 2011] Ken Lay, O. C. Ferrell, and L. Ferrell. The responsibility and accountability of ceos: The last inter- view with ken lay. Journal of Business Ethics, 100(2):209- 219, 2011.
[Li et al., 2014] Wenchao Li, Dorsa Sadigh, Shankar Sastry, and Sanjit Seshia. Synthesis for Human-in-the-Loop Con- trol Systems, pages 470-484. Springer, 2014.
[Malle et al., 2015] Bertram Malle, Matthias Scheutz, Thomas Arnold, John Voiklis, and Corey Cusimano. Sac- rifice one for the good of many?: People apply different moral norms to human and robot agents. In Proc. 10th ACM/IEEE Int. Conf. on Human-Robot Interaction, pages 117-124. ACM, 2015.
[Misener and Shladover, 2006] James Misener and S. Shladover. Path investigations in vehicle-roadside cooperation and safety: A foundation for safety and vehicle-infrastructure integration research. In Intelligent Transportation Systems Conference, 2006, pages 9-16. IEEE, 2006.
[Modgil and Prakken, 2013] Sanjay Modgil and Henry Prakken. A general account of argumentation with preferences. Artificial Intelligence, 195:361-397, 2013.
[Russell and Norvig, 2009] Stuart Russell and P Norvig. Ar- tificial Intelligence: A Modern Approach. Pearson Educa- tion, 3rd edition, 2009.
[Schwartz, 2006a] Shalom Schwartz. An Overview Basic Human Values: Theory, Methods, and Applications Intro- duction to the Values Theory. Jerusalem Hebrew Univer- sity, 2006.
[Schwartz, 2006b] Shalom Schwartz. A theory of cultural value orientations: Explication and applications. Compar- ative sociology, 5(2):137-182, 2006.
[Searle, 2010] John Searle. Making the social world: The structure of human civilization. Oxford UP, 2010.
[Sen, 2004] Amartya Sen. Elements of a theory of human rights. Philosophy & Public Affairs, 32(4):315-356, 2004.
[van de Poel, 2013] Ibo van de Poel. Translating Values into Design Requirements, pages 253-266. Springer Nether- lands, Dordrecht, 2013.
[van den Hoven, 2005] Jeroen van den Hoven. Design for values and values for design. Information Age +, Journal of the Australian Computer Society, 7(2):4-7, 2005.
[Verdiesen et al., 2016] Ilse Verdiesen, Martijn Cligge, Jan Timmermans, Lennart Segers, Virginia Dignum, and Jeroen van den Hoven. MOOD: Massive open online de- liberation platform a practical application. In 1st Workshop on Ethics in the Design of Intelligent Agents, pages 6-11, 2016.
[Wallach and Allen, 2008] Wendall Wallach and Collin Allen. Moral machines: Teaching robots right from wrong. Oxford University Press, 2008.

References were extracted automatically from the PDF and may contain errors.

Full text extracted automatically from the PDF; figures, tables and equations may differ. The PDF is the version of record.

1 Introduction

It is no news that Artificial Intelligence (AI) is increasingly entering the public domain in fields such as transportation, service robots, health-care, education, public safety and security, employment and workplace, and entertainment. Developments in autonomy and learning technologies are rapidly enabling AI systems to decide and act without direct human control. As these advances continue at high speed, there is a growing awareness that a responsible approach to AI is needed to ensure the safe, beneficial and fair use AI technologies, to consider the implications of morally relevant decision making by machines, and the ethical and legal consequences and status of AI.

Design methods and tools to elicit and represent human values, translate these values into technical requirements, and deal with moral overload when numerous values are to be incorporated, are needed to demonstrate that design solutions realize the values wished for.

As an example, recently much attention has been given to the ethical dilemmas self-driving cars face when needing to deal with potentially life-threatening decisions [Bonnefon et al., 2016]. This has been described as an application of the well-known trolley problem [Foot, 1967], an hypothetical scenario, long used in philosophy and ethics discussions, where a runway trolley is speeding down a track to which five people are tied up and unable to move. An observer has control over a lever that can switch tracks before the trolley hits the five people, but also on this alternative track there is one person tied to the tracks. The moral dilemma concerns the decision the observer should make: do nothing and allow the trolley to kill the five, or pull the lever and kill one? This is obviously a hypothetical scenario without any direct practical application. However, it can be seen as an abstraction for many dilemmas involving AI systems now and in the future. Besides the relation to self-driving cars, similar dilemmas will need to be solved by intelligent medicine dispenses faced with the need to choose between two patients when it does not have enough of a needed medicine, by search and rescue robots faced with the need to prioritize victims, or, as we have recently shown, by health-care robots needing to choose between user's desires and optimal care [Cranefield et al., 2017]. In this paper, we use the trolley scenario as illustration of this wide application of moral dilemmas deliberation.

From an ethical perspective, there are no optimal solutions to these dilemmas. In fact, different ethical theories will lead to distinct solutions given that values hold by individuals, groups and societies put different preferences on the action to choose. Understanding of these differences is essential to the design of AI systems able to deal with such dilemmas. Moreover, it is not so that the responsibility lays solely with the individual (human or machine) that operates the lever: societal, legal and physical infrastructures are also means to determine the decision.

If we are to build AI systems that can deal with this type of ethical dilemma, and ensure that AI is developed responsibly incorporating social and ethical values, societal concerns about the ethics of AI must be reflected in design. AI systems should therefore be ground on principles of Accountability, Responsibility and Transparency (ART), extending and characterizing the classic principles of Autonomy, Interactivity and Adaptability described in [Floridi and Sanders, 2004; Russell and Norvig, 2009].

Firstly, Accountability for the decision must be derivable from the algorithms and data used by the system in order to make the decision. This includes the need for representation of the moral values and societal norms holding in the context of operation, which the agent uses for deliberation. Secondly,

Virginia Dignum. 2017. Responsible Autonomy. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. IJCAI, 4698–4704. https://doi.org/10.24963/ijcai.2017/655

@inproceedings{ijcai2017p655,
  title     = {Responsible Autonomy},
  author    = {Dignum, Virginia},
  booktitle = {Proceedings of the Twenty-Sixth International Joint Conference on
               Artificial Intelligence, {IJCAI-17}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Carles Sierra},
  pages     = {4698--4704},
  year      = {2017},
  month     = {8},
  note      = {AI and autonomy track},
  doi       = {10.24963/ijcai.2017/655},
  url       = {https://doi.org/10.24963/ijcai.2017/655},
}

TY  - CPAPER
TI  - Responsible Autonomy
AU  - Dignum, Virginia
T2  - Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2017
SP  - 4698
EP  - 4704
DO  - 10.24963/ijcai.2017/655
UR  - https://doi.org/10.24963/ijcai.2017/655
ER  -

[1] [Aldewereld et al., 2010] Huib Aldewereld, S. Álvarez- Napagao, F. Dignum, and J. Vázquez-Salceda. Making norms concrete. In AAMAS'10, pages 807-814, 2010.

[2] [Allen et al., 2005] C. Allen, I. Smit, and W. Wallach. Ar- tificial morality: Top-down, bottom-up, and hybrid ap- proaches. Ethics and Information Technology, 7(3):149- 155, 2005.

[3] [Binmore, 2005] Ken Binmore. Natural justice. Oxford Uni- versity Press, 2005.

[4] [Björklund, 2003] Fredrik Björklund. Differences in the jus- tification of choices in moral dilemmas: Effects of gen- der, time pressure and dilemma seriousness. Scandinavian Journal of Psychology, 44(5):459-466, 2003.

[5] [Boella et al., 2006] Guido Boella, L. Van Der Torre, and H. Verhagen. Introduction to normative multiagent sys- tems. Computational & Mathematical Organization The- ory, 12(2-3):71-79, 2006.

[6] [Bonnefon et al., 2016] Jean-Franc ¸ois Bonnefon, Azim Shariff, and Iyad Rahwan. The social dilemma of autonomous vehicles. Science, 352(6293):1573-1576, 2016.

[7] [Bryson and Kime, 2011] Joanna Bryson and Philip Kime. Just an artifact: why machines are perceived as moral agents. In IJCAI'11, pages 1641-1646, 2011. IJCAI

[8] [Conci et al., 2012] Mario Conci, Virginia Dignum, Dirk Heylen, and Jonas Beskow. Roadmap for casa (computer as social actors) technologies. final report EIT ICT labs activity RIHA 12124, 2012.

[9] [Conte et al., 1998] Rosaria Conte, C. Castelfranchi, and F. Dignum. Autonomous norm acceptance. In Interna- tional Workshop on Agent Theories, Architectures, and Languages, pages 99-112. Springer, 1998.

[10] [Cranefield et al., 2017] Stephen Cranefield, M. Winikoff, V. Dignum, and F. Dignum. No pizza for you: Value-based plan selection in bdi agents, 2017. IJCAI

[11] [Floridi and Sanders, 2004] Luciano Floridi and J. Sanders. On the morality of artificial agents. Minds and machines, 14(3):349-379, 2004.

[12] [Foot, 1967] Philippa Foot. The problem of abortion and the doctrine of double effect. 1967.

[13] [Friedman et al., 2006] Batya Friedman, Peter Kahn, and Alan Borning. Value sensitive design and information systems. Advances in Management Information Systems, 6:348 -372, 2006.

[14] [Gigerenzer, 2010] Gerd Gigerenzer. Moral satisficing: Re- thinking moral behavior as bounded rationality. Topics in cognitive science, 2(3):528-554, 2010.

[15] [Graham et al., 2011] Jesse Graham, Brian Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter Ditto. Map- ping the moral domain. Journal of personality and social psychology, 101(2):366, 2011.

[16] [Hofstede and Hofstede, 2001] Gert Hofstede and G. Hofst- ede. Culture's consequences: Comparing values, behav- iors, institutions and organizations. Sage, 2001.

[17] [Lay et al., 2011] Ken Lay, O. C. Ferrell, and L. Ferrell. The responsibility and accountability of ceos: The last inter- view with ken lay. Journal of Business Ethics, 100(2):209- 219, 2011.

[18] [Li et al., 2014] Wenchao Li, Dorsa Sadigh, Shankar Sastry, and Sanjit Seshia. Synthesis for Human-in-the-Loop Con- trol Systems, pages 470-484. Springer, 2014.

[19] [Malle et al., 2015] Bertram Malle, Matthias Scheutz, Thomas Arnold, John Voiklis, and Corey Cusimano. Sac- rifice one for the good of many?: People apply different moral norms to human and robot agents. In Proc. 10th ACM/IEEE Int. Conf. on Human-Robot Interaction, pages 117-124. ACM, 2015.

[20] [Misener and Shladover, 2006] James Misener and S. Shladover. Path investigations in vehicle-roadside cooperation and safety: A foundation for safety and vehicle-infrastructure integration research. In Intelligent Transportation Systems Conference, 2006, pages 9-16. IEEE, 2006.

[21] [Modgil and Prakken, 2013] Sanjay Modgil and Henry Prakken. A general account of argumentation with preferences. Artificial Intelligence, 195:361-397, 2013.

[22] [Russell and Norvig, 2009] Stuart Russell and P Norvig. Ar- tificial Intelligence: A Modern Approach. Pearson Educa- tion, 3rd edition, 2009.

[23] [Schwartz, 2006a] Shalom Schwartz. An Overview Basic Human Values: Theory, Methods, and Applications Intro- duction to the Values Theory. Jerusalem Hebrew Univer- sity, 2006.

[24] [Schwartz, 2006b] Shalom Schwartz. A theory of cultural value orientations: Explication and applications. Compar- ative sociology, 5(2):137-182, 2006.

[25] [Searle, 2010] John Searle. Making the social world: The structure of human civilization. Oxford UP, 2010.

[26] [Sen, 2004] Amartya Sen. Elements of a theory of human rights. Philosophy & Public Affairs, 32(4):315-356, 2004.

[27] [van de Poel, 2013] Ibo van de Poel. Translating Values into Design Requirements, pages 253-266. Springer Nether- lands, Dordrecht, 2013.

[28] [van den Hoven, 2005] Jeroen van den Hoven. Design for values and values for design. Information Age +, Journal of the Australian Computer Society, 7(2):4-7, 2005.

[29] [Verdiesen et al., 2016] Ilse Verdiesen, Martijn Cligge, Jan Timmermans, Lennart Segers, Virginia Dignum, and Jeroen van den Hoven. MOOD: Massive open online de- liberation platform a practical application. In 1st Workshop on Ethics in the Design of Intelligent Agents, pages 6-11, 2016.

[30] [Wallach and Allen, 2008] Wendall Wallach and Collin Allen. Moral machines: Teaching robots right from wrong. Oxford University Press, 2008.