Proceedings

Large Language Models for Human-AI Co-Creation of Robotic Dance Performances

Download BibTeX Cite

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. AI, Arts & Creativity. Pages 7627-7635.

https://doi.org/10.24963/ijcai.2024/844

Abstract

This paper focuses on the potential of Generative Artificial Intelligence (AI), particularly Large Language Models (LLMs), in the still unexplored domain of robotic dance creation. In particular, we assess whether a LLM (GPT-3.5 turbo) can create robotic dance choreographies, and we investigate if the feedback provided by human creators can improve the quality of the output. To this end, we design three prompt engineering techniques for robotic dance creation. In the prompts, we gradually introduce human knowledge through examples and feedback in natural language in order to explore the dynamics of human-AI co-creation. The experimental analysis shows that the capabilities of the LLM can be improved through human collaboration, by producing choreographies with a major artistic impact on the evaluation audience. The findings offer valuable insights into the interplay between human creativity and AI generative models, paving the way for enhanced collaborative frameworks in creative domains.

  1. [Aljanabi et al., 2023] Mohammad Aljanabi, Mo- hanad Ghazi Yaseen, Ahmed Hussein Ali, and Mostafa Abdulghafoor Mohammed. Prompt engineering: Guiding the way to effective large language models. Iraqi Journal for Computer Science and Mathematics, 4(4):151-155, 2023.
  2. [Aucouturier et al., 2008] Jean-Julien Aucouturier, Kat- sushi Ikeuchi, Hirohisa Hirukawa, Shin'ichiro Nakaoka, Takaaki Shiratori, Shunsuke Kudoh, Fumio Kanehiro, Tetsuya Ogata, Hideki Kozima, Hiroshi G Okuno, et al. Cheek to chip: Dancing robots and ai's future. IEEE Intelligent Systems, 23(2):74-84, 2008.
  3. [Brown et al., 2006] Steven Brown, Michael J Martinez, and Lawrence M Parsons. The neural basis of human dance. Cerebral cortex, 16(8):1157-1167, 2006.
  4. [Chakrabarty et al., 2023a] Tuhin Chakrabarty, Philippe La- ban, Divyansh Agarwal, Smaranda Muresan, and Chien- Sheng Wu. Art or artifice? large language models and the false promise of creativity, 2023.
  5. [Chakrabarty et al., 2023b] Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and Smaranda Muresan. Creativity support in the age of large language models: An empirical study involving emerging writers. arXiv preprint arXiv:2309.12570, 2023.
  6. [Charness et al., 2012] Gary Charness, Uri Gneezy, and Michael A Kuhn. Experimental methods: Between- subject and within-subject design. Journal of economic behavior & organization, 81(1):1-8, 2012.
  7. [Chen et al., 2023] Banghao Chen, Zhaofeng Zhang, Nico- las Langrené, and Shengxin Zhu. Unleashing the poten- tial of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735, 2023.
  8. [De Filippo et al., 2019] Allegra De Filippo, Michele Lom- bardi, Michela Milano, et al. How to tame your anticipa- tory algorithm. In Proceedings of the Twenty-Eighth Inter- national Joint Conference on Artificial Intelligence, pages 1071-1077, 2019. IJCAI
  9. [De Filippo et al., 2022a] Allegra De Filippo, Andrea Borghesi, Andrea Boscarino, and Michela Milano. Hada: An automated tool for hardware dimensioning of ai applications. Knowledge-Based Systems, 251:109199, 2022.
  10. [De Filippo et al., 2022b] Allegra De Filippo, Paola Mello, and Michela Milano. Do you like dancing robots? ai can tell you why. In PAIS 2022, pages 45-58. IOS Press, 2022.
  11. [De Filippo et al., 2023] Allegra De Filippo, Luca Giu- liani, Eleonora Mancini, Andrea Borghesi, Paola Mello, Michela Milano, et al. Towards symbiotic creativity: A methodological approach to compare human and ai robotic dance creations. In Thirty-Second International Joint Con- ference on Artificial Intelligence, pages 5806-5814, 2023. IJCAI
  12. [Deng and Lin, 2022] Jianyang Deng and Yijia Lin. The benefits and challenges of chatgpt: An overview. Frontiers in Computing and Intelligent Systems, 2(2):81-83, 2022.
  13. [Ekin, 2023] Sabit Ekin. Prompt engineering for chatgpt: A quick guide to techniques, tips, and best practices. Au- thorea Preprints, 2023.
  14. [El-Zanfaly et al., 2023] Dina El-Zanfaly, Yiwei Huang, and Yanwen Dong. Sand-in-the-loop: Investigating embodied co-creation for shared understandings of generative ai. In Companion Publication of the 2023 ACM Designing Inter- active Systems Conference, pages 256-260, 2023.
  15. [Epstein et al., 2023] Ziv Epstein, Aaron Hertzmann, Inves- tigators of Human Creativity, Memo Akten, Hany Farid, Jessica Fjeld, Morgan R Frank, Matthew Groh, Laura Her- man, Neil Leach, et al. Art and the science of generative ai. Science, 380(6650):1110-1111, 2023.
  16. [Franceschelli and Musolesi, 2023] Giorgio Franceschelli and Mirco Musolesi. On the creativity of large language models. arXiv preprint arXiv:2304.00008, 2023.
  17. [Fui-Hoon Nah et al., 2023] Fiona Fui-Hoon Nah, Ruilin Zheng, Jingyuan Cai, Keng Siau, and Langtao Chen. Gen- erative ai and chatgpt: Applications, challenges, and ai- human collaboration, 2023.
  18. [Han et al., 2021] Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, et al. Pre-trained models: Past, present and future. AI Open, 2:225-250, 2021.
  19. [Hong and Curran, 2019] Joo-Wha Hong and Nathaniel Ming Curran. Artificial intelligence, artists, and art: attitudes toward artwork produced by humans vs. artificial intelligence. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 15(2s):1-16, 2019.
  20. [Huang et al., 2020] Cheng-Zhi Anna Huang, Hendrik Vin- cent Koops, Ed Newton-Rex, Monica Dinculescu, and Carrie J Cai. Ai song contest: Human-ai co-creation in songwriting. arXiv preprint arXiv:2010.05388, 2020.
  21. [Joshi and Chakrabarty, 2021] Manish Joshi and Sangeeta Chakrabarty. An extensive review of computational dance automation techniques and applications. Proceedings of the Royal Society A, 477(2251):20210071, 2021.
  22. [Lee et al., 2023] Philseok Lee, Shea Fyffe, Mina Son, Zi- hao Jia, and Ziyu Yao. A paradigm shift from "human writing" to "machine generation" in personality test de- velopment: An application of state-of-the-art natural lan- guage processing. Journal of Business and Psychology, 38(1):163-190, 2023.
  23. [Liu et al., 2020] Yuechang Liu, Dongbo Xie, Hankz Han- kui Zhuo, and Liqian Lai. Plan2dance: Planning based choreographing from music. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13624-13625, 2020.
  24. [McKight and Najab, 2010] Patrick E McKight and Julius Najab. Kruskal-wallis test. The corsini encyclopedia of psychology, pages 1-1, 2010.
  25. [McKnight and Najab, 2010] Patrick E McKnight and Julius Najab. Mann-whitney u test. The Corsini encyclopedia of psychology, pages 1-1, 2010.
  26. [Mirowski et al., 2023] Piotr Mirowski, Kory W Mathew- son, Jaylen Pittman, and Richard Evans. Co-writing screenplays and theatre scripts with language models: Evaluation by industry professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Comput- ing Systems, pages 1-34, 2023.
  27. [Naveed et al., 2023] Humza Naveed, Shi Khan, Asad Ullah afnd Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian. A comprehen- sive overview of large language models. arXiv preprint arXiv:2307.06435, 2023.
  28. [Oliveira et al., 2012] João Lobato Oliveira, Luis Paulo Reis, and Brígida Mónica Faria. Empiric evaluation of robot dancing framework based. TELKOMNIKA, 10(8):1701-8, 2012.
  29. [OpenAI, 2023] OpenAI. Mastering temperature and top_p in chatgpt api. Technical report, 2023. Accessed January 2024.
  30. [OpenAI, 2024] OpenAI. Six strategies for getting better re- sults. Technical report, 2024. Accessed January 2024.
  31. [Ouyang et al., 2022] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instruc- tions with human feedback. Advances in Neural Informa- tion Processing Systems, 35:27730-27744, 2022.
  32. [Pignatiello et al., 2020] Grant A Pignatiello, Richard J Mar- tin, and Ronald L Hickman Jr. Decision fatigue: A con- ceptual analysis. Journal of health psychology, 25(1):123- 135, 2020.
  33. [Ramos et al., 2015] Oscar E Ramos, Nicolas Mansard, Olivier Stasse, Christophe Benazeth, Sovannara Hak, and Layale Saab. Dancing humanoid robots: Systematic use of osid to compute dynamically consistent movements fol- lowing a motion capture pattern. IEEE Robotics & Au- tomation Magazine, 22(4):16-26, 2015.
  34. [Sætra, 2023] Henrik Skaug Sætra. Generative ai: Here to stay, but for good? Technology in Society, 75:102372, 2023.
  35. [Sagasti, 2019] Francisco Sagasti. Information technology and the arts: the evolution of computer choreography dur- ing the last half century. Dance Chronicle, 42(1):1-52, 2019.
  36. [Saravia, 2022] Elvis Saravia. Prompt Engineering Guide. https://github.com/dair-ai/Prompt-Engineering-Guide, 12 2022. Link
  37. [Scaringella et al., 2006] Nicolas Scaringella, Giorgio Zoia, and Daniel Mlynek. Automatic genre classification of mu- sic content: a survey. IEEE Signal Processing Magazine, 23(2):133-141, 2006.
  38. [Shinozaki et al., 2007] Kuniya Shinozaki, Akitsugu Iwatani, and Ryohei Nakatsu. Concept and construction of a dance robot system. In Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts, pages 161-164, 2007.
  39. [Shinozaki et al., 2008] Kuniya Shinozaki, Akitsugu Iwatani, and Ryohei Nakatsu. Construction and evalua- tion of a robot dance system. In Entertainment Computing Symposium, pages 83-94. Springer, 2008.
  40. [Stiennon et al., 2020] Nisan Stiennon, Long Ouyang, Jef- frey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3021, 2020.
  41. [Sturm, 2012] Bob L Sturm. Two systems for automatic mu- sic genre recognition: What are they really recognizing? In Proceedings of the second international ACM work- shop on Music information retrieval with user-centered and multimodal strategies, pages 69-74, 2012.
  42. [Wang et al., 2023a] Chi Wang, Susan Xueqing Liu, and Ahmed H Awadallah. Cost-effective hyperparameter op- timization for large language model generation inference. arXiv preprint arXiv:2303.04673, 2023.
  43. [Wang et al., 2023b] Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. Rethinking the evaluation for conversational recommendation in the era of large language models. arXiv preprint arXiv:2305.13112, 2023.
  44. [Wang, 2022] Zhigang Wang. Music choreography algo- rithm based on feature matching and fragment segmenta- tion. Computational Intelligence and Neuroscience, 2022, 2022.
  45. [Wei et al., 2023] Chengwei Wei, Yun-Cheng Wang, Bin Wang, and C-C Jay Kuo. An overview on language mod- els: Recent developments and outlook. arXiv preprint arXiv:2303.05759, 2023.
  46. [White et al., 2023] Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382, 2023.
  47. [Wu and Hu, 2023] Yangjian Wu and Gang Hu. Explor- ing prompt engineering with gpt language models for document-level machine translation: Insights and find- ings. In Proceedings of the Eighth Conference on Machine Translation, pages 166-169, 2023.
  48. [Zamfirescu-Pereira et al., 2023] JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. Why johnny can't prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-21, 2023.
  49. [Zhou et al., 2023] Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards