Proceedings

Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Survey Track. Pages 10723-10732.

https://doi.org/10.24963/ijcai.2025/1190

Abstract

Embodied multi-agent systems (EMAS) have attracted growing attention for their potential to address complex, real-world challenges in areas such as logistics and robotics. Recent advances in foundation models pave the way for generative agents capable of richer communication and adaptive problem-solving. This survey provides a systematic examination of how EMAS can benefit from these generative capabilities. We propose a taxonomy that categorizes EMAS by system architectures and embodiment modalities, emphasizing how collaboration spans both physical and virtual contexts. Central building blocks, perception, planning, communication, and feedback, are then analyzed to illustrate how generative techniques bolster system robustness and flexibility. Through concrete examples, we demonstrate the transformative effects of integrating foundation models into embodied, multi-agent frameworks. Finally, we discuss challenges and future directions, underlining the significant promise of EMAS to reshape the landscape of AI-driven collaboration.

  1. [Agashe et al., 2023] Saaket Agashe, Yue Fan, et al. Evalu- ating multi-agent coordination abilities in large language models. arXiv, 2023.
  2. [Brohan et al., 2022] Anthony Brohan, Noah Brown, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv, 2022.
  3. [Brohan et al., 2023] Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, et al. Do as i can, not as i say: Grounding language in robotic affordances. In CoRL, 2023.
  4. [Chen et al., 2023a] Huaben Chen, Wenkang Ji, et al. Multi- agent consensus seeking via large language models. arXiv, 2023.
  5. [Chen et al., 2023b] Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, et al. Towards end-to-end embodied deci- sion making with multi-modal large language model: Ex- plorations with gpt4-vision and beyond. In NeurIPS 2023 Foundation Models for Decision Making Workshop, 2023.
  6. [Chen et al., 2023c] Weize Chen, Yusheng Su, Jingwei Zuo, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In ICLR, 2023.
  7. [Chen et al., 2024a] Jiaqi Chen, Yuxian Jiang, et al. S- agents: self-organizing agents in open-ended environment. arXiv, 2024.
  8. [Chen et al., 2024b] Junting Chen, Checheng Yu, Xunzhe Zhou, Tianqi Xu, et al. Emos: Embodiment-aware het- erogeneous multi-robot operating system with llm agents. arXiv, 2024.
  9. [Chen et al., 2024c] Shuaihang Chen, Yuanxing Liu, Wei Han, Weinan Zhang, and Ting Liu. A survey on multi- generative agent system: Recent advances and new fron- tiers. arXiv, 2024.
  10. [Chen et al., 2024d] Yongchao Chen, Jacob Arkin, Charles Dawson, et al. Autotamp: Autoregressive task and motion planning with llms as translators and checkers. In ICRA, 2024.
  11. [Chen et al., 2024e] Yongchao Chen, Jacob Arkin, Yang Zhang, et al. Scalable multi-robot collaboration with large language models: Centralized or decentralized systems? In ICRA, 2024.
  12. [Cui et al., 2022] Kai Cui, Anam Tahir, Ekinci, et al. A sur- vey on large-population systems and scalable multi-agent reinforcement learning. arXiv, 2022.
  13. [Cui et al., 2023] Yuchen Cui, S. Karamcheti, et al. No, to the right: Online language corrections for robotic manipu- lation via shared autonomy. In HRI, 2023.
  14. [Dorri et al., 2018] Ali Dorri, Salil S Kanhere, and Raja Ju- rdak. Multi-agent systems: A survey. IEEE Access, 6:28573-28593, 2018.
  15. [Duan et al., 2022] Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embod- ied ai: From simulators to research tasks. IEEE Transac- tions on Emerging Topics in Computational Intelligence, 6(2):230-244, 2022.
  16. [Firoozi et al., 2023] Roya Firoozi, J. Tucker, Stephen Tian, et al. Foundation models in robotics: Applications, chal- lenges, and the future. The International Journal of Robotics Research, page 02783649241281508, 2023.
  17. [Garg et al., 2024] Kunal Garg, Jacob Arkin, Songyuan Zhang, Nicholas Roy, and Chuchu Fan. Large language models to the rescue: Deadlock resolution in multi-robot systems. arXiv, 2024.
  18. [Guo et al., 2024a] T Guo, X Chen, Y Wang, R Chang, et al. Large language model based multi-agents: A survey of progress and challenges. In IJCAI, 2024. IJCAI
  19. [Guo et al., 2024b] Xudong Guo, Kaixuan Huang, Jiale Liu, et al. Embodied llm agents learn to cooperate in organized teams. arXiv, 2024.
  20. [Gupte et al., 2024] Arjun Gupte, Ruiqi Wang, et al. Rebel: Rule-based and experience-enhanced learning with llms for initial task allocation in multi-human multi-robot teams. arXiv, 2024.
  21. [Hu et al., 2023] Yafei Hu, Quanting Xie, Vidhi Jain, et al. Toward general-purpose robots via foundation models: A survey and meta-analysis. arXiv, 2023.
  22. [Huang et al., 2023] Wenlong Huang, Chen Wang, et al. Voxposer: Composable 3d value maps for robotic manip- ulation with language models. In CoRL, 2023.
  23. [Hunt et al., 2024] William Hunt, Sarvapali D Ramchurn, and Mohammad D Soorati. A survey of language-based communication in robotics. arXiv, 2024.
  24. [Ismail et al., 2018] Z H. Ismail, N. Sariff, et al. A sur- vey and analysis of cooperative multi-agent robot systems: challenges and directions. Applications of Mobile Robots, 5:8-14, 2018.
  25. [Kannan et al., 2024] Shyam Sundar Kannan, Vishnunan- dan LN Venkatesh, and Byung-Cheol Min. Smart-llm: Smart multi-agent robot task planning using large lan- guage models. In IROS, 2024.
  26. [Kolve et al., 2017] Eric Kolve, Roozbeh Mottaghi, et al. AI2-THOR: An Interactive 3D Environment for Visual AI. ArXiv, 2017.
  27. [Landauer and Bellman, 2008] C. Landauer and K. L Bell- man. Computational embodiment: agents as constructed complex systems. In Human Cognition and Social Agent Technology, pages 301-322. 2008.
  28. [Liu et al., 2022] Xinzhu Liu, Xinghang Li, Di Guo, Sinan Tan, Huaping Liu, and Fuchun Sun. Embodied multi-agent task planning from ambiguous instruction. In RSS, 2022.
  29. [Liu et al., 2023] Jijia Liu, C Yu, , et al. Llm-powered hierar- chical language agent for real-time human-ai coordination. arXiv, 2023.
  30. [Liu et al., 2024a] Jie Liu, Pan Zhou, Yingjun Du, Ah-Hwee Tan, et al. Capo: Cooperative plan optimization for effi- cient embodied multi-agent cooperation. arXiv, 2024.
  31. [Liu et al., 2024b] Kehui Liu, Zixin Tang, Dong Wang, Zhi- gang Wang, et al. Coherent: Collaboration of hetero- geneous multi-robot system with large language models. arXiv, 2024.
  32. [Liu et al., 2024c] Songming Liu, Lingxuan Wu, et al. Rdt- 1b: a diffusion foundation model for bimanual manipula- tion. arXiv, 2024.
  33. [Liu et al., 2024d] Xinzhu Liu, Di Guo, Xinyu Zhang, and Huaping Liu. Heterogeneous embodied multi-agent col- laboration. IEEE Robotics and Automation Letters, 2024.
  34. [Liu et al., 2024e] Yang Liu, Weixing Chen, Yongjie Bai, et al. Aligning cyber space with physical world: A com- prehensive survey on embodied ai. arXiv, 2024.
  35. [Liu et al., 2024f] Zhiwei Liu, Weiran Yao, Jianguo Zhang, Liangwei Yang, et al. Agentlite: A lightweight library for building and advancing task-oriented llm agent system. arXiv, 2024.
  36. [Lu et al., 2024] Jinliang Lu, Ziliang Pang, et al. Merge, en- semble, and cooperate! a survey on collaborative strategies in the era of large language models. arXiv, 2024.
  37. [Ma et al., 2024] Yueen Ma, Zixing Song, et al. A survey on vision-language-action models for embodied ai. arXiv, 2024.
  38. [Mahajan et al., 2022] Anuj Mahajan, Mikayel Samvelyan, et al. Generalization in cooperative multi-agent systems. arXiv, 2022.
  39. [Mandi et al., 2024] Zhao Mandi, Shreeya Jain, et al. Roco: Dialectic multi-robot collaboration with large language models. In ICRA, 2024.
  40. [Nayak et al., 2024] Siddharth Nayak, Adelmo Morrison Orozco, et al. Long-horizon planning for multi-agent robots in partially observable environments. arXiv, 2024.
  41. [Obata et al., 2024] Kazuma Obata, Tatsuya Aoki, et al. Lip- llm: Integrating linear programming and dependency graph with large language models for multi-robot task planning. IEEE Robotics and Automation Letters, 2024.
  42. [Orr and Dutta, 2023] James Orr and Ayan Dutta. Multi- agent deep reinforcement learning for multi-robot appli- cations: A survey. Sensors, 23(7):3625, 2023.
  43. [Pan et al., 2024] Haotian Pan, Shibo Huang, et al. Recent advances in robot navigation via large language models: A review. 2024.
  44. [Park et al., 2023] J. Park, Seungwon Lim, et al. Clara: clas- sifying and disambiguating user commands for reliable in- teractive robotic agents. IEEE Robotics and Automation Letters, 2023.
  45. [Park et al., 2024] J. Park, J. Cho, et al. Mr.steve: Instruction-following agents in minecraft with what- where-when memory. arXiv, 2024.
  46. [Patel et al., 2024] Bhrij Patel, Vishnu Sashank Dorbala, et al. Embodied question answering via multi-llm systems. arXiv, 2024.
  47. [Qin et al., 2024] Yiran Qin, Enshen Zhou, Qichang Liu, Zhenfei Yin, et al. Mp5: A multi-modal open-ended embodied system in minecraft via active perception. In CVPR, 2024.
  48. [Ren et al., 2023] Allen Z Ren, Anushri Dixit, Alexandra Bodrova, et al. Robots that ask for help: Uncertainty align- ment for large language model planners. In CoRL, 2023.
  49. [Sheng et al., 2022] Junjie Sheng, Xiangfeng Wang, Bo Jin, Junchi Yan, Wenhao Li, et al. Learning structured communication for multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems, 36(2):50, 2022.
  50. [Shi et al., 2024a] Haochen Shi, Zhiyuan Sun, Xingdi Yuan, et al. Opex: A component-wise analysis of llm-centric agents in embodied instruction following. arXiv, 2024.
  51. [Shi et al., 2024b] Lucy X. Shi, Zheyuan Hu, et al. Yell at your robot: Improving on-the-fly from language correc- tions. arXiv, 2024.
  52. [Singh et al., 2024] Apoorv Singh, Gaurav Raut, and Alka Choudhary. Multi-agent collaborative perception for robotic fleet: A systematic review. arXiv, 2024.
  53. [Sun et al., 2024] Chuanneng Sun, Songjun Huang, and Dario Pompili. Llm-based multi-agent reinforcement learning: Current and future directions. arXiv, 2024.
  54. [Tan et al., 2020] Sinan Tan, Weilai Xiang, Huaping Liu, Di Guo, and Fuchun Sun. Multi-agent embodied question answering in interactive environments. In ECCV, 2020.
  55. [Tan et al., 2023] Sinan Tan, Mengmeng Ge, Di Guo, Huap- ing Liu, and Fuchun Sun. Knowledge-based embodied question answering. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 45(10):11948-11960, 2023.
  56. [Wang et al., 2023] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, et al. Voyager: An open-ended embodied agent with large language models. In NeurIPS, 2023.
  57. [Wang et al., 2024] Jun Wang, Guocheng He, and Yiannis Kantaros. Safe task planning for language-instructed multi-robot systems using conformal prediction. arXiv, 2024.
  58. [Weinberg and Rosenschein, 2004] M. Weinberg and J.S Rosenschein. Best-response multiagent learning in non- stationary environments. In AAMAS, 2004.
  59. [Wu et al., 2024] Yuwei Wu, Yuezhan Tao, Peihan Li, et al. Hierarchical llms in-the-loop optimization for real-time multi-robot target tracking under unknown hazards. arXiv, 2024.
  60. [Yan et al., 2013] Zhi Yan, N. Jouandeau, et al. A survey and analysis of multi-robot coordination. International Jour- nal of Advanced Robotic Systems, 10(12):399, 2013.
  61. [Yang et al., 2023] Kun Yang, Dingkang Yang, Jingyu Zhang, et al. Spatio-temporal domain awareness for multi- agent collaborative perception. In ICCV, 2023.
  62. [Ying et al., 2024] Lance Ying, Kunal Jha, et al. Goma: Proactive embodied cooperative communication via goal- oriented mental alignment. arXiv, 2024.
  63. [Yu et al., 2023] Bangguo Yu, Hamidreza Kasaei, and Ming Cao. Co-navgpt: Multi-robot cooperative visual semantic navigation using large language models. arXiv, 2023.
  64. [Yu et al., 2024] Wenhao Yu, Jie Peng, Yueliang Ying, et al. Mhrc: Closed-loop decentralized multi-heterogeneous robot collaboration with large language models. arXiv, 2024.
  65. [Yuan et al., 2023] Lei Yuan, Ziqian Zhang, et al. Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers. In AAAI, 2023.
  66. [Zhang et al., 2023] Bin Zhang, Hangyu Mao, Jingqing Ruan, et al. Controlling large language model-based agents for large-scale decision-making: An actor-critic ap- proach. arXiv, 2023.
  67. [Zhang et al., 2024a] Hongxin Zhang, Weihua Du, Jiaming Shan, et al. Building cooperative embodied agents modu- larly with large language models. In ICLR, 2024.
  68. [Zhang et al., 2024b] Hongxin Zhang, Zeyuan Wang, Qiushi Lyu, et al. Combo: Compositional world models for em- bodied multi-agent cooperation. arXiv, 2024.
  69. [Zhang et al., 2024c] Xiaopan Zhang, Hao Qin, Fuquan Wang, et al. Lamma-p: Generalizable multi-agent long- horizon task allocation and planning with lm-driven pddl planner. arXiv, 2024.
  70. [Zhang et al., 2024d] Yang Zhang, Shixin Yang, Chenjia Bai, Fei Wu, Xiu Li, Zhen Wang, and Xuelong Li. Towards efficient llm grounding for embodied multi-agent collabo- ration. arXiv, 2024.
  71. [Zhao et al., 2024a] Zhonghan Zhao, Kewei Chen, et al. Hi- erarchical auto-organizing system for open-ended multi- agent navigation. In ICLR 2024 Workshop on LLM Agents, 2024.
  72. [Zhao et al., 2024b] Zhonghan Zhao, Ke Ma, Wenhao Chai, Xuan Wang, et al. Do we really need a complex agent system? distill embodied agent into a single model. arXiv, 2024.
  73. [Zhou et al., 2024] Ce Zhou, Qian Li, Chen Li, et al. A com- prehensive survey on pretrained foundation models: A his- tory from bert to chatgpt. International Journal of Ma- chine Learning and Cybernetics, pages 1-65, 2024.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards