A Speech-driven Sign Language Avatar Animation System for Hearing Impaired Applications

A Speech-driven Sign Language Avatar Animation System for Hearing Impaired Applications

Li Hu, Jiahui Li, Jiashuo Zhang, Qi Wang, Bang Zhang, Ping Tan

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Demo Track. Pages 5912-5915. https://doi.org/10.24963/ijcai.2022/852

Sign language is the communication language used in hearing impaired community. Recently, the research of sign language production has made great progress but still need to cope with some critical challenges. In this paper, we propose a system-level scheme and push forward the implementation of sign language production for practical usage. We build a system capable of translating speech into sign language avatar. Different from previous approach only focusing on single technology, we systematically combine algorithms of language translation, body gesture animation and facial avatar generation. We also develop two applications: Sign Language Interpretation APP and Virtual Sign Language Anchor, to facilitate easy and clear communication for hearing impaired people.
Keywords:
Humans and AI: Cognitive Systems
Humans and AI: Applications
Humans and AI: Human-Computer Interaction
Machine Learning: Applications