Type Anywhere You Want: An Introduction to Invisible Mobile Keyboard

Type Anywhere You Want: An Introduction to Invisible Mobile Keyboard

Sahng-Min Yoo, Ue-Hwan Kim, Yewon Hwang, Jong-Hwan Kim

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence
Main Track. Pages 1757-1764. https://doi.org/10.24963/ijcai.2021/242

Contemporary soft keyboards possess limitations: the lack of physical feedback results in an increase of typos, and the interface of soft keyboards degrades the utility of the screen. To overcome these limitations, we propose an Invisible Mobile Keyboard (IMK), which lets users freely type on the desired area without any constraints. To facilitate a data-driven IMK decoding task, we have collected the most extensive text-entry dataset (approximately 2M pairs of typing positions and the corresponding characters). Additionally, we propose our baseline decoder along with a semantic typo correction mechanism based on self-attention, which decodes such unconstrained inputs with high accuracy (96.0%). Moreover, the user study reveals that the users could type faster and feel convenience and satisfaction to IMK with our decoder. Lastly, we make the source code and the dataset public to contribute to the research community.
Keywords:
Humans and AI: Human-Computer Interaction
Humans and AI: Intelligent User Interfaces
Humans and AI: Personalization and User Modeling