AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition

AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition

HaoJie Ma, Wenzhong Li, Xiao Zhang, Songcheng Gao, Sanglu Lu

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Main track. Pages 3109-3115. https://doi.org/10.24963/ijcai.2019/431

Sensor-based human activity recognition is a fundamental research problem in ubiquitous computing, which uses the rich sensing data from multimodal embedded sensors such as accelerometer and gyroscope to infer human activities. The existing activity recognition approaches either rely on domain knowledge or fail to address the spatial-temporal dependencies of the sensing signals. In this paper, we propose a novel attention-based multimodal neural network model called AttnSense for multimodal human activity recognition. AttnSense introduce the framework of combining attention mechanism with a convolutional neural network (CNN) and a Gated Recurrent Units (GRU) network to capture the dependencies of sensing signals in both spatial and temporal domains, which shows advantages in prioritized sensor selection and improves the comprehensibility. Extensive experiments based on three public datasets show that AttnSense achieves a competitive performance in activity recognition compared with several state-of-the-art methods.
Keywords:
Machine Learning: Deep Learning
Machine Learning Applications: Applications of Supervised Learning
Multidisciplinary Topics and Applications: Ubiquitous Computing Systems