Dynamically Route Hierarchical Structure Representation to Attentive Capsule for Text Classification

Dynamically Route Hierarchical Structure Representation to Attentive Capsule for Text Classification

Wanshan Zheng, Zibin Zheng, Hai Wan, Chuan Chen

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
Main track. Pages 5464-5470. https://doi.org/10.24963/ijcai.2019/759

Representation learning and feature aggregation are usually the two key intermediate steps in natural language processing. Despite deep neural networks have shown strong performance in the text classification task, they are unable to learn adaptive structure features automatically and lack of a method for fully utilizing the extracted features. In this paper, we propose a novel architecture that dynamically routes hierarchical structure feature to attentive capsule, named HAC. Specifically, we first adopt intermediate information of a well-designed deep dilated CNN to form hierarchical structure features. Different levels of structure representations are corresponding to various linguistic units such as word, phrase and clause, respectively. Furthermore, we design a capsule module using dynamic routing and equip it with an attention mechanism. The attentive capsule implements an effective aggregation strategy for feature clustering and selection. Extensive results on eleven benchmark datasets demonstrate that the proposed model obtains competitive performance against several state-of-the-art baselines. Our code is available at https://github.com/zhengwsh/HAC.
Keywords:
Natural Language Processing: Text Classification
Natural Language Processing: Sentiment Analysis and Text Mining
Natural Language Processing: Natural Language Processing