Densely Connected CNN with Multi-scale Feature Attention for Text Classification

Densely Connected CNN with Multi-scale Feature Attention for Text Classification

Shiyao Wang, Minlie Huang, Zhidong Deng

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 4468-4474. https://doi.org/10.24963/ijcai.2018/621

Text classification is a fundamental problem in natural language processing. As a popular deep learning model, convolutional neural network (CNN) has demonstrated great success in this task. However, most existing CNN models apply convolution filters of fixed window size, thereby unable to learn variable n-gram features flexibly. In this paper, we present a densely connected CNN with multi-scale feature attention for text classification. The dense connections build short-cut paths between upstream and downstream convolutional blocks, which enable the model to compose features of larger scale from those of smaller scale, and thus produce variable n-gram features. Furthermore, a multi-scale feature attention is developed to adaptively select multi-scale features for classification. Extensive experiments demonstrate that our model obtains competitive performance against state-of-the-art baselines on five benchmark datasets. Attention visualization further reveals the model's ability to select proper n-gram features for text classification.
Keywords:
Natural Language Processing: Natural Language Semantics
Natural Language Processing: Natural Language Processing
Natural Language Processing: Text Classification
Natural Language Processing: Embeddings