TreeNet: Learning Sentence Representations with Unconstrained Tree Structure

TreeNet: Learning Sentence Representations with Unconstrained Tree Structure

Zhou Cheng, Chun Yuan, Jiancheng Li, Haiqin Yang

Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence
Main track. Pages 4005-4011. https://doi.org/10.24963/ijcai.2018/557

Recursive neural network (RvNN) has been proved to be an effective and promising tool to learn sentence representations by explicitly exploiting the sentence structure. However, most existing work can only exploit simple tree structure, e.g., binary trees, or ignore the order of nodes, which yields suboptimal performance. In this paper, we proposed a novel neural network, namely TreeNet, to capture sentences structurally over the raw unconstrained constituency trees, where the number of child nodes can be arbitrary. In TreeNet, each node is learning from its left sibling and right child in a bottom-up left-to-right order, thus enabling the net to learn over any tree. Furthermore, multiple soft gates and a memory cell are employed in implementing the TreeNet to determine to what extent it should learn, remember and output, which proves to be a simple and efficient mechanism for semantic synthesis. Moreover, TreeNet significantly suppresses convolutional neural networks (CNN) and Long Short-Term Memory (LSTM) with fewer parameters. It improves the classification accuracy by 2%-5% with 42% of the best CNN’s parameters or 94% of standard LSTM’s. Extensive experiments demonstrate TreeNet achieves the state-of-the-art performance on all four typical text classification tasks.
Keywords:
Machine Learning: Neural Networks
Natural Language Processing: Natural Language Semantics
Natural Language Processing: Natural Language Processing
Machine Learning: Deep Learning
Natural Language Processing: Text Classification