Convolutional-Match Networks for Question Answering

Convolutional-Match Networks for Question Answering

Spyridon Samothrakis, Tom Vodopivec, Michael Fairbank, Maria Fasli

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 2686-2692. https://doi.org/10.24963/ijcai.2017/374

In this paper, we present a simple, yet effective, attention and memory mechanism that is reminiscent of Memory Networks and we demonstrate it in question-answering scenarios. Our mechanism is based on four simple premises: a) memories can be formed from word sequences by using convolutional networks; b) distance measurements can be taken at a neuronal level; c) a recursive softmax function can be used for attention; d) extensive weight sharing can help profoundly. We achieve state-of-the-art results in the bAbI tasks, outperforming both Memory Networks and the Differentiable Neural Computer, both in terms of accuracy and stability (i.e. variance) of results.
Keywords:
Machine Learning: Neural Networks
Machine Learning: Deep Learning