Explicit Knowledge-based Reasoning for Visual Question Answering

Explicit Knowledge-based Reasoning for Visual Question Answering

Peng Wang, Qi Wu, Chunhua Shen, Anthony Dick, Anton van den Hengel

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 1290-1296. https://doi.org/10.24963/ijcai.2017/179

We describe a method for visual question answering which is capable of reasoning about an image on the basis of information extracted from a large-scale knowledge base. The method not only answers natural language questions using concepts not contained in the image, but can explain the reasoning by which it developed its answer. It is capable of answering far more complex questions than the predominant long short-term memory-based approach, and outperforms it significantly in testing. We also provide a dataset and a protocol by which to evaluate general visual question answering methods.
Keywords:
Knowledge Representation, Reasoning, and Logic: Common-Sense Reasoning
Natural Language Processing: Question Answering
Knowledge Representation, Reasoning, and Logic: Reasoning about Knowlege and Belief