Composing Neural Learning and Symbolic Reasoning with an Application to Visual Discrimination

Composing Neural Learning and Symbolic Reasoning with an Application to Visual Discrimination

Adithya Murali, Atharva Sehgal, Paul Krogmeier, P. Madhusudan

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 3358-3365. https://doi.org/10.24963/ijcai.2022/466

We consider the problem of combining machine learning models to perform higher-level cognitive tasks with clear specifications. We propose the novel problem of Visual Discrimination Puzzles (VDP) that requires finding interpretable discriminators that classify images according to a logical specification. Humans can solve these puzzles with ease and they give robust, verifiable, and interpretable discriminators as answers. We propose a compositional neurosymbolic framework that combines a neural network to detect objects and relationships with a symbolic learner that finds interpretable discriminators. We create large classes of VDP datasets involving natural and artificial images and show that our neurosymbolic framework performs favorably compared to several purely neural approaches.
Keywords:
Machine Learning: Neuro-Symbolic Methods
Computer Vision: Visual reasoning and symbolic representation
Machine Learning: Explainable/Interpretable Machine Learning
Machine Learning: Few-shot learning
AI Ethics, Trust, Fairness: Trustworthy AI