Proceedings

RuleMatch: Matching Abstract Rules for Semi-supervised Learning of Human Standard Intelligence Tests

Download BibTeX Cite

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence. Main Track. Pages 1613-1621.

https://doi.org/10.24963/ijcai.2023/179

Abstract

Raven's Progressive Matrices (RPM), one of the standard intelligence tests in human psychology, has recently emerged as a powerful tool for studying abstract visual reasoning (AVR) abilities in machines. Although existing computational models for RPM problems achieve good performance, they require a large number of labeled training examples for supervised learning. In contrast, humans can efficiently solve unlabeled RPM problems after learning from only a few example questions. Here, we develop a semi-supervised learning (SSL) method, called RuleMatch, to train deep models with a small number of labeled RPM questions along with other unlabeled questions. Moreover, instead of using pixel-level augmentation in object perception tasks, we exploit the nature of RPM problems and augment the data at the level of abstract rules. Specifically, we disrupt the possible rules contained among context images in an RPM question and force the two augmented variants of the same unlabeled sample to obey the same abstract rule and predict a common pseudo label for training. Extensive experiments show that the proposed RuleMatch achieves state-of-the-art performance on two popular RAVEN datasets. Our work makes an important stride in aligning abstract analogical visual reasoning abilities in machines and humans. Our Code is at https://github.com/ZjjConan/AVR-RuleMatch.

  1. [Bahdanau et al., 2014] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.
  2. [Barrett et al., 2018] David Barrett, Felix Hill, Adam San- toro, Ari Morcos, and Timothy Lillicrap. Measuring ab- stract reasoning in neural networks. In ICML, pages 511- 520. PMLR, 2018.
  3. [Benny et al., 2021] Yaniv Benny, Niv Pekar, and Lior Wolf. Scale-localized abstract reasoning. In CVPR, pages 12557-12565, 2021.
  4. [Berthelot et al., 2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi- supervised learning. In NeurIPS, volume 32, pages 5049- 5059, 2019.
  5. [Bodla et al., 2018] Navaneeth Bodla, Gang Hua, and Rama Chellappa. Semi-supervised fusedgan for conditional im- age generation. In ECCV, pages 669-683, 2018.
  6. [Conneau et al., 2016] Alexis Conneau, Holger Schwenk, Loïc Barrault, and Yann Lecun. Very deep convolu- tional networks for text classification. arXiv preprint arXiv:1606.01781, 2016.
  7. [Donahue et al., 2014] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In ICML, pages 647-655. PMLR, 2014.
  8. [Gibson et al., 2013] Bryan R Gibson, Timothy T Rogers, and Xiaojin Zhu. Human semi-supervised learning. Topics in cognitive science, 5(1):132-172, 2013.
  9. [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.
  10. [Hu et al., 2021a] Sheng Hu, Yuqing Ma, Xianglong Liu, Yanlu Wei, and Shihao Bai. Stratified rule-aware network for abstract visual reasoning. In AAAI, volume 35, pages 1567-1574, 2021.
  11. [Hu et al., 2021b] Zijian Hu, Zhengyu Yang, Xuefeng Hu, and Ram Nevatia. Simple: similar pseudo label exploita- tion for semi-supervised classification. In CVPR, pages 15099-15108, 2021.
  12. [Jahrens and Martinetz, 2020] Marius Jahrens and Thomas Martinetz. Solving raven's progressive matrices with multi-layer relation networks. In IJCNN, pages 1-6. IEEE, 2020.
  13. [Jeong et al., 2019] Jisoo Jeong, Seungeui Lee, Jeesoo Kim, and Nojun Kwak. Consistency-based semi-supervised learning for object detection. In NeurIPS, volume 32, pages 10759-10768, 2019.
  14. [Kalish et al., 2011] Charles W Kalish, Timothy T Rogers, Jonathan Lang, and Xiaojin Zhu. Can semi-supervised learning explain incorrect beliefs about categories? Cog- nition, 120(1):106-118, 2011.
  15. [Karpathy et al., 2014] Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. Large-scale video classification with convolu- tional neural networks. In CVPR, pages 1725-1732, 2014.
  16. [Katsumata et al., 2022] Kai Katsumata, Duc Minh Vo, and Hideki Nakayama. Ossgan: Open-set semi-supervised im- age generation. In CVPR, pages 11185-11193, 2022.
  17. [Kaul et al., 2022] Prannay Kaul, Weidi Xie, and Andrew Zisserman. Label, verify, correct: A simple few shot object detection method. In CVPR, pages 14237-14247, 2022.
  18. [Kim et al., 2016] Jiwon Kim, Jung Kwon Lee, and Ky- oung Mu Lee. Accurate image super-resolution using very deep convolutional networks. In CVPR, pages 1646-1654, 2016.
  19. [Kingma and Ba, 2015] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015.
  20. [Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, volume 25, pages 1097-1105, 2012.
  21. [Laine and Aila, 2016] Samuli Laine and Timo Aila. Tem- poral ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242, 2016.
  22. [Lovett and Forbus, 2017] Andrew Lovett and Kenneth For- bus. Modeling visual problem solving as analogical rea- soning. Psychological review, 124(1):60, 2017.
  23. [Lovett et al., 2010] Andrew Lovett, Kenneth Forbus, and Jeffrey Usher. A structure-mapping model of raven's pro- gressive matrices. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 32, 2010.
  24. [Misra et al., 2015] Ishan Misra, Abhinav Shrivastava, and Martial Hebert. Watch and learn: Semi-supervised learn- ing for object detectors from video. In CVPR, pages 3593- 3602, 2015.
  25. [Raven and Court, 1938] John C Raven and JH Court. Raven's progressive matrices. Western Psychological Ser- vices Los Angeles, CA, 1938.
  26. [Ren et al., 2015] Shaoqing Ren, Kaiming He, Ross Gir- shick, and Jian Sun. Faster r-cnn: Towards real-time ob- ject detection with region proposal networks. In NeurIPS, pages 91-99, 2015.
  27. [Sharif Razavian et al., 2014] Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. Cnn features off-the-shelf: an astounding baseline for recogni- tion. In CVPR workshops, pages 806-813, 2014.
  28. [Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  29. [Sohn et al., 2020] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learning with con- sistency and confidence. In NeurIPS, volume 33, pages 596-608, 2020.
  30. [Spratley et al., 2020] Steven Spratley, Krista Ehinger, and Tim Miller. A closer look at generalisation in raven. In ECCV, pages 601-616. Springer, 2020.
  31. [Tang et al., 2021] Peng Tang, Chetan Ramaiah, Yan Wang, Ran Xu, and Caiming Xiong. Proposal learning for semi- supervised object detection. In WACV, pages 2291-2301, 2021.
  32. [Tarvainen and Valpola, 2017] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight- averaged consistency targets improve semi-supervised deep learning results. In NeurIPS, volume 30, pages 1195- 1204, 2017.
  33. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 6000-6010, 2017.
  34. [Wang et al., 2020] Duo Wang, Mateja Jamnik, and Pietro Lio. Abstract diagrammatic reasoning with multiplex graph networks. arXiv preprint arXiv:2006.11197, 2020.
  35. [Welling and Kipf, 2016] Max Welling and Thomas N Kipf. Semi-supervised classification with graph convolutional networks. In ICLR, 2016.
  36. [Wu et al., 2020] Yuhuai Wu, Honghua Dong, Roger Grosse, and Jimmy Ba. The scattering compositional learner: Dis- covering objects, attributes, relationships in analogical rea- soning. arXiv preprint arXiv:2007.04212, 2020.
  37. [Xu et al., 2021] Mengde Xu, Zheng Zhang, Han Hu, Jian- feng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu. End-to-end semi-supervised object detection with soft teacher. In ICCV, pages 3060-3069, 2021.
  38. [Yamins et al., 2014] Daniel L. K. Yamins, Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and James J. DiCarlo. Performance-optimized hierarchi- cal models predict neural responses in higher visual cor- tex. Proceedings of the National Academy of Sciences, 111(23):8619-8624, 2014.
  39. [Yang et al., 2016] Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In ICML, pages 40-48. PMLR, 2016.
  40. [Zhang et al., 2019a] Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu. Raven: A dataset for relational and analogical visual reasoning. In CVPR, pages 5317-5327, 2019.
  41. [Zhang et al., 2019b] Chi Zhang, Baoxiong Jia, Feng Gao, Yixin Zhu, Hongjing Lu, and Song-Chun Zhu. Learning perceptual inference by contrasting. In NeurIPS, pages 1075-1087, 2019.
  42. [Zhang et al., 2021] Chi Zhang, Baoxiong Jia, Song-Chun Zhu, and Yixin Zhu. Abstract spatial-temporal reasoning via probabilistic abduction and execution. In CVPR, pages 9736-9746, 2021.
  43. [Zheng et al., 2019] Kecheng Zheng, Zheng-Jun Zha, and Wei Wei. Abstract reasoning with distracting features. In NeurIPS, pages 5842-5853, 2019.
  44. [Zhu and Goldberg, 2009] Xiaojin Zhu and Andrew B Gold- berg. Introduction to semi-supervised learning. Synthe- sis lectures on artificial intelligence and machine learning, 3(1):1-130, 2009.
  45. [Zhu et al., 2007] Xiaojin Zhu, Timothy Rogers, Ruichen Qian, and Chuck Kalish. Humans perform semi- supervised classification too. In AAAI, volume 2007, pages 864-870, 2007.
  46. [Zhu et al., 2010] Xiaojin Zhu, Bryan R Gibson, Kwang- Sung Jun, Timothy T Rogers, Joseph Harrison, and Chuck Kalish. Cognitive models of test-item effects in human category learning. In ICML, pages 1247-1254, 2010.
  47. [Zhu, 2005] Xiaojin Zhu. Semi-supervised learning litera- ture survey. Computer Sciences, University of Wisconsin- Madison, 10:1-60, 2005.
  48. [Zhuang et al., 2021] Chengxu Zhuang, Siming Yan, Aran Nayebi, Martin Schrimpf, Michael C Frank, James J Di- Carlo, and Daniel LK Yamins. Unsupervised neural net- work models of the ventral visual stream. Proceedings of the National Academy of Sciences, 118(3):e2014196118, 2021.
  49. [Zhuo and Kankanhalli, 2022] Tao Zhuo and Mohan Kankanhalli. Effective abstract reasoning with dual- contrast network. arXiv preprint arXiv:2205.13720, 2022.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards