Proceedings

Hypergraph Self-supervised Learning with Sampling-efficient Signals

Download BibTeX Cite

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. Main Track. Pages 4398-4406.

https://doi.org/10.24963/ijcai.2024/486

Abstract

Self-supervised learning (SSL) provides a promising alternative for representation learning on hypergraphs without costly labels. However, existing hypergraph SSL models are mostly based on contrastive methods with the instance-level discrimination strategy, suffering from two significant limitations: (1) They select negative samples arbitrarily, which is unreliable in deciding similar and dissimilar pairs, causing training bias. (2) They often require a large number of negative samples, resulting in expensive computational costs. To address the above issues, we propose SE-HSSL, a hypergraph SSL framework with three sampling-efficient self-supervised signals. Specifically, we introduce two sampling-free objectives leveraging the canonical correlation analysis as the node-level and group-level self-supervised signals. Additionally, we develop a novel hierarchical membership-level contrast objective motivated by the cascading overlap relationship in hypergraphs, which can further reduce membership sampling bias and improve the efficiency of sample utilization. Through comprehensive experiments on 7 real-world hypergraphs, we demonstrate the superiority of our approach over the state-of-the-art method in terms of both effectiveness and efficiency.

  1. [Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International conference on machine learn- ing, pages 1247-1255. PMLR, 2013.
  2. [Antelmi et al., 2023] Alessia Antelmi, Gennaro Cordasco, Mirko Polato, Vittorio Scarano, Carmine Spagnuolo, and Dingqi Yang. A survey on hypergraph representation learning. ACM Computing Surveys, 56(1):1-38, 2023.
  3. [Cai et al., 2022] Derun Cai, Moxian Song, Chenxi Sun, Baofeng Zhang, Shenda Hong, and Hongyan Li. Hyper- graph structure learning for hypergraph neural networks. In Proceedings of the Thirty-First International Joint Con- ference on Artificial Intelligence, IJCAI-22, pages 1923- 1929, 2022. IJCAI
  4. [Chang et al., 2018] Xiaobin Chang, Tao Xiang, and Timo- thy M Hospedales. Scalable and effective deep cca via soft decorrelation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1488- 1497, 2018.
  5. [Chien et al., 2021] Eli Chien, Chao Pan, Jianhao Peng, and Olgica Milenkovic. You are allset: A multiset function framework for hypergraph neural networks. In Interna- tional Conference on Learning Representations, 2021.
  6. [Feng et al., 2019] Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural net- works. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 3558-3565, 2019.
  7. [He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for unsu- pervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.
  8. [Hein et al., 2013] Matthias Hein, Simon Setzer, Leonardo Jost, and Syama Sundar Rangapuram. The total vari- ation on hypergraphs-learning on hypergraphs revisited. Advances in Neural Information Processing Systems, 26, 2013.
  9. [Hua et al., 2021] Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, and Hang Zhao. On feature decorrelation in self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9598-9608, 2021.
  10. [Jia et al., 2021] Renqi Jia, Xiaofei Zhou, Linhua Dong, and Shirui Pan. Hypergraph convolutional network for group recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 260-269. IEEE, 2021.
  11. [Jing et al., 2014] Xiao-Yuan Jing, Rui-Min Hu, Yang-Ping Zhu, Shan-Shan Wu, Chao Liang, and Jing-Yu Yang. Intra- view and inter-view supervised correlation analysis for multi-view feature learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
  12. [Kim et al., 2020] Eun-Sol Kim, Woo Young Kang, Kyoung-Woon On, Yu-Jung Heo, and Byoung-Tak Zhang. Hypergraph attention networks for multimodal learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14581-14590, 2020.
  13. [Lee and Shin, 2023] Dongjin Lee and Kijung Shin. I'm me, we're us, and i'm us: Tri-directional contrastive learning on hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8456-8464, 2023.
  14. [Liang et al., 2023] Yongquan Liang, Qiuyu Song, Zhongy- ing Zhao, Hui Zhou, and Maoguo Gong. Ba- gnn: Behavior-aware graph neural network for session- based recommendation. Frontiers of Computer Science, 17(6):176613, 2023.
  15. [Miech et al., 2020] Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zis- serman. End-to-end learning of visual representations from uncurated instructional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9879-9889, 2020.
  16. [Thakoor et al., 2021] Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Rémi Munos, Petar Veličković, and Michal Valko. Bootstrapped representa- tion learning on graphs. In ICLR 2021 Workshop on Geo- metrical and Topological Representation Learning, 2021.
  17. [Veličković et al., 2018] Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. In International Conference on Learning Representations, 2018.
  18. [Wei et al., 2022] Tianxin Wei, Yuning You, Tianlong Chen, Yang Shen, Jingrui He, and Zhangyang Wang. Augmenta- tions in hypergraph contrastive learning: Fabricated and generative. Advances in neural information processing systems, 35:1909-1922, 2022.
  19. [Xia et al., 2022a] Lianghao Xia, Chao Huang, Yong Xu, Ji- ashu Zhao, Dawei Yin, and Jimmy Huang. Hypergraph contrastive collaborative filtering. In Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval, pages 70-79, 2022.
  20. [Xia et al., 2022b] Lianghao Xia, Chao Huang, and Chuxu Zhang. Self-supervised hypergraph transformer for rec- ommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2100-2109, 2022.
  21. [Xiao et al., 2019] Li Xiao, Junqi Wang, Peyman H Kas- sani, Yipu Zhang, Yuntong Bai, Julia M Stephen, Tony W Wilson, Vince D Calhoun, and Yu-Ping Wang. Multi- hypergraph learning-based brain functional connectivity analysis in fmri data. IEEE transactions on medical imag- ing, 39(5):1746-1758, 2019.
  22. [Yadati et al., 2019] Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergcn: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019.
  23. [Yang et al., 2021] Haoran Yang, Hongxu Chen, Lin Li, S Yu Philip, and Guandong Xu. Hyper meta-path contrastive learning for multi-behavior recommendation. In 2021 IEEE International Conference on Data Mining (ICDM), pages 787-796. IEEE, 2021.
  24. [Yang et al., 2022] Chaoqi Yang, Ruijie Wang, Shuochao Yao, and Tarek Abdelzaher. Semi-supervised hypergraph node classification on hypergraph line expansion. In Pro- ceedings of the 31st ACM International Conference on In- formation & Knowledge Management, pages 2352-2361, 2022.
  25. [Yi and Park, 2020] Jaehyuk Yi and Jinkyoo Park. Hyper- graph convolutional recurrent neural network. In Proceed- ings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 3366-3376, 2020.
  26. [You et al., 2020] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph con- trastive learning with augmentations. Advances in neural information processing systems, 33:5812-5823, 2020.
  27. [Yu et al., 2012] Jun Yu, Dacheng Tao, and Meng Wang. Adaptive hypergraph learning and its application in image classification. IEEE Transactions on Image Processing, 21(7):3262-3272, 2012.
  28. [Zhang et al., 2021a] Hengrui Zhang, Qitian Wu, Junchi Yan, David Wipf, and Philip S Yu. From canonical correla- tion analysis to self-supervised graph neural networks. Ad- vances in Neural Information Processing Systems, 34:76- 89, 2021.
  29. [Zhang et al., 2021b] Junwei Zhang, Min Gao, Junliang Yu, Lei Guo, Jundong Li, and Hongzhi Yin. Double-scale self-supervised hypergraph learning for group recommen- dation. In Proceedings of the 30th ACM international con- ference on information & knowledge management, pages 2557-2567, 2021.
  30. [Zhang et al., 2022] Yifei Zhang, Hao Zhu, Zixing Song, Pi- otr Koniusz, and Irwin King. Costa: covariance-preserving feature augmentation for graph contrastive learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2524- 2534, 2022.
  31. [Zhang et al., 2023] Yifei Zhang, Hao Zhu, Zixing Song, Pi- otr Koniusz, and Irwin King. Spectral feature augmen- tation for graph contrastive learning and beyond. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11289-11297, 2023.
  32. [Zhao et al., 2021] Han Zhao, Xu Yang, Zhenru Wang, Erkun Yang, and Cheng Deng. Graph debiased contrastive learning with joint representation clustering. In IJCAI, pages 3434-3440, 2021. IJCAI
  33. [Zhou et al., 2006] Dengyong Zhou, Jiayuan Huang, and Bernhard Schölkopf. Learning with hypergraphs: Clus- tering, classification, and embedding. Advances in neural information processing systems, 19, 2006.
  34. [Zhu et al., 2020] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131, 2020.
  35. [Zhu et al., 2021] Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021, pages 2069-2080, 2021.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards