Proceedings

MGCA-Net: Multi-Graph Contextual Attention Network for Two-View Correspondence Learning

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 1539-1547.

https://doi.org/10.24963/ijcai.2025/172

Abstract

Two-view correspondence learning is a key task in computer vision, which aims to establish reliable matching relationships for applications such as camera pose estimation and 3D reconstruction. However, existing methods have limitations in local geometric modeling and cross-stage information optimization, which make it difficult to accurately capture the geometric constraints of matched pairs and thus reduce the robustness of the model. To address these challenges, we propose a Multi-Graph Contextual Attention Network (MGCA-Net), which consists of a Contextual Geometric Attention (CGA) module and a Cross-Stage Multi-Graph Consensus (CSMGC) module. Specifically, CGA dynamically integrates spatial position and feature information via an adaptive attention mechanism and enhances the capability to capture both local and global geometric relationships. Meanwhile, CSMGC establishes geometric consensus via a cross-stage sparse graph network, ensuring the consistency of geometric information across different stages. Experimental results on two representative YFCC100M and SUN3D datasets show that MGCA-Net significantly outperforms existing SOTA methods in the outlier rejection and camera pose estimation tasks. Source code is available at http://www.linshuyuan.com.

  1. Dataset YFCC100M (%) SUN3D (%) Method Known Scene Unknown Scene Known Scene Unknown Scene P (%) R (%) F (%) P (%) R (%) F (%) P (%) R (%) F (%) P (%) R (%) F (%) RANSAC [Fischler and Bolles, 1981] 47.35 52.39 49.47 43.55 50.65 46.83 51.87 56.27 53.98 44.87 48.82 46.76 PointNet++ [Qi et al., 2017] 49.62 86.19 62.98 46.39 84.17 59.81 52.89 86.25 65.57 46.30 82.72 59.37 LFGC-Net [Yi et al., 2018] 54.43 86.88 66.93 52.84 85.68 65.37 53.70 87.03 66.42 46.11 83.92 59.52 DFE-Net [Ranftl and Koltun, 2018] 56.72 87.16 68.72 54.00 85.56 66.21 53.96 87.23 66.68 46.18 84.01 59.60 ACNe-Net [Sun et al., 2020] 60.02 88.99 71.69 55.62 85.47 67.39 54.11 88.46 67.15 46.16 84.01 59.58 OANet [Zhang et al., 2019] 61.14 88.16 69.73 57.90 85.07 66.53 54.43 88.08 63.72 46.50 83.83 56.32 T-Net [Zhong et al., 2021] 61.18 89.94 70.47 57.18 87.01 66.73 55.01 88.36 64.18 46.50 83.98 56.33 PESA-Net [Zhong et al., 2022] 61.43 89.63 72.90 58.02 87.01 69.62 55.08 88.56 67.92 47.29 84.81 60.72 MSA-Net [Zheng et al., 2022] 59.27 90.28 68.92 56.49 88.60 66.46 56.09 87.57 64.71 48.64 83.81 57.89 MS²DG-Net [Dai et al., 2022] 64.24 89.31 72.49 60.38 86.71 68.96 55.58 89.01 64.63 47.42 84.50 57.12
  2. U-Match [Li et al., 2023] 63.29 92.12 72.56 61.02 90.64 70.59 55.29 89.35 64.53 47.69 85.6 57.53 PGFNet [Liu et al., 2023] 59.21 90.22 68.77 56.38 88.15 66.20 55.80 87.80 64.68 48.37 84.03 57.89 GCA-Net [Guo et al., 2023] 63.82 91.36 72.87 60.44 88.92 69.74 55.00 89.08 64.21 46.88 85.14 56.79 ConvMatch [Zhang and Ma, 2023a] 63.14 91.2 72.21 60.22 89.48 69.65 55.79 89.23 64.89 48.13 85.55 57.87 ConvMatch + [Zhang and Ma, 2023b] 62.75 92.05 72.11 59.41 90.12 69.13 55.51 89.30 64.80 47.60 85.51 57.53 NCM-Net [Liu et al., 2024] 77.92 81.41 79.25 76.83 78.61 77.45 66.15 74.59 69.38 60.92 68.94 64.04 PT-Net [Gong et al., 2024] 65.69 90.61 74.14 62.14 89.22 71.16 55.41 89.17 64.66 47.45 85.52 57.39 DeMatch [Zhang et al., 2024] 61.33 92.77 71.19 58.74 91.02 68.91 56.00 88.90 65.10 48.27 85.24 58.07 BCLNet [Miao et al., 2024] 78.36 82.23 79.87 77.90 80.07 78.73 66.20 74.12 69.19 61.14 68.33 63.92 MSGSA [Lin et al., 2024a] 63.29 91.37 72.40 60.03 89.34 69.53 55.92 88.56 65.08 47.99 84.53 57.81 CGR-Net [Yang et al., 2024] 78.31 82.34 79.90 77.22 79.61 78.14 66.46 74.46 69.51 61.24 68.85 64.18 MGCA-Net 84.84 84.13 83.83 83.62 81.07 81.82 74.91 74.29 74.31 70.03 69.86 69.63 Dataset YFCC100M (%) SUN3D (%) Method Known Scene Unknown Scene Known Scene Unknown Scene 5° 20° 5° 20° 5° 20° 5° 20° RANSAC [Fischler and Bolles, 1981] 5.74 / - 16.67 / - 9.05 / - 22.71 / - 4.43 / - 15.38 / - 2.85 / - 11.23 / - PointNet++ [Qi et al., 2017] 11.88 / - 32.86 / - 15.98 / - 44.82 / - 8.78 / - 31.02 / - 7.22 / - 29.77 / - LFGC-Net [Yi et al., 2018] 14.51 / - 35.82 / - 23.71 / - 50.57 / - 11.93 / - 36.03 / - 9.73 / - 33.09 / - DFE-Net [Ranftl and Koltun, 2018] 19.27 / - 42.14 / - 30.55 / - 59.15 / - 14.18 / - 39.14 / - 12.13 / - 26.26 / - ACNe-Net [Sun et al., 2020] 29.63 / - 52.71 / - 34.00 / - 62.98 / - 19.08 / - 46.32 / - 14.27 / - 39.29 / - OANet [Zhang et al., 2019] 33.75 / 14.49 57.13 / 48.04 40.80 / 17.00 69.26 / 58.61 21.54 / 8.04 48.91 / 40.14 16.37 / 6.09 41.82 / 34.08
  3. T-Net [Zhong et al., 2021] 41.46 / 18.20 64.14 / 54.48 48.40 / 20.70 73.92 / 62.94 23.63 / 9.03 51.04 / 42.08 18.04 / 6.62 43.65 / 35.66 PESA-Net [Zhong et al., 2022] 37.15 / 16.09 59.76 / 50.32 45.03 / 19.73 71.95 / 61.23 22.67 / 9.07 50.02 / 42.46 18.00 / 7.26 44.10 / 37.47 MSA-Net [Zheng et al., 2022] 36.04 / 15.69 58.81 / 49.68 48.70 / 20.95 73.07 / 62.31 19.58 / 7.24 47.03 / 38.54 16.77 / 6.07 42.10 / 34.32 MS²DG-Net [Dai et al., 2022] 36.46 / 15.36 61.94 / 52.18 46.88 / 18.84 74.84 / 63.27 22.57 / 8.51 50.89 / 41.81 17.19 / 6.24 43.27 / 35.22
  4. U-Match [Li et al., 2023] 46.22 / 21.73 67.67 / 57.90 60.15 / 29.59 79.69 / 69.03 26.45 / 10.20 53.56 / 44.38 22.41 / 8.39 48.65 / 40.07 PGFNet [Liu et al., 2023] 33.54 / 14.17 56.73 / 47.57 46.70 / 20.48 72.26 / 61.62 22.63 / 8.64 49.26 / 40.55 19.02 / 7.14 44.70 / 36.61 GCA-Net [Guo et al., 2023] 44.32 / 19.93 67.24 / 57.16 55.70 / 24.90 79.58 / 68.21 22.57 / 8.61 50.39 / 41.49 18.53 / 6.85 44.27 / 36.08 ConvMatch [Zhang and Ma, 2023a] 43.25 / 20.02 65.61 / 55.81 55.45 / 26.69 77.53 / 66.87 27.45 / 10.84 54.65 / 45.35 22.52 / 8.68 48.64 / 40.09 ConvMatch+ [Zhang and Ma, 2023b] 45.79 / 21.19 67.69 / 57.72 58.07 / 27.79 78.88 / 68.00 27.22 / 10.81 54.97 / 45.70 22.67 / 8.62 49.13 / 40.51
  5. NCM-Net [Liu et al., 2024] 50.24 / 25.02 71.27 / 61.31 62.65 / 32.40 82.30 / 71.82 24.99 / 9.96 51.87 / 42.88 20.41 / 7.92 46.42 / 38.16 PT-Net [Gong et al., 2024] 49.16 / 20.23 71.07 / 56.23 61.62 / 27.06 81.21 / 67.45 27.23 / 10.67 54.38 / 45.11 22.88 / 8.59 48.52 / 39.85 DeMatch [Zhang et al., 2024] 47.56 / 22.51 69.14 / 59.17 60.00 / 30.01 80.02 / 69.45 28.49 / 11.34 55.59 / 46.24 23.46 / 9.20 49.84 / 41.20 BCLNet [Miao et al., 2024] 53.21 / 27.48 73.48 / 63.63 67.85 / 37.22 84.57 / 74.44 24.32 / 9.64 51.24 / 42.37 20.06 / 7.72 45.83 / 37.62 MSGSA [Lin et al., 2024a] 53.87 / 21.99 74.04 / 58.26 63.78 / 28.23 82.23 / 68.31 25.28 / 10.43 52.29 / 44.99 20.41 / 7.81 46.12 / 39.00
  6. CGR-Net [Yang et al., 2024] 53.56 / 26.45 73.98 / 62.97 66.47 / 35.01 84.14 / 73.17 26.48 / 10.58 53.41 / 44.28 21.69 / 8.51 47.94 / 39.53 MGCA-Net 65.18 / 35.79 81.40 / 71.35 77.10 / 44.62 89.23 / 79.36 33.06 / 14.18 59.22 / 49.67 25.49 / 10.43 51.54 / 42.80
  7. [Barath and Matas, 2018] Daniel Barath and Jiří Matas. Graph-cut ransac. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6733- 6741, 2018.
  8. [Brachmann and Rother, 2019] Eric Brachmann and Carsten Rother. Neural-guided ransac: Learning where to sample model hypotheses. In Proceedings of the IEEE Interna- tional Conference on Computer Vision, pages 4322-4331, 2019.
  9. [Chen et al., 2024] Changhao Chen, Bing Wang, Chris Xi- aoxuan Lu, Niki Trigoni, and Andrew Markham. Deep learning for visual localization and mapping: A survey. IEEE Transactions on Neural Networks and Learning Sys- tems, 35(12):17000-17020, 2024.
  10. [Dai et al., 2022] Luanyuan Dai, Yizhang Liu, Jiayi Ma, Li- fang Wei, Taotao Lai, Changcai Yang, and Riqing Chen. Ms2dg-net: Progressive correspondence learning via mul- tiple sparse semantics dynamic graph. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8973-8982, 2022.
  11. [DeTone et al., 2018] Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superpoint: Self-supervised in- terest point detection and description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 224-236, 2018.
  12. [Fischler and Bolles, 1981] Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381- 395, 1981.
  13. [Gong et al., 2024] Zhepeng Gong, Guobao Xiao, Ziwei Shi, Shiping Wang, and Riqing Chen. Pt-net: Pyramid trans- former network for feature matching learning. IEEE Transactions on Instrumentation and Measurement, 73:1- 11, 2024.
  14. [Guo et al., 2023] Junwen Guo, Guobao Xiao, Zhimin Tang, Shunxing Chen, Shiping Wang, and Jiayi Ma. Learning for feature matching via graph context attention. IEEE Transactions on Geoscience and Remote Sensing, 61:1- 14, 2023.
  15. [Li et al., 2023] Zizhuo Li, Shihua Zhang, and Jiayi Ma. U- match: Two-view correspondence learning with hierarchy- aware local context aggregation. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 1169-1176, 2023.
  16. [Liao et al., 2023] Tangfei Liao, Xiaoqin Zhang, Yuewang Xu, Ziwei Shi, and Guobao Xiao. Sga-net: A sparse graph attention network for two-view correspondence learning. IEEE Transactions on Circuits and Systems for Video Technology, 33(12):7578-7590, 2023.
  17. [Liao et al., 2024] Tangfei Liao, Xiaoqin Zhang, Li Zhao, Tao Wang, and Guobao Xiao. Vsformer: Visual-spatial fusion transformer for correspondence pruning. In Pro- ceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 3369-3377, 2024.
  18. [Lin et al., 2022] Shuyuan Lin, Hailing Luo, Yan Yan, Guobao Xiao, and Hanzi Wang. Co-clustering on bipar- tite graphs for robust model fitting. IEEE Transactions on Image Processing, 31:6605-6620, 2022.
  19. [Lin et al., 2024a] Shuyuan Lin, Xiao Chen, Guobao Xiao, Hanzi Wang, Feiran Huang, and Jian Weng. Multi-stage network with geometric semantic attention for two-view correspondence learning. IEEE Transactions on Image Processing, 33:3031-3046, 2024.
  20. [Lin et al., 2024b] Shuyuan Lin, Feiran Huang, Taotao Lai, Jianhuang Lai, Hanzi Wang, and Jian Weng. Robust het- erogeneous model fitting for multi-source image corre- spondences. International Journal of Computer Vision, 132:2907-2928, 2024.
  21. [Liu et al., 2023] Xin Liu, Guobao Xiao, Riqing Chen, and Jiayi Ma. Pgfnet: Preference-guided filtering network for two-view correspondence learning. IEEE Transactions on Image Processing, 32:1367-1378, 2023.
  22. [Liu et al., 2024] Xin Liu, Rong Qin, Junchi Yan, and Jufeng Yang. Ncmnet: Neighbor consistency mining network for two-view correspondence pruning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):11254- 11272, 2024.
  23. [Lowe, 2004] David G Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, pages 91-110, 2004.
  24. [Ma et al., 2014] Jiayi Ma, Ji Zhao, Jinwen Tian, Alan L Yuille, and Zhuowen Tu. Robust point matching via vector field consensus. IEEE Transactions on Image Processing, pages 1706-1721, 2014.
  25. [Ma et al., 2019] Jiayi Ma, Ji Zhao, Junjun Jiang, Huabing Zhou, and Xiaojie Guo. Locality preserving matching. International Journal of Computer Vision, 127:512-531, 2019. IJCAI
  26. [Ma et al., 2021] Jiayi Ma, Xingyu Jiang, Aoxiang Fan, Jun- jun Jiang, and Junchi Yan. Image matching from hand- crafted to deep features: A survey. International Journal of Computer Vision, pages 23-79, 2021.
  27. [Miao et al., 2024] Xiangyang Miao, Guobao Xiao, Shiping Wang, and Jun Yu. Bclnet: Bilateral consensus learning for two-view correspondence pruning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4225-4232, 2024.
  28. [Placed et al., 2023] Julio A Placed, Jared Strader, Henry Carrillo, Nikolay Atanasov, Vadim Indelman, Luca Car- lone, and José A Castellanos. A survey on active simulta- neous localization and mapping: State of the art and new frontiers. IEEE Transactions on Robotics, 39(3):1686- 1705, 2023.
  29. [Qi et al., 2017] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the Advances in Neural Information Processing Systems, pages 5099-5108, 2017.
  30. [Raguram et al., 2012] Rahul Raguram, Ondrej Chum, Marc Pollefeys, Jiri Matas, and Jan-Michael Frahm. Usac: A universal framework for random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, pages 2022-2038, 2012.
  31. [Ranftl and Koltun, 2018] René Ranftl and Vladlen Koltun. Deep fundamental matrix estimation. In Proceedings of the European Conference on Computer Vision, pages 284- 299, 2018.
  32. [Sarlin et al., 2020] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4938-4947, 2020.
  33. [Schmied et al., 2023] Aron Schmied, Tobias Fischer, Mar- tin Danelljan, Marc Pollefeys, and Fisher Yu. R3d3: Dense 3d reconstruction of dynamic scenes from multiple cam- eras. In Proceedings of the IEEE International Conference on Computer Vision, pages 3216-3226, 2023.
  34. [Schonberger and Frahm, 2016] Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4104-4113, 2016.
  35. [Sun et al., 2020] Weiwei Sun, Wei Jiang, Eduard Trulls, Andrea Tagliasacchi, and Kwang Moo Yi. Acne: Attentive context normalization for robust permutation-equivariant learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 11286- 11295, 2020.
  36. [Sun et al., 2021] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. Loftr: Detector-free local feature matching with transformers. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 8922-8931, 2021.
  37. [Thomee et al., 2016] Bart Thomee, David A Shamma, Ger- ald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfcc100m: The new data in multimedia research. Communications of the ACM, pages 64-73, 2016.
  38. [Xiao et al., 2013] Jianxiong Xiao, Andrew Owens, and An- tonio Torralba. Sun3d: A database of big spaces recon- structed using sfm and object labels. In Proceedings of the IEEE International Conference on Computer Vision, pages 1625-1632, 2013.
  39. [Yang et al., 2024] Changcai Yang, Xiaojie Li, Jiayi Ma, Fengyuan Zhuang, Lifang Wei, Riqing Chen, and Guodong Chen. Cgr-net: Consistency guided resformer for two-view correspondence learning. IEEE Trans- actions on Circuits and Systems for Video Technology, 34(12):12450-12465, 2024.
  40. [Yi et al., 2018] Kwang Moo Yi, Eduard Trulls, Yuki Ono, Vincent Lepetit, Mathieu Salzmann, and Pascal Fua. Learning to find good correspondences. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2666-2674, 2018.
  41. [Zhang and Ma, 2023a] Shihua Zhang and Jiayi Ma. Con- vmatch: Rethinking network design for two-view corre- spondence learning. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 37, pages 3472- 3479, 2023.
  42. [Zhang and Ma, 2023b] Shihua Zhang and Jiayi Ma. Con- vmatch: Rethinking network design for two-view corre- spondence learning. IEEE Transactions on Pattern Analy- sis and Machine Intelligence, 46(5):2920-2935, 2023.
  43. [Zhang et al., 2019] Jiahui Zhang, Dawei Sun, Zixin Luo, Anbang Yao, Lei Zhou, Tianwei Shen, Yurong Chen, Long Quan, and Hongen Liao. Learning two-view correspon- dences and geometry using order-aware network. In Pro- ceedings of the IEEE International Conference on Com- puter Vision, pages 5845-5854, 2019.
  44. [Zhang et al., 2024] Shihua Zhang, Zizhuo Li, Yuan Gao, and Jiayi Ma. Dematch: Deep decomposition of motion field for two-view correspondence learning. In Proceed- ings of the IEEE Conference on Computer Vision and Pat- tern Recognition, pages 20278-20287, 2024.
  45. [Zhao et al., 2021] Chen Zhao, Yixiao Ge, Feng Zhu, Rui Zhao, Hongsheng Li, and Mathieu Salzmann. Progressive correspondence pruning by consensus learning. In Pro- ceedings of the IEEE International Conference on Com- puter Vision, pages 6464-6473, 2021.
  46. [Zheng et al., 2022] Linxin Zheng, Guobao Xiao, Ziwei Shi, Shiping Wang, and Jiayi Ma. Msa-net: Establishing re- liable correspondences by multiscale attention network. IEEE Transactions on Image Processing, pages 4598- 4608, 2022.
  47. [Zhong et al., 2021] Zhen Zhong, Guobao Xiao, Linxin Zheng, Yan Lu, and Jiayi Ma. T-net: Effective permutation-equivariant network for two-view correspon- dence learning. In Proceedings of the IEEE International Conference on Computer Vision, pages 1950-1959, 2021.
  48. [Zhong et al., 2022] Zhen Zhong, Guobao Xiao, Shiping Wang, Leyi Wei, and Xiaoqin Zhang. Pesa-net: Permutation-equivariant split attention network for corre- spondence learning. Information Fusion, pages 81-89, 2022.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards