Proceedings

DeepSTE: Deep Spectral Temporal Embeddings for Dynamic Graph Representation Learning

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 2671-2679.

https://doi.org/10.24963/ijcai.2026/297

Abstract

Temporal embeddings play a crucial role in dynamic graph neural networks (DGNNs) by capturing the temporal dynamics of interactions. However, existing Random Fourier Feature (RFF)-based methods in DGNNs directly sample Fourier frequencies from a fixed, data-independent distribution, neglecting the temporal characteristics of dynamic graphs and thereby limiting representational capacity. We propose DeepSTE, a deep spectral temporal embedding framework for dynamic graphs. DeepSTE learns RFF representations via Monte Carlo importance sampling with a tractable proposal distribution (e.g., Gaussian) to approximate the feature map of a shift-invariant or positive-definite kernel whose latent spectral density is analytically intractable. DeepSTE adopts a data-dependent scale parameter, estimated from interaction intervals, to construct the frequency proposal distribution reflecting time–frequency uncertainty. The frequency DNN and the importance-weighting DNN, initialized from the proposal distribution, are jointly optimized to model the importance-sampled spectral representation and learn adaptive temporal features. Experiments demonstrate the effectiveness of DeepSTE on dynamic link prediction and node classification tasks, while also revealing insights such as temporal embedding decay and accelerated convergence.

  1. [Cho et al., 2014] Kyunghyun Cho, Bart Van Merriënboer, C ¸aglar Gulc ¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of the 2014 confer- ence on empirical methods in natural language processing (EMNLP), pages 1724-1734, 2014.
  2. [Chung et al., 2025] Hsing-Huan Chung, Shravan Chaud- hari, Xing Han, Yoav Wald, Suchi Saria, and Joydeep Ghosh. Between linear and sinusoidal: Rethinking the time encoder in dynamic graph learning. arXiv preprint arXiv:2504.08129, 2025.
  3. [Cortes et al., 2010] Corinna Cortes, Mehryar Mohri, and Ameet Talwalkar. On the impact of kernel approxima- tion on learning accuracy. In Proceedings of the thir- teenth international conference on artificial intelligence and statistics, pages 113-120. JMLR Workshop and Con- ference Proceedings, 2010.
  4. [Feng et al., 2025] ZhengZhao Feng, Rui Wang, TianXing Wang, Mingli Song, Sai Wu, and Shuibing He. A compre- hensive survey of dynamic graph neural networks: Mod- els, frameworks, benchmarks, experiments and challenges. IEEE Transactions on Knowledge and Data Engineering, 2025.
  5. [Hornik et al., 1989] Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359-366, 1989.
  6. [Huang et al., 2023] Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael Bronstein, Guillaume Rabusseau, and Reihaneh Rabbany. Temporal graph benchmark for machine learning on temporal graphs. Advances in Neural Information Processing Systems, 36:2056-2073, 2023.
  7. [Huang et al., 2025] Xuanwen Huang, Wei Chow, Yize Zhu, Yang Wang, Ziwei Chai, Chunping Wang, Lei Chen, and Yang Yang. Enhancing cross-domain link prediction via evolution process modeling. In Proceedings of the ACM on Web Conference 2025, pages 2158-2171, 2025.
  8. [Jin et al., 2022] Ming Jin, Yuan-Fang Li, and Shirui Pan. Neural temporal walks: Motif-aware representation learn- ing on continuous-time dynamic graphs. Advances in Neural Information Processing Systems, 35:19874-19886, 2022.
  9. [Joachims, 2006] Thorsten Joachims. Training linear svms in linear time. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 217-226, 2006.
  10. [Kazemi et al., 2020] Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart. Representation learning for dynamic graphs: A survey. Journal of Machine Learning Research, 21(70):1-73, 2020.
  11. [Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  12. [Kumar et al., 2019] Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD international conference on knowl- edge discovery & data mining, pages 1269-1278, 2019.
  13. [Li et al., 2019] Yanjun Li, Kai Zhang, Jun Wang, and San- jiv Kumar. Learning adaptive random features. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 33, pages 4229-4236, 2019.
  14. [Lu et al., 2017] Zhou Lu, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei Wang. The expressive power of neural networks: A view from the width. Advances in neu- ral information processing systems, 30, 2017.
  15. [Lu et al., 2022] Mingxuan Lu, Zhichao Han, Susie Xi Rao, Zitao Zhang, Yang Zhao, Yinan Shan, Ramesh Raghu- nathan, Ce Zhang, and Jiawei Jiang. Bright-graph neural networks in real-time fraud detection. In Proceedings of the 31st ACM international conference on information & knowledge management, pages 3342-3351, 2022.
  16. [Pareja et al., 2020] Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kaneza- shi, Tim Kaler, Tao Schardl, and Charles Leiserson. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5363-5370, 2020.
  17. [Qian et al., 2024] Hao Qian, Hongting Zhou, Qian Zhao, Hao Chen, Hongxiang Yao, Jingwei Wang, Ziqi Liu, Fei Yu, Zhiqiang Zhang, and Jun Zhou. Mdgnn: Multi- relational dynamic graph neural network for comprehen- sive and dynamic stock investment prediction. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 38, pages 14642-14650, 2024.
  18. [Rahimi and Recht, 2007] Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Ad- vances in neural information processing systems, 20, 2007.
  19. [Rossi et al., 2020] Emanuele Rossi, Ben Chamberlain, Fab- rizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arXiv preprint arXiv:2006.10637, 2020.
  20. [Si et al., 2017] Si Si, Cho-Jui Hsieh, and Inderjit S Dhillon. Memory efficient kernel approximation. Journal of Ma- chine Learning Research, 18(20):1-32, 2017.
  21. [Trivedi et al., 2019] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learn- ing representations over dynamic graphs. In International conference on learning representations, 2019.
  22. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  23. [Veličković et al., 2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017.
  24. [Wang et al., 2021a] Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia, Qi Yang, Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang, Bowen Sun, et al. Apan: Asyn- chronous propagation attention network for real-time tem- poral graph embedding. In Proceedings of the 2021 inter- national conference on management of data, pages 2628- 2638, 2021.
  25. [Wang et al., 2021b] Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representa- tion learning in temporal networks via causal anonymous walks. In International Conference on Learning Represen- tations (ICLR), 2021.
  26. [Wu et al., 2024] Yuxia Wu, Yuan Fang, and Lizi Liao. On the feasibility of simple transformer for dynamic graph modeling. In Proceedings of the ACM web conference 2024, pages 870-880, 2024.
  27. [Xu et al., 2019] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Self-attention with functional time representation learning. Advances in neu- ral information processing systems, 32, 2019.
  28. [Xu et al., 2020] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive repre- sentation learning on temporal graphs. arXiv preprint arXiv:2002.07962, 2020.
  29. [Xu et al., 2021] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. A temporal kernel approach for deep learning with continuous-time informa- tion. arXiv preprint arXiv:2103.15213, 2021.
  30. [Ying et al., 2018] Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web- scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 974-983, 2018.
  31. [Yu et al., 2023] Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architec- ture and unified library. Advances in Neural Information Processing Systems, 36:67686-67700, 2023.
  32. [Yuan et al., 2025] Qi Yuan, Yang Liu, Yateng Tang, Xin- huan Chen, Xuehao Zheng, Qing He, and Xiang Ao. Dy- namic graph learning with static relations for credit risk assessment. In Proceedings of the AAAI conference on ar- tificial intelligence, volume 39, pages 13133-13141, 2025.
  33. [Zhang et al., 2022] Mengqi Zhang, Shu Wu, Xueli Yu, Qiang Liu, and Liang Wang. Dynamic graph neural net- works for sequential recommendation. IEEE Transactions on Knowledge and Data Engineering, 35(5):4741-4753, 2022.
  34. [Zhou et al., 2022] Hongkuan Zhou, Da Zheng, Israt Nisa, Vasileios Ioannidis, Xiang Song, and George Karypis. Tgl: a general framework for temporal gnn training on billion-scale graphs. Proceedings of the VLDB Endow- ment, 15(8):1572-1580, 2022.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards