Proceedings

Exploiting Label Skewness for Spiking Neural Networks in Federated Learning

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 6895-6903.

https://doi.org/10.24963/ijcai.2025/767

Abstract

The energy efficiency of deep spiking neural networks (SNNs) aligns with the constraints of resource-limited edge devices, positioning SNNs as a promising foundation for intelligent applications leveraging the extensive data collected by these devices. To safeguard data privacy, federated learning (FL) facilitates collaborative SNN-based model training by leveraging data distributed across edge devices without transmitting local data to a central server. However, existing FL approaches encounter challenges in handling label-skewed data across devices, inducing drift in the local SNN model and consequently impairing the performance of the global SNN model. To tackle these problems, we propose a novel framework called FedLEC, which incorporates intra-client label weight calibration to balance the learning intensity across local labels and inter-client knowledge distillation to mitigate local SNN model bias caused by label absence. Extensive experiments with three different structured SNNs across five datasets (i.e., three non-neuromorphic and two neuromorphic datasets) demonstrate the efficiency of FedLEC. Compared to seven state-of-the-art FL algorithms, FedLEC achieves an average accuracy improvement of approximately 11.59% for the global SNN model under various label skew distribution settings.

  1. [Anumasa et al., 2024] Srinivas Anumasa, Bhaskar Mukhoty, Velibor Bojkovic, Giulia De Masi, Huan Xiong, and Bin Gu. Enhancing training of spiking neural network with stochastic latency. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10900-10908, 2024.
  2. [Aouedi and Kandaraj, 2024] Ons Aouedi and Piamrat Kan- daraj. Surfs: Sustainable intrusion detection with hierar- chical federated spiking neural networks. ICC 2024, 2024.
  3. [Aouedi et al., 2023] Ons Aouedi, Kandaraj Piamrat, and Mario Sûdholt. Hfedsnn: Efficient hierarchical federated learning using spiking neural networks. In Proceedings of the Int'l ACM Symposium on Mobility Management and Wireless Access, pages 53-60, 2023.
  4. [Deng et al., 2023] Shikuang Deng, Hao Lin, Yuhang Li, and Shi Gu. Surrogate module learning: Reduce the gradi- ent error accumulation in training spiking neural networks. In International Conference on Machine Learning, pages 7645-7657. PMLR, 2023.
  5. [Di et al., 2024] Yu Di, Du Xin, Linshan Jiang, Wentao Tong, and Shuiguang Deng. Ec-snn: Splitting deep spik- ing neural networks for edge devices. In Proceedings of the Thirty-ThirdInternational Joint Conference on Artifi- cial Intelligence (2024)., volume 271507864, 2024. IJCAI
  6. [Diao et al., 2024] Yiqun Diao, Qinbin Li, and Bingsheng He. Exploiting label skews in federated learning with model concatenation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 38, pages 11784- 11792, 2024.
  7. [El Ouadrhiri and Abdelhadi, 2022] Ahmed El Ouadrhiri and Ahmed Abdelhadi. Differential privacy for deep and federated learning: A survey. IEEE access, 10:22359- 22380, 2022.
  8. [Hu et al., 2024] Yifan Hu, Lei Deng, Yujie Wu, Man Yao, and Guoqi Li. Advancing spiking neural networks toward deep residual learning. IEEE Transactions on Neural Net- works and Learning Systems, 2024.
  9. [Huang, 2005] Jonathan Huang. Maximum likelihood esti- mation of dirichlet distribution parameters. CMU Tech- nique report, 76, 2005.
  10. [Karimireddy et al., 2020] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In Interna- tional conference on machine learning, pages 5132-5143. PMLR, 2020.
  11. [Li and Zhan, 2021] Xin-Chun Li and De-Chuan Zhan. Fe- drs: Federated learning with restricted softmax for label distribution non-iid data. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 995-1005, 2021.
  12. [Li et al., 2019] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. arXiv preprint arXiv:1907.02189, 2019.
  13. [Li et al., 2020] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Pro- ceedings of Machine learning and systems, 2:429-450, 2020.
  14. [Li et al., 2022] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering (ICDE), pages 965-978. IEEE, 2022.
  15. [Lim et al., 2020] Wei Yang Bryan Lim, Nguyen Cong Lu- ong, Dinh Thai Hoang, Yutao Jiao, Ying-Chang Liang, Qiang Yang, Dusit Niyato, and Chunyan Miao. Feder- ated learning in mobile edge networks: A comprehen- sive survey. IEEE communications surveys & tutorials, 22(3):2031-2063, 2020.
  16. [Maass, 1997] Wolfgang Maass. Networks of spiking neu- rons: the third generation of neural network models. Neu- ral networks, 10(9):1659-1671, 1997.
  17. [McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
  18. [Menon et al., 2021] Aditya Krishna Menon, Sadeep Jaya- sumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit ad- justment. In International Conference on Learning Repre- sentations, 2021.
  19. [Su et al., 2023] Qiaoyi Su, Yuhong Chou, Yifan Hu, Jian- ing Li, Shijie Mei, Ziyang Zhang, and Guoqi Li. Deep directly-trained spiking neural networks for object detec- tion. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 6555-6565, 2023.
  20. [Tumpa et al., 2023] Sadia Anjum Tumpa, Sonali Singh, Md Fahim Faysal Khan, Mahmut Tylan Kandemir, Vi- jaykrishnan Narayanan, and Chita R Das. Federated learn- ing with spiking neural networks in heterogeneous sys- tems. In 2023 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pages 1-6. IEEE, 2023.
  21. [Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
  22. [Venkatesha et al., 2021] Yeshwanth Venkatesha, Youngeun Kim, Leandros Tassiulas, and Priyadarshini Panda. Feder- ated learning with spiking neural networks. IEEE Trans- actions on Signal Processing, 69:6183-6194, 2021.
  23. [Wang et al., 2020] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective in- consistency problem in heterogeneous federated optimiza- tion. Advances in neural information processing systems, 33:7611-7623, 2020.
  24. [Wang et al., 2023] Ziming Wang, Runhao Jiang, Shuang Lian, Rui Yan, and Huajin Tang. Adaptive smoothing gradient learning for spiking neural networks. In Inter- national Conference on Machine Learning, pages 35798- 35816. PMLR, 2023.
  25. [Wu et al., 2019] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 1311-1318, 2019.
  26. [Xia et al., 2024] Tong Xia, Abhirup Ghosh, Xinchi Qiu, and Cecilia Mascolo. Flea: Addressing data scarcity and label skew in federated learning via privacy-preserving feature augmentation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3484-3494, 2024.
  27. [Xie et al., 2022] Kan Xie, Zhe Zhang, Bo Li, Jiawen Kang, Dusit Niyato, Shengli Xie, and Yi Wu. Efficient feder- ated learning with spike neural networks for traffic sign recognition. IEEE Transactions on Vehicular Technology, 71(9):9980-9992, 2022.
  28. [Yao et al., 2024] Man Yao, JiaKui Hu, Tianxiang Hu, Yifan Xu, Zhaokun Zhou, Yonghong Tian, Bo XU, and Guoqi Li. Spike-driven transformer v2: Meta spiking neural net- work architecture inspiring the design of next-generation neuromorphic chips. In The Twelfth International Confer- ence on Learning Representations, 2024.
  29. [Yurochkin et al., 2019] Mikhail Yurochkin, Mayank Agar- wal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, and Yasaman Khazaeni. Bayesian nonparametric feder- ated learning of neural networks. In International con- ference on machine learning, pages 7252-7261. PMLR, 2019.
  30. [Zhang et al., 2022] Jie Zhang, Zhiqi Li, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Chao Wu. Federated learning with label distribution skew via logits calibration. In International Conference on Machine Learning, pages 26311-26329. PMLR, 2022.
  31. [Zhang et al., 2024] Mengjun Zhang, Bin Li, Hongfu Liu, and Chenglin Zhao. Federated learning for radar gesture recognition based on spike timing dependent plasticity. IEEE Transactions on Aerospace and Electronic Systems, 2024.
  32. [Zuo et al., 2024] Lin Zuo, Yongqi Ding, Mengmeng Jing, Kunshan Yang, and Yunqian Yu. Self-distillation learning based on temporal-spatial consistency for spiking neural networks. arXiv preprint arXiv:2406.07862, 2024.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards