Proceedings

Where Does This Data Come From? Enhanced Source Inference Attacks in Federated Learning

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 4815-4823.

https://doi.org/10.24963/ijcai.2025/536

Abstract

Federated learning (FL) enables collaborative model training without exposing raw data, offering a privacy-aware alternative to centralized learning. However, FL remains vulnerable to various privacy attacks that exploit shared model updates, including membership inference, property inference, and gradient inversion. Source inference attacks further threaten FL by identifying which client contributed a specific training sample, posing severe risks to user and institutional privacy. Existing source inference attacks mainly assume passive adversaries and overlook more realistic scenarios where the server actively manipulates the training process. In this paper, we present an enhanced source inference attack that demonstrates how a malicious server can amplify behavioral differences between clients to more accurately infer data origin. Our approach introduces active training manipulation and data augmentation to expose client-specific patterns. Experimental results across five representative FL algorithms and multiple datasets show that our method significantly outperforms prior passive attacks. These findings reveal a deeper level of privacy vulnerability in FL and call for stronger defense mechanisms under active threat models.

  1. [Arivazhagan et al., 2019] Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818, 2019.
  2. [Bagdasaryan et al., 2020] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pages 2938-2948. PMLR, 2020.
  3. [Bonawitz et al., 2019] Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečnỳ, Stefano Mazzocchi, Brendan McMahan, et al. Towards federated learning at scale: System design. Proceedings of Machine Learning and Systems, 1:374-388, 2019.
  4. [Caldas et al., 2018] Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, H Bren- dan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018.
  5. [Devakumar et al., 2020] Delan Devakumar, Geordan Shan- non, Sunil S Bhopal, and Ibrahim Abubakar. Racism and discrimination in covid-19 responses. The Lancet, 395(10231):1194, 2020.
  6. [Durmus et al., 2021] Alp Emre Durmus, Zhao Yue, Matas Ramon, Mattina Matthew, Whatmough Paul, and Saligrama Venkatesh. Federated learning based on dy- namic regularization. In International conference on learning representations, 2021.
  7. [Feng et al., 2024] Bingdao Feng, Di Jin, Xiaobao Wang, Fangyu Cheng, and Siqi Guo. Backdoor attacks on unsu- pervised graph representation learning. Neural Networks, 180:106668, 2024.
  8. [Geiping et al., 2020] Jonas Geiping, Hartmut Bauermeis- ter, Hannah Dröge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in federated learn- ing? Advances in neural information processing systems, 33:16937-16947, 2020.
  9. [Hu et al., 2021] Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, and Xuyun Zhang. Source inference at- tacks in federated learning. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1102-1107. IEEE, 2021.
  10. [Hu et al., 2023] Hongsheng Hu, Xuyun Zhang, Zoran Sal- cic, Lichao Sun, Kim-Kwang Raymond Choo, and Gillian Dobbie. Source inference attacks: Beyond membership in- ference attacks in federated learning. IEEE Transactions on Dependable and Secure Computing, 21(4):3012-3029, 2023.
  11. [Jin et al., 2025] Di Jin, Yujun Zhang, Bingdao Feng, Xi- aobao Wang, Dongxiao He, and Zhen Wang. Backdoor attack on propagation-based rumor detectors. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 39, pages 17680-17688, 2025.
  12. [Karimireddy et al., 2020] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebas- tian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, pages 5132-5143. PMLR, 2020.
  13. [Khodak et al., 2021] Mikhail Khodak, Renbo Tu, Tian Li, Liam Li, Maria-Florina F Balcan, Virginia Smith, and Ameet Talwalkar. Federated hyperparameter tuning: Challenges, baselines, and connections to weight-sharing. Advances in Neural Information Processing Systems, 34:19184-19197, 2021.
  14. [Kumar et al., 2023] Kummari Naveen Kumar, Chalavadi Krishna Mohan, and Linga Reddy Cenkera- maddi. The impact of adversarial attacks on federated learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2672-2691, 2023.
  15. [Li and Wang, 2019] Daliang Li and Junpu Wang. Fedmd: Heterogenous federated learning via model distillation. arXiv preprint arXiv:1910.03581, 2019.
  16. [Li et al., 2020a] Tian Li, Anit Kumar Sahu, Ameet Tal- walkar, and Virginia Smith. Federated learning: Chal- lenges, methods, and future directions. IEEE Signal Pro- cessing Magazine, 37(3):50-60, 2020.
  17. [Li et al., 2020b] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Pro- ceedings of Machine Learning and Systems, 2:429-450, 2020.
  18. [Li et al., 2021a] Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He. A survey on federated learning systems: vision, hype and reality for data privacy and protection. IEEE Transactions on Knowledge and Data Engineering, 2021.
  19. [Li et al., 2021b] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learn- ing through personalization. In International conference on machine learning, pages 6357-6368. PMLR, 2021.
  20. [Li et al., 2025] Jiaxin Li, Marco Arazzi, Antonino Nocera, and Mauro Conti. Subject data auditing via source infer- ence attack in cross-silo federated learning. Journal of In- formation Security and Applications, 90:104034, 2025.
  21. [Lyu et al., 2020] Lingjuan Lyu, Han Yu, and Qiang Yang. Threats to federated learning: A survey. arXiv preprint arXiv:2003.02133, 2020.
  22. [Lyu et al., 2022] Lingjuan Lyu, Han Yu, Xingjun Ma, Chen Chen, Lichao Sun, Jun Zhao, Qiang Yang, and S Yu Philip. Privacy and robustness in federated learning: At- tacks and defenses. IEEE transactions on neural networks and learning systems, 2022.
  23. [McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics, pages 1273-1282.
  24. PMLR, 2017.
  25. [Melis et al., 2019] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unin- tended feature leakage in collaborative learning. In 2019 IEEE Symposium on Security and Privacy (S&P), pages 691-706. IEEE, 2019.
  26. [Nasr et al., 2019] Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (S&P), pages 739- 753. IEEE, 2019.
  27. [Shokri et al., 2017] Reza Shokri, Marco Stronati, Con- gzheng Song, and Vitaly Shmatikov. Membership infer- ence attacks against machine learning models. In 2017 IEEE symposium on security and privacy (S&P), pages 3- 18. IEEE, 2017.
  28. [Smith et al., 2017] Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. Federated multi-task learning. Advances in neural information processing systems, 30, 2017.
  29. [Tolpegin et al., 2020] Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu. Data poisoning attacks against federated learning systems. In Computer security-ESORICs 2020: 25th European symposium on research in computer security, ESORICs 2020, guildford, UK, September 14-18, 2020, proceedings, part i 25, pages 480-501. Springer, 2020.
  30. [Wang et al., 2020] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective in- consistency problem in heterogeneous federated optimiza- tion. Advances in neural information processing systems, 33:7611-7623, 2020.
  31. [Wang et al., 2022] Zhibo Wang, Yuting Huang, Mengkai Song, Libing Wu, Feng Xue, and Kui Ren. Poisoning- assisted property inference attack against federated learn- ing. IEEE Transactions on Dependable and Secure Com- puting, 2022.
  32. [Wei et al., 2025] Jiaheng Wei, Yanjun Zhang, Leo Yu Zhang, Chao Chen, Shirui Pan, Kok-Leong Ong, Jun Zhang, and Yang Xiang. Extracting private training data in federated learning from clients. IEEE Transactions on Information Forensics and Security, 2025.
  33. [Xie et al., 2022] Yuanyuan Xie, Bing Chen, Jiale Zhang, and Wenjuan Li. Algans: Enhancing membership infer- ence attacks in federated learning with gans and active learning. In 2022 IEEE International Symposium on Prod- uct Compliance Engineering-Asia (ISPCE-ASIA), pages 1-6. IEEE, 2022.
  34. [Xu et al., 2020] Yongchao Xu, Liya Ma, Fan Yang, Yanyan Chen, Ke Ma, Jiehua Yang, Xian Yang, Yaobing Chen, Chang Shu, Ziwei Fan, et al. A collaborative online ai engine for ct-based covid-19 diagnosis. medRxiv, 2020.
  35. [Zhang and Xia, 2024] Lening Zhang and Hui Xia. Active source inference attack based on label-flipping in federated learning. In 2024 IEEE 23rd International Conference on Trust, Security and Privacy in Computing and Communi- cations (TrustCom), pages 1675-1680. IEEE, 2024.
  36. [Zhao et al., 2020] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610, 2020.
  37. [Zhou et al., 2022] Chunyi Zhou, Yansong Gao, Anmin Fu, Kai Chen, Zhiyang Dai, Zhi Zhang, Minhui Xue, and Yuqing Zhang. Ppa: Preference profiling attack against federated learning. In Network and Distributed Systems Security Symposium 2022. Internet Society, 2022.
  38. [Zhu et al., 2019] Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in Neural Infor- mation Processing Systems, 32, 2019.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards