Adaptive Gradient Learning for Spiking Neural Networks by Exploiting Membrane Potential Dynamics
Adaptive Gradient Learning for Spiking Neural Networks by Exploiting Membrane Potential Dynamics
Jiaqiang Jiang, Lei Wang, Runhao Jiang, Jing Fan, Rui Yan
Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence
Main Track. Pages 4164-4172.
https://doi.org/10.24963/ijcai.2025/464
Recent advancements have focused on directly training high-performance spiking neural networks (SNNs) by estimating the approximate gradients of spiking activity through a continuous function with constant sharpness, known as surrogate gradient (SG) learning. However, as spikes propagate within neurons and among layers, the distribution of membrane potential dynamics (MPD) will deviate from the gradient-available interval of fixed SG, hindering SNNs from searching the optimal solution space. To maintain the stability of gradient flows, SG needs to align with evolving MPD. Here, we propose a novel adaptive gradient learning for SNNs by exploiting MPD, namely MPD-AGL. It fully accounts for the underlying factors contributing to membrane potential shifts and establishes a dynamic association between SG and MPD at different timesteps to relax gradient estimation, which provides a new degree of freedom for SG learning. Experimental results demonstrate that our method achieves excellent performance at low latency. Moreover, it increases the proportion of neurons that fall into the gradient-available interval compared to fixed SG, effectively mitigating the gradient vanishing problem. Code is available at https://github.com/jqjiang1999/MPD-AGL.
Keywords:
Humans and AI: HAI: Cognitive modeling
Humans and AI: HAI: Cognitive systems
