Listen and Count: Expanding the Frontier of Zero-shot Object Counting to Sound-centric Counting

Listen and Count: Expanding the Frontier of Zero-shot Object Counting to Sound-centric Counting

Mingjie Wang, Zhuohang Li, Qi Zhang, Jun Zhou, Zili Yi, Minglun Gong

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence
Main Track. Pages 1740-1748. https://doi.org/10.24963/ijcai.2026/194

While class-agnostic object counting has recently evolved from image-exemplar to language-guided paradigms, existing methods are limited by text polysemy and the lack of prompts in audio-sensing scenarios. To overcome these challenges, we introduce a sound-centric counting paradigm, enabling models to "listen and count" using the vivid and discriminative signatures of sound cues. We propose S2ICount, which achieves seamless, fine-grained audio-visual alignment for sound-guided counting. At its core, SoundMamba leverages linear-complexity long-range semantic modelling to align cross-modal features, while the Sound Calibration Module designs a circular scanning mechanism to progressively reconcile modality discrepancies using multi-scale cues from a Stable Diffusion backbone. To enhance saliency awareness, a Multi-tier Hybrid Optimization strategy enforces consistency across magnitude, semantics, and distribution. In addition, we introduce SoundCount, the first large-scale sound-oriented counting dataset, comprising over 2,000 scenes and 1,650 audio sources across 66 object categories. Extensive experiments demonstrate that S2ICount achieves state-of-the-art performance, validating the effectiveness of sound as a robust guidance modality for counting. The dataset, demo and implementation code are made publicly accessible at https://github.com/ZSTU-CV-Lab/S2ICount.
Keywords:
Computer Vision: Applications and Systems
Computer Vision: Multimodal learning
Computer Vision: Scene analysis and understanding