Acoustic NLOS Imaging with Cross Modal Knowledge Distillation

Acoustic NLOS Imaging with Cross Modal Knowledge Distillation

Ui-Hyeon Shin, Seungwoo Jang, Kwangsu Kim

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence
Main Track. Pages 1405-1413. https://doi.org/10.24963/ijcai.2023/156

Acoustic non-line-of-sight (NLOS) imaging aims to reconstruct hidden scenes by analyzing reflections of acoustic waves. Despite recent developments in the field, existing methods still have limitations such as sensitivity to noise in a physical model and difficulty in reconstructing unseen objects in a deep learning model. To address these limitations, we propose a novel cross-modal knowledge distillation (CMKD) approach for acoustic NLOS imaging. Our method transfers knowledge from a well-trained image network to an audio network, effectively combining the strengths of both modalities. As a result, it is robust to noise and superior in reconstructing unseen objects. Additionally, we evaluate real-world datasets and demonstrate that the proposed method outperforms state-of-the-art methods in acoustic NLOS imaging. The experimental results indicate that CMKD is an effective solution for addressing the limitations of current acoustic NLOS imaging methods. Our code, model, and data are available at https://github.com/shineh96/Acoustic-NLOS-CMKD.
Keywords:
Computer Vision: CV: Applications
Computer Vision: CV: Neural generative models, auto encoders, GANs  
Machine Learning: ML: Multi-modal learning