A Human-in-the-Loop Tool for Annotating Passive Acoustic Monitoring Datasets

A Human-in-the-Loop Tool for Annotating Passive Acoustic Monitoring Datasets

Hannes Kath, Thiago S. GouvĂȘa, Daniel Sonntag

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence
Demo Track. Pages 7140-7144. https://doi.org/10.24963/ijcai.2023/835

Deep learning methods are well suited for data analysis in several domains, but application is often limited by technical entry barriers and the availability of large annotated datasets. We present an interactive machine learning tool for annotating passive acoustic monitoring datasets created for wildlife monitoring, which are time-consuming and costly to annotate manually. The tool, designed as a web application, consists of an interactive user interface implementing a human-in-the-loop workflow. Class label annotations provided manually as bounding boxes drawn over a spectrogram are consumed by a deep generative model (DGM) that learns a low-dimensional representation of the input data, as well as the available class labels. The learned low-dimensional representation is displayed as an interactive interface element, where new bounding boxes can be efficiently generated by the user with lasso-selection; alternatively, the DGM can propose new, automatically generated bounding boxes on demand. The user can accept, edit, or reject annotations suggested by the model, thus owning final judgement. Generated annotations can be used to fine-tune the underlying model, thus closing the loop. Investigations of the prediction accuracy and first empirical experiments show promising results on an artificial data set, laying the ground for application to a real life scenario.
Keywords:
Machine Learning: ML: Feature extraction, selection and dimensionality reduction
Multidisciplinary Topics and Applications: MDA: Energy, environment and sustainability
Machine Learning: ML: Autoencoders
Machine Learning: ML: Incremental learning
Machine Learning: ML: Representation learning
Machine Learning: ML: Classification
Machine Learning: ML: Active learning