Fine-tuning Deep Neural Networks by Interactively Refining the 2D Latent Space of Ambiguous Images
Fine-tuning Deep Neural Networks by Interactively Refining the 2D Latent Space of Ambiguous Images
Jiafu Wei, Haoran Xie, Chia-Ming Chang, Xi Yang
Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Demo Track. Pages 5948-5951.
https://doi.org/10.24963/ijcai.2022/861
Deep neural networks (DNNs) have achieved excellent results currently in classification, while they may still suffer from ambiguous images which are similar across classes. By contrast, humans have a relatively good ability to distinguish these categories of images. Therefore, we propose a human-in-the-loop solution to assist the network to better classify the images by leveraging human knowledge. To achieve this, we project the high-dimensional latent space trained by the network onto a two-dimensional workspace. The users can interactively modify the projected coordinates of inputs on the workspace using our designed tools, then the modified information will be fed back to the network to fine-tune it, which in turn affects the network's classification results, thereby improving the accuracy of network classification.
Keywords:
Humans and AI: Human-Computer Interaction
Humans and AI: Human-AI Collaboration