Dress like an Internet Celebrity: Fashion Retrieval in Videos

Dress like an Internet Celebrity: Fashion Retrieval in Videos

Hongrui Zhao, Jin Yu, Yanan Li, Donghui Wang, Jie Liu, Hongxia Yang, Fei Wu

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Main track. Pages 1054-1060. https://doi.org/10.24963/ijcai.2020/147

Nowadays, both online shopping and video sharing have grown exponentially. Although internet celebrities in videos are ideal exhibition for fashion corporations to sell their products, audiences do not always know where to buy fashion products in videos, which is a cross-domain problem called video-to-shop. In this paper, we propose a novel deep neural network, called Detect, Pick, and Retrieval Network (DPRNet), to break the gap between fashion products from videos and audiences. For the video side, we have modified the traditional object detector, which automatically picks out the best object proposals for every commodity in videos without duplication, to promote the performance of the video-to-shop task. For the fashion retrieval side, a simple but effective multi-task loss network obtains new state-of-the-art results on DeepFashion. Extensive experiments conducted on a new large-scale cross-domain video-to-shop dataset shows that DPRNet is efficient and outperforms the state-of-the-art methods on video-to-shop task.
Keywords:
Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation
Multidisciplinary Topics and Applications: Recommender Systems
Machine Learning: Recommender Systems
Data Mining: Mining Text, Web, Social Media