Scribble-to-Painting Transformation with Multi-Task Generative Adversarial Networks

Scribble-to-Painting Transformation with Multi-Task Generative Adversarial Networks

Jinning Li, Yexiang Xue

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence
AI for Improving Human Well-being. Pages 5916-5922. https://doi.org/10.24963/ijcai.2019/820

We propose the Dual Scribble-to-Painting Network (DSP-Net), which is able to produce artistic paintings based on user-generated scribbles. In scribble-to-painting transformation, a neural net has to infer additional details of the image, given relatively sparse information contained in the outlines of the scribble. Therefore, it is more challenging than classical image style transfer, in which the information content is reduced from photos to paintings. Inspired by the human cognitive process, we propose a multi-task generative adversarial network, which consists of two jointly trained neural nets -- one for generating artistic images and the other one for semantic segmentation. We demonstrate that joint training on these two tasks brings in additional benefit. Experimental result shows that DSP-Net outperforms state-of-the-art models both visually and quantitatively. In addition, we publish a large dataset for scribble-to-painting transformation.
Keywords:
Special Track on AI for Improving Human-Well Being: Entertainment;games applications (Special Track on AI and Human Wellbeing)