Video Frame Interpolation Based on Deformable Kernel Region

Video Frame Interpolation Based on Deformable Kernel Region

Haoyue Tian, Pan Gao, Xiaojiang Peng

Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence
Main Track. Pages 1349-1355. https://doi.org/10.24963/ijcai.2022/188

Video frame interpolation task has recently become more and more prevalent in the computer vision field. At present, a number of researches based on deep learning have achieved great success. Most of them are either based on optical flow information, or interpolation kernel, or a combination of these two methods. However, these methods have ignored that there are grid restrictions on the position of kernel region during synthesizing each target pixel. These limitations result in that they cannot well adapt to the irregularity of object shape and uncertainty of motion, which may lead to irrelevant reference pixels used for interpolation. In order to solve this problem, we revisit the deformable convolution for video interpolation, which can break the fixed grid restrictions on the kernel region, making the distribution of reference points more suitable for the shape of the object, and thus warp a more accurate interpolation frame. Experiments are conducted on four datasets to demonstrate the superior performance of the proposed model in comparison to the state-of-the-art alternatives.
Keywords:
Computer Vision: Machine Learning for Vision
Computer Vision: Video analysis and understanding