Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

Deep Optical Flow Estimation Via Multi-Scale Correspondence Structure Learning

Shanshan Zhao, Xi Li, Omar El Farouk Bourahla

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 3490-3496. https://doi.org/10.24963/ijcai.2017/488

As an important and challenging problem in computer vision, learning based optical flow estimation aims to discover the intrinsic correspondence structure between two adjacent video frames through statistical learning. Therefore, a key issue to solve in this area is how to effectively model the multi-scale correspondence structure properties in an adaptive end-to-end learning fashion. Motivated by this observation, we propose an end-to-end multi-scale correspondence structure learning (MSCSL) approach for optical flow estimation. In principle, the proposed MSCSL approach is capable of effectively capturing the multi-scale inter-image-correlation correspondence structures within a multi-level feature space from deep learning. Moreover, the proposed MSCSL approach builds a spatial Conv-GRU neural network model to adaptively model the intrinsic dependency relationships among these multi-scale correspondence structures. Finally, the above procedures for correspondence structure learning and multi-scale dependency modeling are implemented in a unified end-to-end deep learning framework. Experimental results on several benchmark datasets demonstrate the effectiveness of the proposed approach.
Keywords:
Machine Learning: Deep Learning
Robotics and Vision: Vision and Perception