A Dense Optical Flow Field Estimation with Variational Refinement

Article Sidebar

Fulltext
Published: Dec 17, 2019
DOI: https://doi.org/10.33093/jetap.2019.1.2.30
Keywords:
Optical flow, variational formulation, dense estimation, Markov Random Field, flow initialization

Main Article Content

Wei Yong Eng
Faculty of Engineering and Technology, Multimedia University
Yang Lang Chang
National Taipei University of Technology, Taiwan
Tien Sze Lim
Faculty of Engineering and Technology, Multimedia University
Voon Chet Koo
Faculty of Engineering and Technology, Multimedia University

Abstract

Optical flow has long been a focus of research study in computer vision community. Researchers have established extensive work to solve the optical flow estimation. Among the published works, a notable work using variational energy minimization has been a baseline of optical flow estimation for a long time. Variational optical flow optimization solves an approximate global minimum in a well-defined nonlinear Markov Energy formulation. It works by first linearizing the energy model and uses a numerical method specifically successive over-relaxation (SOR) method to solve the resulting linear model. An initialization scheme is required for optical flow field in this iterative optimization method. In the original work, a zero initialization is proposed and it works well on the various environments with photometric and geometric distortion. In this work, we have experimented with different flow field initialization scheme under various environment setting. We found out that variational refinement with a good initial flow estimate using state-of-art optical flow algorithms can further improve its accuracy performance.

Article Details

Issue
Vol. 1 No. 2 (2019): https://doi.org/10.33093/jetap.2019.1.2
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

References

[1] B. D. Lucas and T. Kanade, “An Iterative Image Registration Technique with An Application to Stereo Vision,”, in Proc. of the 7th Int. Joint Conf. on Artificial Intelligence, vol. 2, pp. 674-679, 1981.
[2] B. K. Horn and B. G. Schunck, “Determining Optical Flow,” Artificial Intelligence, vol. 17, no. 1-3, pp. 185-203, 1981. [3] C. Zach, T. Pock and H. Bischof, “A Duality based Approach for Realtime TV-L 1 Optical Flow,” in Joint Pattern Recognition Symp., Springer, Berlin, pp. 214-223, 2007.
[4] J. Y. Bouguet, “Pyramidal Implementation of The Affine Lucas Kanade Feature Tracker Description of The Algorithm,” Intel Corporation, vol. 5, no. 1-10, 4, 2001.
[5] T. Brox, A. Bruhn, N. Papenberg and J. Weickert, “High Accuracy Optical Flow Estimation based on A Theory for Warping,” in European Conf. on Computer Vision, Springer, Berlin, pp. 25-36 2004.
[6] D. G. Lowe, “Object Recognition from Local Scale-invariant Features,” in Proc. of the Int. Conf. on Computer Vision, vol. 99, no. 2, pp. 1150-1157, 1999.
[7] H. Bay, A. Ess, T. Tuytelaars and L. Van Gool, “Speeded-up Robust Features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346-359, 2008.
[8] E. Tola, V. Lepetit and P. Fua, “Daisy: An Efficient Dense Descriptor Applied to Wide-baseline Stereo,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, 2009.
[9] C. Liu, J. Yuen and A. Torralba, “Sift flow: Dense Correspondence Across Scenes and Its Applications,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 978-994, 2010.
[10] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” in Proc. of the Int. Conf. on Computer Vision, vol. 98, no. 1, pp. 839-846, 1998.
[11] K. He, J. Sun and X. Tang, “Guided Image Filtering,” in European Conf. on Computer Vision, pp. 1-14, Springer, Berlin, 2010.
[12] M. J. Black and P. Anandan, “The Robust Estimation of Multiple Motions: Parametric and Piecewise-smooth Flow Fields,” Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[13] T. Brox and J. Malik, “Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 500-513, 2010.
[14] C. Vogel, K. Schindler and S. Roth, “Piecewise Rigid Scene Flow,” in Proc. of the IEEE Int. Conf. on Computer Vision, pp. 1377-1384, 2013.
[15] Y. LeCun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-based Learning Applied to Document Recognition,” Proc. of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[16] A. Krizhevsky, L. Sutskever and G. E. Hinton, “Imagenet Classification with Deep Convolutional Neural Networks,” in Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.
[17] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-scale Image Recognition,” Computer Vision and Pattern Recognition, preprint arXiv:1409.1556, 2014.
[18] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov and A. Rabinovich, “Going Deeper with Convolutions,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1-9, 2015.
[19] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
[20] P. Weinzaepfel, J. Revaud, Z. Harchaoui and C. Schmid, “DeepFlow: Large Displacement Optical Flow with Deep Matching,” in Proc. of the IEEE Int. Conf. on Computer Vision, pp. 1385-1392, 2013.
[21] W. C. Ma, S. Wang, R. Hu, Y. Xiong and R. Urtasun, “Deep Rigid Instance Scene Flow,” Computer Vision and Pattern Recognition, preprint arXiv:1904.08913, 2019.
[22] M. Menze and A. Geiger, “Object Scene Flow for Autonomous Vehicles,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 3061-3070, 2015.
[23] E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy and T. Brox, “Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks,” in Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 2462-2470, 2017.
[24] K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky and L. Van Gool, “A Comparison of Affine Region Detectors,” Int. J. of Computer Vision, vol. 65, no. 1-2, pp. 43-72, 2005.