In this work, we aim to detect and classify different dynamic textures representing scenes of outdoor and indoor environments from video sequences. These scenes constitute the vast majority of events in the world, and their detection offers a wide range of applications. Optical flow is one of the most popular methods for motion estimation due to its efficiency and low computational cost. It is based on the brightness constancy assumption, which assumes a constant brightness of the objects between each two frames over time. However, this assumption is not always verified for dynamic textures with non-uniform surface brightness, due to reflections, shadows, transparency or material diffusion. As an alternative, we propose a new flow estimation method based on texture constancy assumption, which describes the spatial texture components motion. The spatial texture of each point of the image, computed using the LBP operator, is assumed to be constant over time. The resulting flow is called texture flow. From its velocity vectors, we extract the magnitude and orientation, which we combine with the texture spatial features to form a shallow hybrid spatiotemporal descriptor. Experimental results on a benchmark database demonstrate both the ability of our method to distinguish between different types of dynamic textures, and its stability with respect to inter and intra-class differences.

Dynamic texture, Flow estimation, Motion analysis, Texture constancy, Spatiotemporal descriptors

[1] D. Chetverikov and R. Péteri, "A brief survey of dynamic texture description and recognition," In: Computer Recognition Systems. Springer, p. 17–26, 2005.
[2]G. Doretto, E. Jones and S. Soatto, "Spatially homogeneous dynamic textures," In: Computer Vision-ECCV 2004. Springer, pp. 591-602, 2004.
[3] P. Turaga, R. Chellappa and A. Veeraraghavan, "Advances in video-based human activity analysis: challenges and approaches," Advances in Computers, vol. 80, pp. 237-290, 2010.
[4] R. Péteri, S. Fazekas and Mark J. Huiskes, "DynTex: A comprehensive database of dynamic textures," Pattern Recognition Letters, vol. 31, Issue. 12, pp. 1627-1632, 2010.
[5]F. Yang, G.-S. Xia, G. Liu, L. Zhang and X. Huang, "Dynamic texture recognition by aggregating spatial and temporal features via ensemble {SVMs}," Neurocomputing, vol. 173, pp. 1310-1321, 2016.
[6]G. Doretto, A. Chiuso, Y. Wu and S. Soatto, "Dynamic Textures," International Journal of Computer Vision, vol. 51, Issue. 2, pp. 91-109, 2003.
[7] A. Chan and N. Vasconcelos, "Modeling, Clustering, and Segmenting Video with Mixtures of Dynamic Textures," IEEE transactions on pattern analysis and machine intelligence, vol. 30, Issue. 5, pp. 909-926, 2008.
[8] T. Ojala, M. Pietikainen and T. Maenpaa, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns," Pattern Anal. Mach. Intell., vol. 24, Issue. 7, p. 971–987, 2002.
[9] O. Lahdenoja, J. Poikonen and M. Laiho, "Towards understanding the formation of uniform local binary patterns," ISRN Machine Vision, 2013.
[10] Z. Guo, L. Zhang and D. Zhang, "A completed modeling of local binary pattern operator for texture classi?cation," IEEE Transactions on Image Processing, vol. 19, Issue. 6, p. 1657–1663, 2010.
[11] D. Tiwari and V. Tyagi, "Dynamic texture recognition based on completed volume local binary pattern," Multidimensional Systems and Signal Processing, vol. 27, pp. 563-575, 2016.
[12]L. Liu, L. Zhao, Y. Long, G. Kuang and P. Fieguth, "Extended local binary patterns for texture classification," Image and Vision Computing, vol. 30, p. 86–99, 2012.
[13] R. Maurya, "Melanoma Detection Using Modified Extended LBP," International Journal of Computer Sciences and Engineering, vol. 6, Issue. 7, pp. 698-703, 2018.
[14]G. Zhao and M. Pietikäinen, "Dynamic texture recognition using volume local binary patterns," In: Dynamic Vision. Springer, pp. 165-177, 2007.
[15]G. Zhao and M. Pietikäinen, "Dynamic texture recognition using local binary patterns with an application to facial expressions," IEEE transactions on pattern analysis and machine intelligence, vol. 29, Issue. 6, pp. 915-928, 2007.
[16] C.-H. Peh and L.-F. Cheong, "Synergizing spatial and temporal texture," IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, pp. 1179-1191, 2002.
[17]R. Péteri and D. Chetverikov, "Dynamic Texture Recognition Using Normal Flow and Texture Regularity," Iberian Conference on Pattern Recognition and Image Analysis, pp. 223-230, 2005.
[18] S. Fazekas and D. Chetverikov, "Dynamic texture recognition using optical flow features and temporal periodicity," In: 2007 International Workshop on Content-Based Multimedia Indexing, pp. 25-32, 2007.
[19] S. Fazekas, T. Amiaz, D. Chetverikov and N. Kiryati, "Dynamic Texture Detection Based on Motion Analysis," International Journal of Computer Vision, vol. 82, pp. 48-63, 2009.
[20]J. Chen, G. Zhao, M. Salo, E. Rahtu and M. Pietikainen, "Automatic dynamic texture segmentation using local descriptors and optical flow," IEEE transactions on image processing: a publication of the IEEE Signal Processing Society, vol. 22, Issue. 1, pp. 326-339, 2013.
[21] R. Chaudhry, A. Ravichandran, G. Hager and R. Vidal, "Histograms of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions," In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1932-1939, 2009.
[22]J. Pers, V. Kenk, M. Kristan, M. Perše, K. Polanec and S. Kova?i?, "Histograms of optical flow for efficient representation of body motion," Pattern Recognition Letters, vol. 31, pp. 1369-1376, 2010.
[23] S. S and A. E. Mendonca, "Unusual Events Detection via Global Optical Flow and SVM," International Journal of Computer Sciences and Engineering, vol. 4, Issue. 3, pp. 179-183, 2016.
[24]V. Kaltsa, K. Avgerinakis, A. Briassouli, I. Kompatsiaris and M. Strintzis, "Dynamic texture recognition and localization in machine vision for outdoor environments," Computers in Industry, vol. 98, 2018.
[25] B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision (IJCAI)," Proceedings of Imaging Understanding Workshop, pp. 121-130, 1981.
[26] N. Papenberg, A. Bruhn, T. Brox, S. Didas and J. Weickert, "Highly Accurate Optic Flow Computation with Theoretically Justified Warping," International Journal of Computer Vision, vol. 67, pp. 141-158, 2006.
[27] Y. Mileva, A. Bruhn and J. Weickert, "Illumination-Robust Variational Optical Flow with Photometric Invariants," Pattern Recognition, pp. 152-162, 2007.
[28]P. Mettes, R. T. Tan and R. C. Veltkamp, "Water detection through spatio-temporal invariant descriptors," Computer Vision and Image Understanding, vol. 154, pp. 182-191, 2017.