Current era is to make the connection amongst humans and their manufactured accomplices (Computers) and make communication more reliable and easier. One of the real challenges is the utilization of speech recognition. Speech recognition can be improved by visual information of human face. Visual speech recognition (Lip reading) assumes a fundamental part in automatic speech recognition and is an essential stride towards exact and robust speech recognition. In this paper, the technique is developed for visual speech recognition in detail. Optical Flow component is used to extract the feature vector and Artificial Neural Networks (ANN) for training. The effect of variation in velocity of speaking on the execution of the system is minimized by eliminating the zero energy frames and normalizing the number of frames. The efficiency of both approaches (Optical Flow and ANN) is used to evaluate words individually. Considered words are numerical numbers in Indian language (Hindi) from zero to nine, such as ek, do, theen, and so on.

Visual speech recognition, lip localization, Optical Flow, ANN, Indian Language

[1] Bor-Shing Lin, Yu-Hsien Yao, Ching-Feng Liu, Ching-Feng Lien, Bor-Shyh Lin, “Development of Novel Lip-reading Recognition Algorithm”, IEEE Access, vol. 5, no.1 , pp. 794-801, 2017.
[2] Jun Shiraishi, Takeshi Saitoh, “Optical Flow based Lip Reading using Non-Rectangular ROI and Head Motion Reduction”, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, pp. 1-6, 2015.
[3] Ahmad B. A. Hassanat, “Visual Passwords Using Automatic Lip Reading”, IJSBAR, Vol.13, No.1, pp.218-231, 2013.
[4] SS. Morade, “Suprava Patnaik Lip Reading Using DWT and LSDA”, IEEE International Advance Computing Conference, India, pp.1000-1003, 2014.
[5] WR. Butt, L. Lombardi, “A Survey of Automatic Lip Reading Approaches”, Eighth International Conference on Digital Information Management (ICDIM 2013), Islamabad, pp. 299-302, 2013.
[6] SS. Morade, B.S. Patnaik, “Automatic Lip Tracking and Extraction of Lip Geometric Features for Lip Reading”, International Journal of Machine Learning and Computing, Vol. 3, No. 2, pp.23-30, 2013
[7] JI. Newman, SJ. Cox, “language identification using visual features”, IEEE transactions on audio speech and language processing, vol. 20, no.7, pp. 1936-1947, 2012.
[8] AA. Shaikh, DK. Kumar, WC. Yau, M. Z. Che Azemin, J. Gubbi, “Lip Reading using Optical Flow and Support Vector Machines”, 3rd International Congress on Image and Signal Processing (CISP2010), India, pp.327-330, 2010.
[9] W.C. Yau, D.K. Kumar, S.P. Arjunan, “Visual speech recognition using dynamic features and support vector machines, International Journal of Image and Graphics, vol.8, Issue.3, pp. 419-437, 2008.
[10] Salah Werda, Walid Mahdi, Abdel Majid, “Lip Localization and Viseme Classification for Visual Speech Recognition”, International Journal of Computing & Information Sciences, Vol.5, No.1, pp.67-75, 2007.
[11] X. Hong, H. Yao, Y. Wan, R. Chen, “A PCA based visual DCT feature extraction method for lip-reading”, in Proc. Int. Conf. Intell Inf. Hiding Multimedia Signal Process, CA, pp. 321-326, 2006.
[12] T. Chen, R.R. Rao, "Audio-Visual Integration in Multimodal Communication", Special Issue on Multimedia Signal Processing, IEEE Proceedings, vol. 86, Issue.5, pp. 837-852, 1998.