Face and Facial Expressions Recognition System for Blind People Using ResNet50 Architecture and CNN
Article Sidebar
Main Article Content
Abstract
Many blind individuals have difficulties in recognizing people’s facial expression which may impact their social interaction. With the recognition, the blind individuals can accurately interpret and respond to the emotions. There is a lack in the existing application with the combination of face and facial expressions recognition. The blind individuals have to rely on multiple applications to accomplish the same task, making it difficult and time-consuming for them to use. The paper aims to recognize faces and facial expressions for blind individuals and provides feedback in real-time. Three face detection algorithms of Haar Cascade Classifier, Dlib, and RetinaFace are compared. Dlib is chosen to process with Histogram of Oriented Gradients (HOG) and Support Vector Machine (SVM). It loads the pre-trained model, computes the HOG features, slide the window scanning at different scales, classify the windows using the SVM classifier, generate bounding boxes, and applying non-maximum suppression. ResNet50 architecture is employed to recognize face and Convolutional Neural Networks (CNN) is applied to recognize facial expression. The training accuracy is 70% and validation accuracy is 60%.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
B. Islam, F. Mahmud, and A. Hossain, “High performance facial expression recognition system using facial region segmentation, fusion of HOG and LBP features and multiclass SVM”, International Conference on Electrical and Computer Engineering (ICECE),pp. 42-45, 2018. doi: 10.1109/ICECE.2018.8636780.
J. L. Joseph and S. P. Mathew, “Facial expression recognition for the blind using deep learning”, International conference on computing, power and communication technologies (GUCON), pp. 1-5, 2021. doi: 10.1109/GUCON50781.2021.9574035.
S. Y. Lin, Y. W. Tseng, C. R. Wu, Y. C. Kung, Y. Z. Chen, and C. M. Wu, “A continuous facial expression recognition model based on deep learning method”, International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), pp. 1-2, 2019. doi: 10.1109/ISPACS48206.2019.8986360.
D. Liu, H. Zhang, and P. Zhou, “Video-based facial expression recognition using graph convolutional networks”, International Conference on Pattern Recognition (ICPR), pp. 607-614, 2021. doi: 10.1109/ICPR48806.2021.9413094.
S. R. Supta, M. R. Sahriar, M. G. Rashed, D. Das, and R. Yasmin, “An Effective Facial Expression Recognition System”, International Women in Engineering (WIE) Conference on Electrical and Computer Engineering (WIECON-ECE), pp. 66-69, 2020. doi: 10.1109/WIECON-ECE52138.2020.9397965.
X. Xu, Z. Ruan, and L. Yang, “Facial expression recognition based on graph neural network”, International Conference on Image, Vision and Computing (ICIVC), pp. 211-214, 2020. doi: 10.1109/ICIVC50857.2020.9177430.
S. Ullah, A. Jan, and G. M. Khan, “Facial expression recognition using machine learning techniques”, International Conference on Engineering and Emerging Technologies (ICEET), pp. 1-6, 2021. doi: 10.1109/ICEET53442.2021.9659631.
J. Zou, X. Cao, S. Zhang, and B. Ge, “A facial expression recognition based on improved convolutional neural network”, International Conference of Intelligent Applied Systems on Engineering (ICIASE), pp. 301-304, 2019. doi: 10.1109/ICIASE45644.2019.9074074.
D. Wang, H. Yu, D. Wang, and G. Li, “Face recognition system based on CNN”, International Conference on Computer Information and Big Data Applications (CIBDA), pp. 470-473, 2020. doi: 10.1109/CIBDA50819.2020.00111.
B. T. Chinimilli, T. Anjali, A. Kotturi, V. R. Kaipu, and J. V. Mandapati, “Face recognition based attendance system using Haar cascade and local binary pattern histogram algorithm”, International conference on trends in electronics and informatics (ICOEI)(48184), pp. 701-704), 2020. doi: 10.1109/ICOEI48184.2020.9143046.
L. Fu and X. Shao, “Research and implementation of face detection, tracking and recognition based on video”, International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS), pp. 914-917, 2020. doi: 10.1109/ICITBS49701.2020.00202.
M. G. Sarwar, A. Dey, and A. Das, “Developing a LBPH-based face recognition system for visually impaired people”, International Conference on Artificial Intelligence and Data Analytics (CAIDA),pp. 286-289, 2021. doi: 10.1109/CAIDA51941.2021.9425275.
A. U. Naik and N. Guinde, “LBPH algorithm for frontal and side profile face recognition on GPU”, International Conference on Smart Systems and Inventive Technology (ICSSIT), pp. 776-779, 2020. doi: 10.1109/ICSSIT48917.2020.9214228.
R. R. Flores and M. J. Domínguez, “Real time automatic face recognition system using LBPH technique”, CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON), pp. 1-6, 2021. doi: 10.1109/CHILECON54041.2021.9702960.
X. Wang, L. Liu, and S. Yang, “An Improved Face Recognition Algorithm based on Fisherface and Machine Learning”, International Conference on Machine Learning and Computer Application (ICMLCA), pp. 1-4, 2021.
R. Rosnelly, M. S. Simanjuntak, A. C. Sitepu, M. Azhari, and S. Kosasi, “Face recognition using eigenface algorithm on laptop camera”, International Conference on Cyber and IT Service Management (CITSM), pp. 1-4, 2020. doi: 10.1109/CITSM50537.2020.9268907.
P. Mohite, K. Vaibhav, and P. K. Annapurani, “Thermal and Visual Face Recognition using Eigenfaces and Transfer Learning”, International Conference on Applied Artificial Intelligence and Computing (ICAAIC, pp. 656-662, 2022. doi: 10.1109/ICAAIC53929.2022.9792914.
E. Winarno, I. H. Al Amin, H. Februariyanti, P. W. Adi, W. Hadikurniawati, and M. T. Anwar, “Attendance system based on face recognition system using cnn-pca method and real-time camera”, International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), pp. 301-304, 2019. doi: 10.1109/ISRITI48646.2019.9034596.
Y. Lim, K. W. Ng, P. Naveen, and S. C. Haw, “Emotion Recognition by Facial Expression and Voice: Review and Analysis”, Journal of Informatics and Web Engineering, vol. 1, no. 2, pp. 45-54, 2022. https://doi.org/10.33093/jiwe.2022.1.2.4
M. Xin, L. W. Ang, and S. Palaniappan, “A Data Augmented Method for Plant Disease Leaf Image Recognition based on Enhanced GAN Model Network”, Journal of Informatics and Web Engineering, vol. 2, no. 1, pp. 1-12, 2023. https://doi.org/10.33093/jiwe.2023.2.1.1