Quranic Malaysian Sign Language Detection Using Yolo for Real-Time Translation
Article Sidebar
Main Article Content
Abstract
This study presents the development of a real-time Quranic Malaysian Sign Language (MySL) recognition system designed to enhance accessibility to Quranic teachings for the deaf and hard-of-hearing community. Despite the critical necessity for inclusive religious education, the deaf Muslim community faces a profound technological deficit. There is a severe lack of automated systems capable of accurately translating the unique, linguistically complex, and culturally sensitive gestures of Quranic Sign Language in real-time. By leveraging deep learning, the system uses the YOLO (You Only Look Once) object detection model combined with TensorFlow and OpenCV to detect hand gestures representing Quranic letters. A Quranic Sentence Builder was also developed to convert gesture sequences into meaningful sentences. The system achieved high performance, recording 96.54% precision, 95.72% recall, and a 95.26% F1-score. The findings demonstrate the potential of deep learning in supporting inclusive religious education through accurate and efficient sign language translation.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
[1] F. M. Talaat et al., "Real-time Arabic Avatar for Deaf-mute Communication Enabled by Deep Learning Sign Language Translation," Comput. and Electr. Eng., vol. 119, Part A, pp. 109475, 2024.
[2] M. A. Mahmod and A. Zeki, "Quranic Sign Language for Deaf People: Quranic Recitation Classification and Verification," Int. J. Percept. and Cognit. Comput., vol. 4, no. 1, pp. 7–11, 2018.
[3] F. S. Alamri, A. Rehman, S. B. Abdullahi and T. Saba, "Intelligent Real-life Key-pixel Image Detection System for Early Arabic Sign Language Learners," Peer J. Comput. Sci., vol. 10, pp. e2063, 2024.
[4] J. Han et al., "You Only Look Once–Aluminum: A Detection Model for Complex Aluminum Surface Defects Based on Improved YOLOv8," Symmetry, vol. 17, no. 5, pp. 724, 2025.
[5] H. A. AbdElghfar et al., "A Model for Qur’anic Sign Language Recognition Based on Deep Learning Algorithms," J. Sensors, vol. 2023, no. 1, pp. 9926245, 2023.
[6] R. B. Rafiq, S. A. Hakim and T. Tabashum, "Real-time Vision-based Bangla Sign Language Detection using Convolutional Neural Network," in 2021 Int. Conf. Adv. Comput. and Commun., Kochi, Kakkanad, India, pp. 1–5, 2021.
[7] K. Sinha, A. O. Miranda and S. Mishra, "Real-time Sign Language Translator," in Cognit. Informat. and Soft Comput.: Proc. CISC 2021, Springer Nature Singapore, pp. 477–489, 2022.
[8] A. Wadhawan and P. Kumar, "Sign Language Recognition Systems: A Decade Systematic Literature Review," Arch. Computat. Meth. Eng., vol. 28, no. 3, pp. 785–813, 2021.
[9] P. Verma and K. Badli, "Real-Time Sign Language Detection using Tensor Flow, Open CV and Python," Int. J. Res. Appl. Sci. Eng. Technol., vol. 10, no. 5, pp. 4483–4488, 2022.
[10] S. Srivastava, A. Gangwar, R. Mishra and S. Singh, "Sign Language Recognition System using TensorFlow Object Detection API," in Int. Conf. Adv. Netw. Technol. and Intell. Comput., Cham: Springer, pp. 634–646, 2021.
[11] G. Tharwat, A. M. Ahmed and B. Bouallegue, "Arabic Sign Language Recognition System for Alphabets Using Machine Learning Techniques," J. Electr. Comput. Eng., vol. 2021, pp. 2995851, 2021.
[12] B. J. Erickson and F. Kitamura, "Performance Metrics for Machine Learning Models," Radiol. Artif. Intell., vol. 3, no. 3, pp. e200126, 2021.
[13] C. Luna-Jiménez, M. Gil-Martín, R. Kleinlein, R. San-Segundo and F. Fernández-Martínez, "Interpreting Sign Language Recognition using Transformers and Mediapipe Landmarks," in Proc. 25th Int. Conf. Multimodal Interact., Paris, France, pp. 373–377, 2023.
[14] A. H. Althubiti and H. Algethami, "Dynamic Gesture Recognition using A Transformer and Mediapipe," Int. J. Adv. Comput. Sci. and Appl., vol. 15, no. 6, pp. 1424-1439, 2024.
[15] C. Artamma and M. Rahardi, "L2IC and MobileViT-XXS for BISINDO Alphabet Recognition," J. Appl. Informat. and Comput., vol. 9, no. 6, pp. 3410–3418, 2025.
[16] "Comparing YOLOv8, SSD, and Faster-RCNN for Real-time Object Detection," ReadyTensor AI. [Available Online on 20 April 2026]
[17] "YOLOv8 vs Faster R-CNN: A Comparative Analysis," Keylabs AI Blog. [Available Online on 20 April 2026] https://keylabs.ai/blog/yolov8-vs-faster-r-cnn-a-comparative-analysis/.