Real-Time Emotion Detection Using Artificial Intelligence: A Review
Article Sidebar
Main Article Content
Abstract
The integration of artificial intelligence (AI) in emotion recognition has significantly transformed human-computer interaction and revolutionized fields such as medicine, education, and entertainment. This paper reviews 30 papers on the detection of emotional signs through various biometric inputs, including electroencephalography (EEG), electrocardiography (ECG), facial expressions, and speech patterns. Despite advancements in AI-driven emotion recognition systems, challenges persist, particularly in data variability, computational inefficiency, and ethical dilemmas associated with privacy, security, and algorithmic bias. Recent innovations in feature extraction techniques, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have enhanced the precision of emotional state recognition across multiple input channels. The transition to edge computing has further enabled real-time processing with low latency, facilitating integration into wearable devices and IoT ecosystems. Multimodal systems, which leverage data sources such as physiological signals, facial expressions, and speech, show great promise but face challenges related to inclusivity and system fragility. To address these issues, the study recommends for robust training datasets, ethical guidelines, and hardware optimizations. Incorporating contextual information and accounting for individual differences can improve recognition accuracy and user trust. However, ethical concerns remain critical, emphasizing the need for strict standards of privacy, security, and equitable access to ensure AI emotion recognition systems are trustworthy and inclusive. Overall, this paper highlights the potential of AI-driven emotion recognition systems while underscoring the importance of continuous research to address technical and ethical challenges, paving the way for broader applications in pattern recognition, cognitive studies, and specialized tools.
Manuscript received:4 Jan 2025 | Revised: 13 Feb 2025 | Accepted: 27 Feb 2025 | Published: 31 Mar 2025
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
W. Fang, K. Wang, N. Fahier, Y. Ho and Y. Huang, "Development and Validation of an EEG-Based Real-Time Emotion Recognition System Using Edge AI Computing Platform With Convolutional Neural Network System-on-Chip Design," IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 4, pp. 645-657, 2019.
DOI: https://doi.org/10.1109/JETCAS.2019.2951232
A. Hassouneh, A. Mutawa and M. Murugappan, "Development of a Real-Time Emotion Recognition System Using Facial Expressions and EEG based on machine learning and deep neural network methods," Informatics in Medicine Unlocked, vol. 20, pp. 100372, 2020.
DOI: https://doi.org/10.1016/j.imu.2020.100372
S. Turabzadeh, H. Meng, R. Swash, M. Pleva and J. Juhar, "Facial Expression Emotion Detection for Real-Time Embedded Systems," Technologies, vol. 6, no. 1, pp. 17, 2018.
DOI: https://doi.org/10.3390/technologies6010017
S. Jaiswal and G.C. Nandi, "Robust real-time emotion detection system using CNN architecture," Neural Computing and Applications, vol. 32, no. 15, pp. 11253-11262, 2020.
DOI: https://doi.org/10.1007/s00521-019-04564-4
A. R. M., B. M., D.-J. M., and J. M. M. Kartali, "Real-time algorithms for facial emotion recognition: A comparison of different approaches," 14th Symposium on Neural Networks and Applications (NEUREL), pp. 1-6. 2018.
DOI: https://doi.org/10.1109/NEUREL.2018.8587011
S. Gupta, P. Kumar and R.K. Tekchandani, "Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models," Multimedia Tools and Applications, vol. 82, no. 8, pp. 11365-11394, 2023.
DOI: https://doi.org/10.1007/s11042-022-13558-9
M. Oki, S. Akintola, A. Adediran, P. Ikubanni and B.T. Ogunsemi, "Reinforcement bar Corrosion - Causes and Management," Journal of Physics: Conference Series, vol. 1378, no. 2, pp. 022095, 2019.
DOI: https://doi.org/10.1088/1742-6596/1378/2/022095
S. Gupta, "Facial emotion recognition in real-time and static images," 2nd International Conference on Inventive Systems and Control (ICISC), pp. 108, 2018.
DOI: https://doi.org/10.1109/ICISC.2018.8399046
M. Awais, M. Raza, N. Singh, K. Bashir, U. Manzoor, S.U. Islam and J.J.P.C. Rodrigues, "LSTM-Based Emotion Detection Using Physiological Signals: IoT Framework for Healthcare and Distance Learning in COVID-19," IEEE Internet of Things Journal, vol. 8, no. 23, pp. 16863-16871, 2021.
DOI: https://doi.org/10.1109/JIOT.2020.3044031
A. R., A. K., and D. S. Jaiswal, "Facial emotion detection using deep learning," International Conference for Emerging Technology (INCET), pp. 1-6, 2020.
DOI: https://doi.org/10.1109/INCET49848.2020.9154121
K. Zhang, Y. Li, J. Wang, E. Cambria and X. Li, "Real-Time Video Emotion Recognition Based on Reinforcement Learning and Domain Knowledge," IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 1034-1047, 2022.
DOI: https://doi.org/10.1109/TCSVT.2021.3072412
F. M. Talaat, "Real-time facial emotion recognition system among children with autism based on deep learning and IoT," Neural Computing and Applications, vol. 35, no. 2, pp. 1817–1830, 2023.
DOI: https://doi.org/10.1007/s00521-023-08372-9
Y. Jiang, W. Li, M. S. Hossain, M. Chen, A. Alelaiwi, and M. Al-Hammadi, "A snapshot research and implementation of multimodal information fusion for data-driven emotion recognition," Information Fusion, vol. 53, pp. 55–62, 2020.
DOI: https://doi.org/10.1016/j.inffus.2019.06.003
K. Bahreini, W.v.d. Vegt and W. Westera, "A fuzzy logic approach to reliable real-time recognition of facial emotions," Multimedia Tools and Applications, vol. 78, no. 14, pp. 18943-18966, 2019.
DOI: https://doi.org/10.1007/s11042-019-7250-z
M.A. Helal, A.I. Chowdhury, A. Islam, E. Ahmed, M.S. Mahmud and S. Hossain, "An Optimization Approach to Improve Classification Performance in Cancer and Diabetes Prediction," 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE), 2019.
DOI: https://doi.org/10.1109/ECACE.2019.8679413
M.A. Ozdemir, B. Elagoz, A. Alaybeyoglu, R. Sadighzadeh and A. Akan, "Real Time Emotion Recognition from Facial Expressions Using CNN Architecture," 2019 Medical Technologies Congress (TIPTEKNO), pp. 1-4, 2019.
DOI: https://doi.org/10.1109/TIPTEKNO.2019.8895215
L. Zahara, P. Musa, E.P. Wibowo, I. Karim and S.B. Musa, "The Facial Emotion Recognition (FER-2013) Dataset for Prediction System of Micro-Expressions Face Using the Convolutional Neural Network (CNN) Algorithm based Raspberry Pi," 2020 Fifth International Conference on Informatics and Computing (ICIC), pp. 1-9, 2020.
DOI: https://doi.org/10.1109/ICIC50835.2020.9288560
E. Pranav, S. Kamal, C.S. Chandran and M. Supriya, "Facial Emotion Recognition Using Deep Convolutional Neural Network," 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), 2020.
DOI: https://doi.org/10.1109/ICACCS48705.2020.9074302
A.I. Siam, N.F. Soliman, A.D. Algarni, F.E.A. El-Samie and A. Sedik, "Deploying Machine Learning Techniques for Human Emotion Detection," Computational Intelligence and Neuroscience, vol. 2022, pp. 1-16, 2022.
DOI: https://doi.org/10.1155/2022%2F8032673
K. S. Mustaqeem, "Deep feature-based speech emotion recognition using the hierarchical ConvLSTM network," Mathematics, vol. 8, no. 8, 2020.
DOI: https://doi.org/10.3390/math8081328
P. M., J. B., D. K., P. N. C. S. D., and C. P. B. Ghate, "Optimized intelligent speech signal verification system for identifying authorized users," Engineering & Applied Science Research, vol. 50, no. 3, pp. 233–245, 2023.
DOI: https://doi.org/10.14456/easr.2023.20
S. R. Shahamiri, “An optimized enhanced-multi learner approach towards speaker identification based on single-sound segments,” Multimedia Tools and Applications, vol. 83, pp. 24541–24562, 2024.
DOI: https://doi.org/10.1007/s11042-023-16507-2
M.J.A. Nahian, T. Ghosh, M.N. Uddin, M.M. Islam, M. Mahmud and M.S. Kaiser, "Towards Artificial Intelligence Driven Emotion Aware Fall Monitoring Framework Suitable for Elderly People with Neurological Disorder," Lecture Notes in Computer Science, pp. 275-286, 2020.
DOI: https://doi.org/10.1007/978-3-030-59277-6_25
K. S. Mustaqeem, "A CNN-assisted enhanced audio signal processing for speech emotion recognition," Sensors, vol. 19, no. 23, 2019.
DOI: https://doi.org/10.3390/s19235080
V. Vielzeuf, C. Kervadec, S. Pateux, A. Lechervy and F. Jurie, "An Occam's Razor View on Learning Audiovisual Emotion Recognition with Small Training Sets," Proceedings of the 20th ACM International Conference on Multimodal Interaction, pp. 589-593, 2018.
DOI: https://doi.org/10.1145/3242969.3264980
K. S. Mustaqeem, "Optimal feature selection based speech emotion recognition using two-stream deep convolutional neural network," International Journal of Intelligent Systems, vol. 36, no. 2, pp. 708–731, 2021.
DOI: https://doi.org/10.1002/int.22254
P. Mohan, L. Sharma and P. Narayan, "Optimal Path Finding using Iterative SARSA," 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), pp. 811-817, 2021.
DOI: https://doi.org/10.1109/ICICCS51141.2021.9432202
I. Sergi, T. Montanaro, F.L. Benvenuto and L. Patrono, "A Smart and Secure Logistics System Based on IoT and Cloud Technologies," Sensors, vol. 21, no. 6, pp. 2231, 2021.
DOI: https://doi.org/10.3390/s21062231
R. Venkatesan, S. Shirly, M. Selvarathi and T.J. Jebaseeli, "Human Emotion Detection Using DeepFace and Artificial Intelligence," RAiSE-2023, pp. 37, 2023.
DOI: https://doi.org/10.3390/engproc2023059037
J. Chen, K. Yu, F. Wang, Z. Zhou, Y. Bi, S. Zhuang and D. Zhang, "Temporal Convolutional Network-Enhanced Real-Time Implicit Emotion Recognition with an Innovative Wearable fNIRS-EEG Dual-Modal System," Electronics, vol. 13, no. 7, pp. 1310, 2024.
DOI: https://doi.org/10.3390/electronics13071310
Y. Xu, Y. Lin, X. Zhou and X. Shan, "Utilizing emotion recognition technology to enhance user experience in real-time," Computing and Artificial Intelligence, vol. 2, no. 1, pp. 1388, 2024.
DOI: https://doi.org/10.59400/cai.v2i1.1388
B. Bakariya, A. Singh, H. Singh, P. Raju, R. Rajpoot and K.K. Mohbey, "Facial emotion recognition and music recommendation system using CNN-based deep learning techniques," Evolving Systems, vol. 15, no. 2, pp. 641-658, 2024.
DOI: https://doi.org/10.1007/s12530-023-09506-z
C. Suneetha and R. Anitha, "Advancements in Speech-Based Emotion Recognition and PTSD Detection through Machine and Deep Learning Techniques: A Comprehensive Survey," International Journal of Electronics and Communication Engineering, vol. 11, no. 5, pp. 220-234, 2024.
DOI: https://doi.org/10.14445/23488549/IJECE-V11I5P121
H.B.U. Haq, W. Akram, M.N. Irshad, A. Kosar and M. Abid, "Enhanced Real-Time Facial Expression Recognition Using Deep Learning," Acadlore Transactions on AI and Machine Learning, vol. 3, no. 1, pp. 24-35, 2024.
DOI: https://doi.org/10.56578/ataiml030103
A. Mutawa and A. Hassouneh, "Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods," Biomedical Signal Processing and Control, vol. 91, pp. 105942, 2024.
DOI: https://doi.org/10.1016/j.bspc.2023.105942
M.S. Rao, A. Kiran, S.L. Naik, A.P. Kumar, K.K. Swamy and P. Devika, "Facial Emotion Recognition with Convolutional Neural Networks using DL," 2024 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC), pp. 1-5, 2024.