Anomaly Detection in Network Traffic for Insider Threat Identification: A Comparative Study of Unsupervised and Supervised Machine Learning Approaches
Article Sidebar
Main Article Content
Abstract
Insider threats pose a significant and growing risk to organizational cybersecurity, with recent studies indicating a 47% increase in insider incidents from 2018 to 2022. This paper presents a comparative analysis of unsupervised and supervised machine learning approaches for detecting potential insider threats through network traffic anomaly identification. We develop and evaluate an Isolation Forest (unsupervised) and a Random Forest (supervised) model, training them on a simulated dataset representing six months of network logs from a mid-sized company. Our study introduces a unique feature set combining traditional network metrics with temporal and behavioral indicators, enhancing the models' detection capabilities. Results show that the Random Forest classifier outperforms the Isolation Forest, with F1-scores of 0.6425 and 0.4624, respectively. However, the unsupervised approach shows promise in scenarios lacking labeled data. Key findings reveal that increased connection frequency and data transfer volume are critical indicators of potential threats, with temporal patterns also playing a significant role. This study provides valuable insights into the strengths and limitations of each approach, offering practical implications for real-world digital forensics investigations. We contribute to the field by proposing a hybrid approach that leverages the strengths of both methods, potentially improving the accuracy and adaptability of insider threat detection systems. These findings pave the way for more robust, context-aware cybersecurity measures in the digital age.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
Gurucul, "2023 Insider Threat Report," Cybersecurity Insiders, 2023. [*Online]. Available: https://library.cyentia.com/report/report_014103.html
P. Chattopadhyay, L. Wang, and Y. P. Tan, "Scenario-based insider threat detection from cyber activities," IEEE Trans. Comput. Soc. Syst., vol. 5, no. 3, pp. 660-675, 2018. doi: 10.1109/TCSS.2018.2857473.
Z. Azam, M. M. Islam, and M. N. Huda, "Comparative Analysis of Intrusion Detection Systems and Machine Learning-Based Model Analysis Through Decision Tree," IEEE Access, vol. 11, pp. 80348-80391, 2023. doi: 10.1109/ACCESS.2023.3296444.
G. Apruzzese, M. Colajanni, L. Ferretti, A. Guido, and M. Marchetti, "On the effectiveness of machine and deep learning for cyber security," in Proc. 10th Int. Conf. Cyber Conflict (CyCon), 2018, pp. 371-390. doi: 10.23919/CYCON.2018.8405026.
Y. Xin et al., "Machine learning and deep learning methods for cybersecurity," IEEE Access, vol. 6, pp. 35365-35381, 2018. doi: 10.1109/ACCESS.2018.2836950.
Proofpoint, "Cost of Insider Threats Global Report," Ponemon Institute, 2022. [*Online]. Available: https://www.novipro.com/blog/2022-ponemon-cost-of-insider-threats-global-report
H. Rai, J. Yoo, and S. Agarwal, "The improved network intrusion detection techniques using the feature engineering approach with Boosting Classifiers," Mathematics, vol. 12, no. 24, p. 3909, 2024. doi: 10.3390/math12243909.
Tuor, S. Kaplan, B. Hutchinson, N. Nichols, and S. Robinson, "Deep learning for unsupervised insider threat detection in structured cybersecurity data streams," The AAAI Workshop on Artificial Intelligence for Cyber Security, 2017. doi: 10.48550/arXiv.1710.00811.
L. Chen et al., "Insider threat detection using Deep Neural Networks with Attention Mechanism," IEEE Trans. Netw. Sci. Eng., vol. 9, no. 3, pp. 1695-1707, 2022.
M. Lakshmi et al., "Evaluating the Isolation Forest Method for anomaly detection in software-defined networking security," J. Electr. Syst., vol. 19, no. 4, pp. 279-297, 2023. doi: 10.52783/jes.639.
S. Ness et al., "Anomaly Detection in Network Traffic Using Advanced Machine Learning Techniques," IEEE Access, pp. 1-1, 2025. doi: 10.1109/ACCESS.2025.3526988.
A. Nisioti, G. Loukas, A. Laszka, and E. Panaousis, "Data-Driven decision support for optimizing cyber forensic investigations," IEEE Trans. Inf. Forensics Security, vol. 16, pp. 2397-2412, 2021. doi: 10.1109/TIFS.2021.3054966.
A. Shamshari and H. Najaf, "Machine learning approaches for anomaly detection in network security," Eastern Eur. J. Multidiscip. Res., vol. 1, no. 1, pp. 22–29, 2024.
E. F. Agyemang, "Anomaly detection using unsupervised machine learning algorithms: A simulation study," Sci. Afr., vol. 26, e02386, 2024. doi: 10.1016/j.sciaf.2024.e02386.
L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001. doi: 10.1023/A:1010933404324.
R. Pandey, M. Pandey, and A. Nazarov, "Advancing Network Anomaly Detection: Comparative Analysis of Machine Learning Models," in Cryptology and Network Security with Machine Learning, A. Chaturvedi et al., Eds., Springer, vol. 918, 2024. doi: 10.1007/978-981-97-0641-9_41.
M. C. Aswathy, and T. Rajkumar, "Real Time Anomaly Detection in Network Traffic: A Comparative Analysis of Machine Learning Algorithms," Int. Res. J. Adv. Eng. Hub, vol. 2, pp. 1968-1977, 2024. doi: 10.47392/IRJAEH.2024.0269.
L. I. Khalaf et al., "Deep Learning-Based Anomaly Detection in Network Traffic for Cyber Threat Identification," in Proc. Cognitive Models and Artificial Intelligence Conf. (AICCONF '24), pp. 303-309, 2024. doi: 10.1145/3660853.3660932.
V. P. M. Vishnu Priya and S. Soumya, "Advancements in Anomaly Detection Techniques in Network Traffic: The Role of Artificial Intelligence and Machine Learning," J. Sci. Res. Technol., vol. 2, no. 6, pp. 38-48, 2024. doi: 10.61808/jsrt114.
P. Gupta and P. Tripathy, "Unsupervised Learning for Real-Time Data Anomaly Detection: A Comprehensive Approach," Int. J. Comput. Sci. Eng., vol. 11, pp. 1-11, 2024. doi: 10.14445/23488387/IJCSE-V11I10P101.
R. Liu, J. Shi, X. Chen, and C. Lu, "Network anomaly detection and security defense technology based on machine learning: A review," Comput. Electr. Eng., vol. 119, p. 109581, 2024. doi: 10.1016/j.compeleceng.2024.109581.
J. Ahmad and A. W. Khan, “Empirical investigation of security awareness and training for distributed teams to safeguard from cyber attacks,” in Computing and Data Science, S.-C. Haw, L. Lee, M. M. Alam, A. Khan, M. Z. Asghar, and F. U. Khan, Eds. MMU Press, 2024, pp. 63–75.