Predictive Test Case Prioritization in Agile Workflows: A Machine Learning Approach with SMOTE-ENN and PCA
Article Sidebar
Main Article Content
Abstract
Test Case Prioritization (TCP) is an essential part of agile software development, allowing for the identification and execution of critical test cases early in the process to maximize fault detection efficiency. Traditional TCP techniques often depend on static metrics, like code complexity and historical pass/fail rates, that do not conform with the dynamic nature of Agile workflows. To overcome this, the proposed research presents a discriminative machine learning-based predictive prioritization model that incorporates historical defect data, execution history, as well as requirement criticality to improve auspiciousness of TCP. The study assesses a range of machine learning models like Logistic Regression, Decision Trees, Random Forest, and Gradient Boosting across two datasets, namely CarFinTest and MISPriority. Though the target variable in the MISPriority dataset was nicely structured with respect to the feature set, allowing models to reach a predictive accuracy up to 78.63%, the CarFinTest showed a high degree of imbalance in classes along with poor class separability. Through iterative improvements, the addition of SMOTE-ENN to balance classes, dimensionality reduction via PCA, and feature engineering techniques all greatly increased model performance, with Random Forest and Gradient Boosting performing the best. In addition, the proposed research presents an automatic dashboard for preprocessing dataset, training models, and prioritizing them for implementation into an Agile environment for practical usability. Through adaptive machine learning-based techniques, this research connects traditional and dynamic test prioritization methods, realizing a scalable and data-oriented approach compatible with Agile processes. Future work should focus on enhancing real-time prioritization through incremental learning techniques and expanding dataset generalization across diverse software development projects.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
[1] I. Ibriwesh, S.-B. Ho, I. Chai, and C.-H. Tan, “A controlled experiment on comparison of data perspectives for software requirements documentation,” Arabian Journal for Science and Engineering, vol. 42, no. 8, pp. 3175–3189, Feb. 2017, doi: 10.1007/s13369-017-2425-2.
[2] D. Marijan, “Comparative study of machine learning test case prioritization for continuous integration testing,” Software Quality Journal, vol. 31, no. 4, pp. 1415–1438, Dec. 2023, doi: 10.1007/s11219-023-09646-0.
[3] M. Mahdieh, S. H. Mirian-Hosseinabadi, and M. Mahdieh, “Test case prioritization using test case diversification and fault-proneness estimations,” Automated Software Engineering, vol. 29, no. 2, Nov. 2022, doi: 10.1007/s10515-022-00344-y.
[4] H. Salin and Y. Rybarczyk, “Decision-Making in Agile Software Engineering: A Systematic Literature Review of Models, Methods, Actors and Lifecycle Contexts,” Software, vol. 5, no. 2, p. 17, Apr. 2026, doi: 10.3390/software5020017.
[5] N. Tasneem, H. B. Zulzalil, and S. Hassan, “Enhancing Agile Software Development: A Systematic Literature Review of Requirement Prioritization and Reprioritization Techniques,” IEEE Access, vol. 13, pp. 32993–33034, 2025, doi: 10.1109/ACCESS.2025.3539357.
[6] P. G. Obike, V. E. Ekong, and O. U. Obot, “A Novel Scoring-based Agile Sprint Deliverability Prediction and Prioritization using Machine Learning,” IJCA, vol. 187, no. 49, pp. 40–53, Oct. 2025, doi: 10.5120/ijca20259258
[7] N. A. Goyal, “Driving Continuous Improvement in Engineering Projects with AI-Enhanced Agile Testing and Machine Learning,” International Journal of Advanced Research in Science Communication and Technology, pp. 1320–1331, Jul. 2023, doi: 10.48175/ijarsct-14000t.
[8] S. Hudda, “Prioritization of Product Backlog Items in Agile Development Using Rank Ordering and Weighting Method,” in Complex, Intelligent and Software Intensive Systems, vol. 260, L. Barolli, T. Enokido, and I. Woungang, Eds., Cham: Springer Nature Switzerland, 2025, pp. 218–230. doi: 10.1007/978-3-031-96099-4_21.
[9] A. M. Radwan, M. A. Abdel-Fattah, and W. Mohamed, “AI-Driven Prioritization Techniques of Requirements in Agile Methodologies: A Systematic Literature Review,” ijacsa, vol. 15, no. 9, 2024, doi: 10.14569/IJACSA.2024.0150983.
[10] J. C. Pang, F. Y. Lee, L. H. Chin, P. L. Ng, and S. Y. Te, “E-Learning Instructional Management For Students with Intellectual Disabilities,” IJSE, vol. 39, no. 1, pp. 1–10, Apr. 2024, doi: 10.52291/ijse.2024.39.1.
[11] J. C. Pang, F. Y. Lee, L. H. Chin, P. L. Ng, and S. Y. Te, “Developing Work Aspiration in Students with Learning Disabilities in Malaysia,” APJEE, vol. 40, no. 1, 299–316 (2025). doi: 10.21315/apjee2025.40.1.15.
[12] J. R. Romero, A. Ramírez, and C. García-Martínez, “Automated machine learning for test case prioritisation,” Empir Software Eng, vol. 31, no. 5, p. 120, Sep. 2026, doi: 10.1007/s10664-026-10862-y.
[13] A. Zarrad, T. Armstrong, and J. Jemai, “A Hybrid Approach to Software Testing Efficiency: Stacked Ensembles and Deep Q-Learning for Test Case Prioritization and Ranking,” CMC, vol. 0, no. 0, pp. 1–10, 2025, doi: 10.32604/cmc.2025.072768.
[14] R. Haque, S.-B. Ho, I. Chai, and A. Abdullah, “Parameter and hyperparameter optimisation of deep neural network model for personalised predictions of asthma,” Journal of Advances in Information Technology, vol. 13, no. 5, Jan. 2022, doi: 10.12720/jait.13.5.512-517.
[15] F. Al Farid, A. Bari, A. S. M. Miah, S. Mansor, J. Uddin, and S. P. Kumaresan, “A Structured and Methodological Review on Multi-View Human Activity Recognition for Ambient Assisted Living,” J. Imaging, vol. 11, no. 6, p. 182, Jun. 2025, doi: 10.3390/jimaging11060182.
[16] A. Karatayev, A. Ogorodova, and P. Shamoi, “Fuzzy Inference System for Test Case Prioritization in Software Testing,” 2024, arXiv. doi: 10.48550/ARXIV.2404.16395.
[17] H. Kumar and V. Saxena, “Machine Learning for Test Case Prioritization in Continuous Integration: A Comprehensive Analysis,” IJANA, vol. 17, no. 02, pp. 6218–6228, 2024, doi: 10.35444/IJANA.2024.15609.
[18] S. Sowmyadevi, and S. Mehrotra, “Deep-TCP: An Intelligent Deep Learning Model for Automated Test Case Prioritization Systems using Neural Networks,” IJETT, vol. 74, no. 6, pp. 294–308, Jul. 2026, doi: 10.14445/22315381/IJETT-V74I6P121.
[19] S. Al-Juboori, J. Pua, S.-C. Haw, L. D. Krisnawati, and G.-K. Tong, “Ontology-Based E-Commerce Recommender System: A Hybrid Semantic Filtering Approach,” JIWE, vol. 1, no. 5, p. 86, Feb. 2026, doi: 10.33093/jiwe.2026.5.1.6.
[20] K. Garg and S. Shekhar, “Test case prioritization based on fault sensitivity analysis using ranked NSGA-2,” Int. j. inf. tecnol., vol. 16, no. 5, pp. 2875–2881, Jun. 2024, doi: 10.1007/s41870-024-01868-0.
[21] Z. Sakhrawi and T. Labidi, “Test case selection and prioritization approach for automated regression testing using ontology and COSMIC measurement,” Autom Softw Eng, vol. 31, no. 2, p. 51, Nov. 2024, doi: 10.1007/s10515-024-00447-8.
[22] R. Cheng, S. Wang, R. Jabbarvand, and D. Marinov, “Revisiting Test-Case Prioritization on Long-Running Test Suites,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, Vienna Austria: ACM, Sep. 2024, pp. 615–627. doi: 10.1145/3650212.3680307.
[23] T. Mattis, L. Böhme, E. Krebs, M. C. Rinard, and R. Hirschfeld, “Faster Feedback with AI? A Test Prioritization Study,” in Companion Proceedings of the 8th International Conference on the Art, Science, and Engineering of Programming, Lund Sweden: ACM, Mar. 2024, pp. 32–40. doi: 10.1145/3660829.3660837.
[24] I. Ibriwesh, S.-B. Ho, I. Chai, and C.-H. Tan, “Prioritizing solution-oriented software requirements using the multiple perspective prioritization technique algorithm: An empirical investigation,” Concurrent Engineering, vol. 27, no. 1, pp. 68–79, Mar. 2019, doi: 10.1177/1063293X18808559.
[25] R. Pan, M. Bagherzadeh, T. A. Ghaleb, and L. Briand, “Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review,” 2021, doi: 10.48550/ARXIV.2106.13891.
[26] Y. Zhao, D. Hao, and L. Zhang, “Revisiting Machine Learning based Test Case Prioritization for Continuous Integration,” 2023, arXiv. doi: 10.48550/ARXIV.2311.13413.
[27] G. Rao and D. Nandal, “Dynamic prioritization of test cases for regression testing using machine learning,” Int J Syst Assur Eng Manag, Feb. 2026, doi: 10.1007/s13198-026-03144-9.
[28] H. A. Ramzan, K. Islam, M. A. Hussain, R. M. Monim, S. M. Asad, and S. Ramzan, “Data-Driven Test Case Prioritization (DD-TCP): A Machine Learning Framework for Intelligent Software Quality Assurance,” Computers, Materials & Continua (CMC), pp. 1–10, 2026, doi: 10.32604/cmc.2026.077782.
[29] C.-W. Teoh, S.-B. Ho, K. S. Dollmat, and I. Chai, “An Evolutionary Algorithm-Based Optimization Ensemble Learning Model for Predicting Academic Performance,” in 2022 11th International Conference on Software and Computer Applications, Melaka Malaysia: ACM, Feb. 2022, pp. 102–107. doi: 10.1145/3524304.3524320.
[30] R. Haque et al., “Intelligent Health Informatics with Personalisation in Weather-Based Healthcare Using Machine Learning,” in Innovative Systems for Intelligent Health Informatics, vol. 72, F. Saeed, F. Mohammed, and A. Al-Nahari, Eds., Cham: Springer International Publishing, 2021, pp. 29–40. doi: 10.1007/978-3-030-70713-2_4.