Empirical Analysis of CI/CD Tools Usage in GitHub Actions Workflows
Article Sidebar
Main Article Content
Abstract
As software systems grow larger and more complex, with rapidly changing requirements, manually managing code integration, testing, and deployment becomes extremely challenging. Continuous Integration and Continuous Deployment (CI/CD) practices and tools have emerged to help automate these processes. This research explores the usage of different categories of CI/CD tools within GitHub Actions workflow configurations across GitHub repositories. The five-tool categories analyzed are Version Control Management, Static Code Analysis, Build Automation, Test Automation, and CI/CD Servers. The data used in this research is from a dataset of GitHub Actions workflow configuration files. From the data, the usage is extracted and the concurrent usage of the tools is calculated. Next, the tools are labeled based on their taxonomy. In our finding, the build automation has the biggest number of uses, while the test automation has the least number of uses. Our finding indicates the correlation between the tool category and the programming language used in the software project. Meanwhile, some tools cannot be classified into the existing taxonomy. This can lead to reevaluating the taxonomy structure of CI/CD tools.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, “Agile Software Development Methods: Review and Analysis,” Sep. 2017.
M. Shahin, M. Ali Babar, and L. Zhu, “Continuous Integration, Delivery and Deployment: A Systematic Review on Approaches, Tools, Challenges and Practices,” IEEE Access, vol. 5, pp. 3909–3943, 2017, doi: 10.1109/ACCESS.2017.2685629.
Y. Ska and R. Publications, “A STUDY AND ANALYSIS OF CONTINUOUS DELIVERY, CONTINUOUS INTEGRATION IN SOFTWARE DEVELOPMENT ENVIRONMENT,” SSRN Electron. J., vol. 6, pp. 96–107, Apr. 2019.
P. Rostami Mazrae, T. Mens, M. Golzadeh, and A. Decan, “On the usage, co-usage and migration of CI/CD tools: A qualitative analysis,” Empir. Softw. Eng., vol. 28, no. 2, p. 52, Mar. 2023, doi: 10.1007/s10664-022-10285-5.
M. Wessel, T. Mens, A. Decan, and P. R. Mazrae, “The GitHub Development Workflow Automation Ecosystems,” in Software Ecosystems, Cham: Springer International Publishing, 2023, pp. 183–214.
P. O. Cano, A. M. Mejia, S. De Gyves Avila, G. E. Z. Dominguez, I. S. Moreno, and A. N. Lepe, “A Taxonomy on Continuous Integration and Deployment Tools and Frameworks,” 2021, pp. 323–336.
D. D. R. Barros, F. Horita, and D. G. Fantinato, “Data mining tool to discover DevOps trends from public repositories,” in Proceedings of the XXXIV Brazilian Symposium on Software Engineering, Oct. 2020, pp. 658–663, doi: 10.1145/3422392.3422501.
F. Chatziasimidis and I. Stamelos, “Data collection and analysis of GitHub repositories and users,” in 2015 6th International Conference on Information, Intelligence, Systems and Applications (IISA), Jul. 2015, pp. 1–6, doi: 10.1109/IISA.2015.7388026.
R. Hebig, T. H. Quang, M. R. V. Chaudron, G. Robles, and M. A. Fernandez, “The quest for open source projects that use UML,” in Proceedings of the ACM/IEEE 19th International Conference on Model Driven Engineering Languages and Systems, Oct. 2016, pp. 173–183, doi: 10.1145/2976767.2976778.
R. Peters and A. Zaidman, “Evaluating the Lifespan of Code Smells using Software Repository Mining,” in 2012 16th European Conference on Software Maintenance and Reengineering, Mar. 2012, pp. 411–416, doi: 10.1109/CSMR.2012.79.
J. Lima, C. Treude, F. F. Filho, and U. Kulesza, “Assessing developer contribution with repository mining-based metrics,” in 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME), Sep. 2015, pp. 536–540, doi: 10.1109/ICSM.2015.7332509.
M. M. Hussain, S. Akbar, S. A. Hassan, M. W. Aziz, and F. Urooj, “Prediction of Student’s Academic Performance through Data Mining Approach,” J. Informatics Web Eng., vol. 3, no. 1, pp. 241–251, Feb. 2024, doi: 10.33093/jiwe.2024.3.1.16.
K. L. Lew, C. Y. Kew, K. S. Sim, and S. C. Tan, “Adaptive Gaussian Wiener Filter for CT-Scan Images with Gaussian Noise Variance,” J. Informatics Web Eng., vol. 3, no. 1, pp. 169–181, Feb. 2024, doi: 10.33093/jiwe.2024.3.1.11.
J. Loeliger and M. McCullough, Version Control with Git: Powerful tools and techniques for collaborative software development. “ O’Reilly Media, Inc.,” 2012.
C. Artho and A. Biere, “Combined Static and Dynamic Analysis,” Electr. Notes Theor. Comput. Sci., vol. 131, pp. 3–14, 2005, doi: 10.1016/j.entcs.2005.01.018.
M. Prakash, “Software Build Automation Tools a Comparative Study between Maven, Gradle, Bazel and Ant,” Int. J. Softw. Eng. & Appl. DOI https//doi. org/10.5121/ijsea.
G. Mohan, Full Stack Testing. “ O’Reilly Media, Inc.,” 2022.
J. Heaton, “Secondary analysis of qualitative data: An overview,” Hist. Soc. Res. Sozialforsch., pp. 33–45, 2008.
G. Cardoen, “A dataset of GitHub Actions workflow histories.” Zenodo, 2024, doi: 10.5281/ZENODO.10566003.
Amazon Web Services (2023, July 24). Practicing Continuous Integration and Continuous Delivery on AWS, AWS Whitepaper. Accessed on: April 30, 2024. [Online]. Available: https://docs.aws.amazon.com/pdfs/whitepapers/latest/practicing-continuous-integration-continuous-delivery/practicing-continuous-integration-continuous-delivery.pdf