Exploring Big Data Management Approaches and Applications: A Case Study of Real-Time Data Analytics in Air Traffic Management
Article Sidebar
Main Article Content
Abstract
The rapid proliferation of digital devices has generated vast amounts of data, presenting significant challenges in collection, processing, and analysis that traditional systems struggle to overcome. This study investigates big data management approaches, explicitly focusing on technologies capable of efficiently handling real-time data at scale. Within the context of Air Operations, we propose a Hadoop-based architecture designed to support the Observe-Orient-Decide-Act (OODA) loop and enhance air traffic management. By leveraging a distributed system deployed on a cloud-based platform, we demonstrate a cost-effective solution for optimised data processing and improved decision-making capabilities. Our analysis highlights the advantages of using Hadoop's distributed file system (HDFS) for managing both structured and unstructured data generated by various sensors and devices. Additionally, we explore the integration of real-time processing technologies, such as Apache Kafka and Spark, to facilitate timely insights essential for operational effectiveness. Cloud deployment not only enhances resource accessibility but also offers flexibility and scalability, which are crucial for adapting to the dynamic nature of defence operations. We also address critical considerations for security and compliance when handling sensitive military data in cloud environments and recommend strategies to mitigate potential risks. The study concludes with recommendations for addressing future technological needs in big data management, including the incorporation of machine learning for predictive analytics and improved data visualisation tools. By implementing our proposed architecture, the military/ civil aviation can enhance its operational efficiency and decision-making processes, positioning itself to meet future challenges in an increasingly data-driven environment.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
T. Berners-Lee, “Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web by Its Inventor,” HarperSanFrancisco, 1999.
M. Lesk, “How Much Information is There in the World?,” 1997. [Online]. Available: https://www.lesk.com/mlesk/ksg97/ksg.html.
F. X. Diebold, “A Personal Perspective on the Origin (s) and Development of a Big Data: The Phenomenon, the Term, and the Discipline,” Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, No. 13-003, 2012.
T. O’Reilly, “What is Web 2.0? Design patterns and business models for the next generation of software,” O'Reilly Media, 2005. [Online]. Available: https://www.oreilly.com/pub/a/web2/archive/what-is-web-20.html.
E. Schmidt, “Every two days we create as much information as we did up to 2003,” [Online]. Available: https://techonomy.com/conf/10/techonomy2010/.
B. Marr, “Big data case study collection: 7 amazing companies that really get big data,” 2015
A. De Mauro, M. Greco, and M. Grimaldi, “A formal definition of big data based on its essential features,” Library review, 65(3), pp.122-135. doi: 10.1108/LR-06-2015-0061.
R. Holt, “Twitter in numbers,” Telegraph, 21 Mar. 2013. [Online]. Available: http://www.telegraph.co.uk/technology/twitter/9945505/Twitter-innumbers.html.
B. Marr, “What is big data? A super simple explanation for everyone,” Bernard Marr & Co., 2016.
R. Roshini, and M. Raikar, “Map reduce based analysis of live website traffic integrated with improved performance for small files using Hadoop,”.
A. Mumuni, and F. Mumuni, “Automated data processing and feature engineering for deep learning and big data applications: A survey,” Journal of Information and Intelligence, 2024. doi: 10.1016/j.jiixd.2024.01.002.
J. Archenaa, and E.M. Anita, “A survey of big data analytics in healthcare and government,” Procedia Computer Science, vol. 50, pp. 408–413, 2015. doi: 10.1016/j.procs.2015.04.021.
M. Manikandan, P. Venkatesh, T. Illakya, M. Krishnamoorthi, C. R. Senthilnathan, and K. Maran, “The significance of big data analytics in the global healthcare market,” in 2024 International Conference on Communication, Computing and Internet of Things (IC3IoT), Apr. 2024, pp. 1–4. doi: 10.1109/IC3IoT60841.2024.10550417.
Y. Zhang, “Application of big data in smart agriculture,” Adv. Resour. Res., vol. 4, no. 2, pp. 221–230, 2024. doi: 10.50908/arr.4.2_221.
K. Vijayalakshmi, S. Uma, R. Bhuvanya, and A. Suresh, “A demand for wearable devices in health care,” Int. J. Eng. Technol, 7(1-7), 4. 2018.
K. Sravanthi and T.S. Reddy, “Applications of big data in various fields,” vol. 6, no. 5, pp. 4629–4632, 2015.
C. Snijders, U. Matzat, and U. Reips, “Big Data: Big gaps of knowledge in the field of internet science,” Int. J. Internet Sci., vol. 7, no. 1, pp. 1–5, 2012.
S. Singh, T. Firdaus, and A. K. Sharma, “Survey on Big Data Using Data Mining,” International Journal of Engineering Development and Research, vol. 3, no. 4, pp. 135–143, 2015.
M. R. Trifu, and M. L. Ivan, “Big data: Present and future,” Database Systems Journal, vol. 5, no. 1, pp. 32–41, May 2014.
J. Y. Zhu, J. Xu, and V. O. Li, “A four-layer architecture for online and historical big data analytics,” In 2016 IEEE 14th Intl Conf on Dependable, Autonomic and Secure Computing, 14th Intl Conf on Pervasive Intelligence and Computing, 2nd Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), pp. 634–639, 2016. doi: 10.1109/DASC-PICom-DataCom-CyberSciTec.2016.115.
G. Guerreiro, P. Figueiras, R. Silva, R. Costa, and R. Jardim-Goncalves, “An architecture for big data processing on intelligent transportation systems: An application scenario on highway traffic flows,” in Proc. 2016 IEEE 8th Int. Conf. Intell. Syst. (IS), pp. 65–72, 2016. doi: 10.1109/IS.2016.7737393.
T. C. Vance, T. Huang, and K. A. Butler, “Big data in Earth science: Emerging practice and promise,” Science, vol. 383, no. 6688, 2024. doi: 10.1126/science.adh9607.
A. Oussous, F. Z. Benjelloun, A. Ait Lahcen, and S. Belfkih, “Big data technologies: A survey,” Journal of King Saud University-Computer and Information Sciences, 30(4), pp. 431-448, 2018. doi: 10.1016/j.jksuci.2017.06.001.
S. Chaudhary, “Implementation and performance analysis of cloud,” International Journal, vol. 3, no. 9, pp. 317–322, 2013.
L. Johansson, “Part 1: RabbitMQ for beginners – What is RabbitMQ?,” CloudAMQP, 2015. [Online]. Available: https://www.cloudamqp.com/blog/part1-rabbitmq-for-beginners-what-is-rabbitmq.html.
L. U. Laboshin, A. A. Lukashin, and V. S. Zaborovsky, “The big data approach to collecting and analyzing traffic data in large-scale networks,” Procedia Computer Science, vol. 103, pp. 536–542, 2017. doi: 10.1016/j.procs.2017.01.048.
N. Brunetti-Lihach, “Information Warfare: Past, Present, and Future,” Nov. 14, 2018.
G. Zeng, “Application of big data in intelligent traffic system,” IOSR Journal of Computer Engineering, vol. 17, no. 1, pp. 2278–661, 2015. doi: 10.9790/0661-17160104.
R. Kumar, L. Adwani, S. Kumawat, and S. K. Jangir, “OpenNebula: Open source IaaS cloud computing software platforms,” In National Conference on Computational and Mathematical Sciences (COMPUTATIA-IV), Technically Sponsored By: ISITA and RAOPS, Jaipur, 2014. doi: 10.13140/2.1.3584.4805.
M. L. Hamzeh Khazaei, S. Zareian, and R. Veleda, “Sipresk: A big data analytic platform for smart transportation,” Procedia Computer Science, vol. 166, Nov. 2016.