Adaptive and Learned Query Optimization: Techniques, Architectures, and Emerging Trends
Article Sidebar
Main Article Content
Abstract
As databases continue to grow and complexity there is an urgent need to achieve more efficient and faster query processing. This paper presents a technical review of methods that work towards optimization of various types of queries. It progresses by focusing on basic approaches like rules-based optimization and cost-based optimization. Based on this basis, the paper presents a synthesized reference architecture for adaptive and learned query optimizers as a generic framework derived from prior system. It explains on how the classical optimizer components can be enhanced through runtime monitoring integration and feedback mechanisms. From this perspective, it becomes possible to explain how learning modules and execution feedback should be added to the standard optimization pipelines instead of requiring any revolutionary changes in the current database systems. Next, various Adaptive Query Processing (AQP) approaches are reviewed to highlight their strengths and limitations. A fundamental aspect to consider with respect to decision-making process is covered in how effective these methods are from a re-optimization point of view, as well as with adaptive operators. The final section discussed the emerging trends in this field, including some current developments and what is expected to come regarding query optimization alongside machine learning integration, and distributed queries handling advancement issues. By synthesizing insights from existing literature, this review equips researchers with a robust understanding of the current state and future potential of query optimization and adaptive processing techniques.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
[1] Y. Zhang, Y. Chronis, J. M. Patel, and T. Rekatsinas, “Simple Adaptive Query Processing vs. Learned Query Optimizers: Observations and Analysis,” Proceedings of the VLDB Endowment, vol. 16, no. 11, pp. 2962–2975, Jul. 2023, doi: 10.14778/3611479.3611501.
[2] R. Heinrich, M. Luthra, J. Wehrstein, H. Kornmayer, and C. Binnig, “How Good are Learned Cost Models, Really? Insights from Query Optimization Tasks,” Proceedings of the ACM on Management of Data, vol. 3, no. 3, pp. 1–27, Jun. 2025, doi: 10.1145/3725309.
[3] E. A. Anaam, S.-C. Haw, and P. Naveen, “Applied fuzzy and analytic hierarchy process in hybrid recommendation approaches for E-CRM,” JOIV International Journal on Informatics Visualization, vol. 6, no. 2-2, p. 553, Aug. 2022, doi: 10.30630/joiv.6.2-2.1043.
[4] J. Shajeena, R. M. Shiny, K. Gayathri Devi, and M. Mary Vespa, “A Review of Query Optimization Techniques: From Traditional to Reinforcement Learning Approaches,” in 2026 International Conference on Electronics and Renewable Systems (ICEARS), Tuticorin, India, Feb. 11–13, 2026, pp. 1185–1189, doi: 10.1109/ICEARS67481.2026.11416629.
[5] I. Trummer, “Cost-Based Query Optimization for Quantum Computation,” in Proceedings of the 2nd Workshop on Quantum Computing and Quantum-Inspired Technology for Data-Intensive Systems and Applications, New York, NY, USA: ACM, Jun. 2025, pp. 1–2. doi: 10.1145/3736393.3736690.
[6] S.-J. Qiao et al., “Learning database optimization techniques: the state-of-the-art and prospects,” Front. Comput. Sci., vol. 19, no. 12, p. 1912612, Dec. 2025, doi: 10.1007/s11704-025-41116-7.
[7] S. Subramaniam, S.-C. Haw, and L.-K. Soon, “Improved Centralized XML Query Processing Using Distributed Query Workload,” IEEE Access, vol. 9, pp. 29127–29142, 2021, doi: 10.1109/ACCESS.2021.3058383.
[8] M. Acosta, “Machine Learning for Query Optimization in Knowledge Graphs,” 2026, pp. 183–191. doi: 10.1007/978-3-032-05727-3_18.
[9] H. Su-Cheng and L. Chien-Sing, “Efficient Preprocesses for Fast Storage and Query Retrieval in Native XML Database,” IETE Technical Review, vol. 26, no. 1, p. 28, 2009, doi: 10.4103/0256-4602.48466.
[10] E. Song and S.-C. Haw, “XML-REG: Transforming XML Into Relational Using Hybrid-Based Mapping Approach,” IEEE Access, vol. 8, pp. 177623–177639, 2020, doi: 10.1109/ACCESS.2020.3026006.
[11] B. Milicevic and Z. Babovic, “A systematic review of deep learning applications in database query execution,” J. Big Data, vol. 11, no. 1, p. 173, Dec. 2024, doi: 10.1186/s40537-024-01025-1.
[12] S. Subramaniam, S.-C. Haw, and L.-K. Soon, “Performance Evaluation of XML Query Processing in Centralized and Distributed Environment,” in Proceedings of the 2019 2nd International Conference on Computational Intelligence and Intelligent Systems, New York, NY, USA: ACM, Nov. 2019, pp. 129–133, doi: 10.1145/3372422.3372431.
[13] “Improving Chatbot Performance using Hybrid Deep Learning Approach,” Journal of System and Management Sciences, vol. 13, no. 3, May 2023, doi: 10.33168/JSMS.2023.0334.
[14] N. Hettiarachchi and P. Yapa, “Advancements in SQL Query Optimization: A Review of Join Order and Index Selection,” in 2025 5th International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), IEEE, Jun. 2025, pp. 31–39, doi: 10.1109/MLISE66443.2025.11100192.
[15] S. Salihoglu, J. Chen, Y. Huang, M. Wang, and K. Salem, “Cardinality Estimation Graphs,” Communications of the ACM, vol. 69, no. 2, pp. 99–109, Jan. 2026, doi: 10.1145/3780104.
[16] L. Woltmann, J. Thiessat, C. Hartmann, D. Habich, and W. Lehner, “FASTgres: Making Learned Query Optimizer Hinting Effective,” Proceedings of the VLDB Endowment, vol. 16, no. 11, pp. 3310–3322, Jul. 2023, doi: 10.14778/3611479.3611528.
[17] B. Ding, R. Zhu, and J. Zhou, “Learned Query Optimizers,” Foundations and Trends in Databases, vol. 13, no. 4, pp. 250–310, Sep. 2024, doi: 10.1561/1900000082.
[18] R. Radha et al., “AI based Self-Managed Query Optimization Framework for tightly coupled multistore big data systems,” SN Computer Science, vol. 7, no. 6, Jun. 2026, doi: 10.1007/s42979-026-05170-7.
[19] O. Gheibi, D. Weyns, and F. Quin, “Applying Machine Learning in Self-adaptive Systems,” ACM Transactions on Autonomous and Adaptive Systems, vol. 15, no. 3, pp. 1–37, Sep. 2020, doi: 10.1145/3469440.
[20] P. Mu, A. C. Carniel, A. Barbalace, and A. Shaikhha, “[Experiment, Analysis, and Benchmark] Systematic Evaluation of Plan-Based Adaptive Query Processing,” in 2026 IEEE 42nd International Conference on Data Engineering (ICDE), 2026, pp. 3501–3514, doi: 10.1109/ICDE65706.2026.00257.
[21] S. A. Peious, M. Kaushik, M. Shahin, R. Sharma, and D. Draheim, “On Choosing the Columnar In-Memory Database Hyrise as High-Performant Implementation Platform for the GrandReport Tool,” in 2025 International Conference on Next Generation Information System Engineering (NGISE), IEEE, Mar. 2025, pp. 1–8, doi: 10.1109/NGISE64126.2025.11085185.
[22] Y. Izenov, A. Datta, F. Rusu, and J. H. Shin, “COMPASS: Online Sketch-based Query Optimization for In-Memory Databases,” in Proceedings of the 2021 International Conference on Management of Data, New York, NY, USA: ACM, Jun. 2021, pp. 804–816, doi: 10.1145/3448016.3452840.
[23] G. T. Kakkar, J. Cao, A. Sengupta, J. Arulraj, and H. Kim, “Hydro: Adaptive Query Processing of ML Queries,” arXiv preprint arXiv:2403.14902, Mar. 2024, doi: 10.48550/arXiv.2403.14902.
[24] R. Marcus, P. Negi, H. Mao, N. Tatbul, M. Alizadeh, and T. Kraska, “Bao: Making Learned Query Optimization Practical,” in Proceedings of the 2021 International Conference on Management of Data, New York, NY, USA: ACM, Jun. 2021, pp. 1275–1288. doi: 10.1145/3448016.3452838.
[25] D. Lindner, D. Ritter, and F. Naumann, “Enabling Data Dependency-based Query Optimization,” arXiv preprint arXiv:2406.06886, Jun. 2024, doi: 10.48550/arXiv.2406.06886.
[26] Z. Yang, W.-L. Chiang, S. Luan, G. Mittal, M. Luo, and I. Stoica, “Balsa: Learning a Query Optimizer Without Expert Demonstrations,” in Proceedings of the 2022 International Conference on Management of Data, New York, NY, USA: ACM, Jun. 2022, pp. 931–944. doi: 10.1145/3514221.3517885.
[27] R. Zhu et al., “Lero: A Learning-to-Rank Query Optimizer,” Proceedings of the VLDB Endowment, vol. 16, no. 6, pp. 1466–1479, 2023, doi: 10.14778/3583140.3583160.
[28] L. Weng, R. Zhu, D. Wu, B. Ding, B. Zheng, and J. Zhou, “Eraser: Eliminating Performance Regression on Learned Query Optimizer,” Proceedings of the VLDB Endowment, vol. 17, no. 5, pp. 926–938, Jan. 2024, doi: 10.14778/3641204.3641205.
[29] T. Chen, J. Gao, H. Chen, and Y. Tu, “LOGER: A Learned Optimizer towards Generating Efficient and Robust Query Execution Plans,” Proceedings of the VLDB Endowment, vol. 16, no. 7, pp. 1777–1789, 2023, doi: 10.14778/3587136.3587150.
[30] C. Anneser et al., “AutoSteer: Learned Query Optimization for Any SQL Database,” Proceedings of the VLDB Endowment, vol. 16, no. 12, pp. 3515–3527, Aug. 2023, doi: 10.14778/3611540.3611544.
[31] V. Panwar, “AI-Driven Query Optimization: Revolutionizing Database Performance and Efficiency,” International Journal of Computer Trends and Technology, vol. 72, no. 3, pp. 18–26, Mar. 2024, doi: 10.14445/22312803/IJCTT-V72I3P103.
[32] Z. Yao, H. Li, J. Zhang, C. Li, and H. Chen, “A Query Optimization Method Utilizing Large Language Models,” arXiv preprint arXiv:2503.06902, Mar. 2025, doi: 10.48550/arXiv.2503.06902.
[33] F. Eppinger and U. Storl, “Tuning Cassandra through Machine Learning,” Lecture Notes in Informatics (LNI), Gesellschaft für Informatik, Bonn, Germany, 2023, pp. 93–104, doi: 10.18420/BTW2023-04.
[34] Y. Du, Z. Cai, and Z. Ding, “Query Optimization in Distributed Database Based on Improved Artificial Bee Colony Algorithm,” Applied Sciences, vol. 14, no. 2, p. 846, Jan. 2024, doi: 10.3390/app14020846.
[35] C. Zhang, O. Papaemmanouil, and J. Hanna, “Multi-agent Databases via Independent Learning,” arXiv preprint arXiv:2205.14323, 2022, doi: 10.48550/arXiv.2205.14323.
[36] T.-G.-B. Nguyen et al., “Towards Explainable Polypharmacy Risk Warnings Using Reinforcement Learning on Knowledge Graphs,” 2025, pp. 582–600, doi: 10.1007/978-3-031-90194-2_41.
[37] S. Milani, N. Topin, M. Veloso, and F. Fang, “Explainable Reinforcement Learning: A Survey and Comparative Review,” ACM Comput. Surv., vol. 56, no. 7, pp. 1–36, Jul. 2024, doi: 10.1145/3616864.
[38] A. Kamali, V. Kantere, C. Zuzarte, and V. Corvinelli, “Robust Plan Evaluation based on Approximate Probabilistic Machine Learning,” Proceedings of the VLDB Endowment, vol. 18, no. 8, pp. 2626–2638, 2025, doi: 10.14778/3742728.3742753.
[39] S. Yue, Y. Shao, and S. Hayes, “OntoTune: Ontology-Driven Learning for Query Optimization with Convolutional Models,” Arxiv, Nov. 2025, https://doi.org/10.48550/arXiv.2511.06780
[40] L. D. Krisnawati, A. W. Mahastama, S.-C. Haw, K.-W. Ng, and P. Naveen, “Indonesian-English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS),” CommIT (Communication and Information Technology) Journal, vol. 18, no. 2, pp. 183–195, Sep. 2024, https://doi.org/10.21512/commit.v18i2.11274
[41] J. R. Haritsa, “Robust Query Processing: A Survey,” Foundations and Trends in Databases, vol. 15, no. 1, pp. 1–114, Dec. 2024, doi: 10.1561/1900000089.
[42] J. Zhu, L. Chen, X. Ke, Z. Fang, T. Li, Y. Gao, and C. S. Jensen, “Beyond Relational: Semantic-Aware Multi-Modal Analytics with LLM-Native Query Optimization,” Proceedings of the ACM on Management of Data, vol. 4, no. 1, Art. no. 14, pp. 1–25, 2026, doi: 10.1145/3786628
[43] S.-C. Haw, J. W. May, and S. Subramaniam, “Mapping Relational Databases to Ontology Representation,” in Proceedings of the 1st International Conference on Digital Technology in Education, New York, NY, USA: ACM, Aug. 2017, pp. 54–58, doi: 10.1145/3134847.3134852.
[44] D. Satriani, E. Veltri, D. Santoro, S. Rosato, S. Varriale, and P. Papotti, “Logical and Physical Optimizations for SQL Query Execution over Large Language Models,” Proceedings of the ACM on Management of Data, vol. 3, no. 3, pp. 1–28, Jun. 2025, doi: 10.1145/3725411.
[45] L. Shi, Z. Tang, N. Zhang, X. Zhang, and Z. Yang, “A Survey on Employing Large Language Models for Text-to-SQL Tasks,” ACM Comput. Surv., vol. 58, no. 2, pp. 1–37, Jan. 2026, doi: 10.1145/3737873.
[46] J. Tan et al., “Can Large Language Models Be Query Optimizer for Relational Databases?,” Proceedings of the ACM on Management of Data, vol. 3, no. 6, pp. 1–28, Dec. 2025, doi: 10.1145/3769771.