Structured Data Preprocessing Techniques for a Multilingual Healthcare Dataset in Machine Learning Applications
Article Sidebar
Main Article Content
Abstract
Data preprocessing is a foundational stage in any machine learning (ML) workflow, especially when dealing with complex, multilingual, and real-world healthcare datasets. The quality, consistency, and structure of input data strongly influence the accuracy, fairness, and interpretability of predictive models. This paper presents a comprehensive and structured preprocessing framework designed for a large multilingual healthcare dataset containing 19,536 patient records with 67 features encompassing demographic, clinical, and socioeconomic information. The framework systematically addresses data heterogeneity, linguistic diversity, and structural inconsistencies through multiple stages of cleaning and transformation. The preprocessing process includes translation of Malay text into English to ensure linguistic uniformity, standardized handling of missing values using transparent placeholders, and column splitting of composite attributes to improve granularity and feature representation. Outlier detection was performed using statistical approaches such as Interquartile Range (IQR) and Z-Score methods to identify and remove extreme or erroneous values, thereby improving data integrity and model stability. Furthermore, categorical variables were converted into numerical form using one-hot encoding, ensuring compatibility with diverse ML algorithms while preventing artificial ordering and bias. An experimental evaluation using a Random Forest classifier demonstrated that these preprocessing techniques substantially enhanced predictive performance. Model accuracy improved from 72.4% to 86.7%, while the area under the curve (AUC) increased from 0.78 to 0.92, indicating more robust classification and generalization. The resulting dataset is cleaner, more consistent, and better suited for downstream ML applications. Overall, this study provides a reproducible and practical framework for managing multilingual healthcare data, underscoring that systematic preprocessing is not merely a preliminary task but a critical foundation for building reliable, transparent, and interpretable machine learning solutions in healthcare and other data-intensive domains.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
[1] P. Dhawas, A. Dhore, D. Bhagat, R. D. Pawar, A. Kukade, and K. Kalbande, “Big data preprocessing, techniques, integration, transformation, normalisation, cleaning, discretization, and binning,” in Advances in business information systems and analytics book series, 2024, pp. 159–182. doi: 10.4018/979-8-3693-0413-6.ch006.
[2] S. Dong, S. Sahri, and T. Palpanas, “Data Quality Awareness: A Journey from Traditional Data Management to Data Science Systems,” ACM SIGMOD Record, vol. 55, no. 2, pp. 23–24, Jul. 2026, doi: 10.1145/3829088.3829090.
[3] A. M. Sharifnia, D. E. Kpormegbey, D. K. Thapa, and M. Cleary, "A primer of data cleaning in quantitative research: handling missing values and outliers," Journal of Advanced Nursing, vol. 82, no. 1, pp. 970–975, Jan. 2026, doi: 10.1111/jan.16908.
[4] C. E. Morr, M. Jammal, H. Ali-Hassan, and W. Ei-Hallak, Machine learning for practical decision making. 2022. doi: 10.1007/978-3-031-16990-8.
[5] R. Thirunavukarasu, G. P. D. C, G. R, M. Gopikrishnan, and V. Palanisamy, “Towards computational solutions for precision medicine based big data healthcare system using deep learning models: A review,” Computers in Biology and Medicine, vol. 149, p. 106020, Aug. 2022, doi: 10.1016/j.compbiomed.2022.106020.
[6] M. Greenacre, P. J. F. Groenen, T. Hastie, A. I. D’Enza, A. Markos, and E. Tuzhilina, “Publisher Correction: Principal component analysis,” Nature Reviews Methods Primers, vol. 3, no. 1, Mar. 2023, doi: 10.1038/s43586-023-00209-y.
[7] H. Jeon, and S. Oh, “Hybrid-Recursive feature elimination for efficient feature selection,” Applied Sciences, vol. 10, no. 9, p. 3211, May 2020, doi: 10.3390/app10093211.
[8] M. A. K. Akhtar, M. Kumar, and A. Nayyar, “Transparency and Accountability in Explainable AI: Best Practices,” in Studies in systems, decision and control, 2024, pp. 127–164. doi: 10.1007/978-3-031-66489-2_5.
[9] A. N. Prasad, “Data quality and preprocessing,” in Apress eBooks, 2024, pp. 109–223. doi: 10.1007/979-8-8688-1023-7_3.
[10] K. M. Rao, G. Saikrishna, and K. Supriya, “Data preprocessing techniques: emergence and selection towards machine learning models - a practical review using HPA dataset,” Multimedia Tools and Applications, vol. 82, no. 24, pp. 37177–37196, Mar. 2023, doi: 10.1007/s11042-023-15087-5.
[11] Md. Hossain, A. Devnath, and P. Karmokar, “Handling missing values and outliers in advanced data pre-processing: An enhancement of diabetes classification accuracy,” Research Square (Research Square), Sep. 2023, doi: 10.21203/rs.3.rs-3364064/v1.
[12] Y. Gong, G. Liu, Y. Xue, R. Li, and L. Meng, “A survey on dataset quality in machine learning,” Information and Software Technology, vol. 162, p. 107268, Jun. 2023, doi: 10.1016/j.infsof.2023.107268.
[13] M. K. Dahouda, and I. Joe, “A Deep-Learned embedding technique for categorical features encoding,” IEEE Access, vol. 9, pp. 114381–114391, Jan. 2021, doi: 10.1109/access.2021.3104357.
[14] F. Bolikulov, R. Nasimov, A. Rashidov, F. Akhmedov, and Y.-I. Cho, “Effective methods of categorical data encoding for artificial intelligence algorithms,” Mathematics, vol. 12, no. 16, p. 2553, Aug. 2024, doi: 10.3390/math12162553.
[15] F. Dominguez, J. Unger, M. Traube, B. Mant, C. Ertler, and W. Lechner, “Encoding-independent optimization problem formulation for quantum computing,” Frontiers in Quantum Science and Technology, vol. 2, Sep. 2023, doi: 10.3389/frqst.2023.1229471.
[16] C. K. Williams I., “Naive bayes classifiers and one-hot encoding of categorical variables,” arXiv (Cornell University), Apr. 2024, doi: 10.48550/arxiv.2404.18190.
[17] F. Clerici and N. Nobani, “Categorical variable encoding methods for tabular data: a benchmarking study,” International Journal of Data Science and Analytics, vol. 22, no. 1, Feb. 2026, doi: 10.1007/s41060-025-00886-w.
[18] Ch. S. K. Dash, A. K. Behera, S. Dehuri, and A. Ghosh, “An outliers detection and elimination framework in classification task of data mining,” Decision Analytics Journal, vol. 6, p. 100164, Jan. 2023, doi: 10.1016/j.dajour.2023.100164.
[19] H. P. Vinutha, B. Poornima, and B. M. Sagar, “Detection of Outliers Using Interquartile Range Technique from Intrusion Dataset,” in Advances in intelligent systems and computing, 2018, pp. 511–518. doi: 10.1007/978-981-10-7563-6_53.
[20] D. Wu, X. Ma, and D. L. Olson, “Financial distress prediction using integrated Z-score and multilayer perceptron neural networks,” Decision Support Systems, vol. 159, p. 113814, May 2022, doi: 10.1016/j.dss.2022.113814.
[21] B. L. Ortiz et al., “Data preprocessing techniques for AI and Machine learning readiness: Scoping Review of wearable sensor data in cancer care,” JMIR Mhealth and Uhealth, vol. 12, p. e59587, Aug. 2024, doi: 10.2196/59587.
[22] S. U. Hassan, J. Ahamed, and K. Ahmad, “Analytics of machine learning-based algorithms for text classification,” Sustainable Operations and Computers, vol. 3, pp. 238–248, Jan. 2022, doi: 10.1016/j.susoc.2022.03.001.