CRATSM: An Effective Hybridization of Deep Neural Models for Customer Retention Prediction in the Telecom Industry Manuscript Received: 8 April 2024, Accepted: 5 June 2024, Published: 15 September 2024, ORCiD: 0000-0001-5539-1959, https://doi.org/10.33093/jetap.2024.6.2.10

Article Sidebar

Article
Published: Sep 15, 2024
DOI: https://doi.org/10.33093/jetap.2024.6.2.10
Keywords:
Optimization, Residual Network, Attention Mechanism, Tree-structured Network, Cost-sensitive

Main Article Content

Johnson Olanrewaju Victor
School of Computer Sciences, Universiti Sains Malaysia, Pulau Pinang, Malaysia.
XinYing Chew
School of Computer Sciences, Universiti Sains Malaysia, Pulau Pinang, Malaysia.
Khai Wah Khaw
School of Management, Universiti Sains Malaysia, Pulau Pinang, Malaysia.
Zhi Lin Chong
Department of Electronic Engineering, Faculty of Engineering and Green Technology, Universiti Tunku Abdul Rahman, Kampar, Malaysia

Abstract

In the dynamic field of Customer Retention Prediction (CRP), strategic marketing and promotion efforts targeting specific customers are crucial. Understanding customer behavior and identifying churn indicators are vital for devising effective retention strategies. However, identifying customers likely to terminate services presents a challenge, leading to data imbalance issues. Existing CRP studies using Machine Learning (ML) techniques and data imbalance methods face problems such as overfitting and computational complexity. Similarly, recent CRP studies employing Deep Learning (DL) approaches rely on data sampling techniques, which can result in overfitting and a lack of cost sensitivity. Additionally, DL approaches struggle with slow convergence and get stuck in local minima. This paper introduces an effective hybrid of Deep Learning (DL) classifiers focusing on cost-metric integration to address data imbalance issues and period-shift Cosine Annealing Learning Rate (ps-CALR) to accelerate model training, ultimately enhancing performance. Three Telecom datasets, namely IBM, Iranian, and Orange, were used to assess the model performance. Empirical findings show that the hybrid DL classifiers significantly improved CRP over conventional ML. This paper contributes methodological advancements and practical insights for effective customer retention in the telecom industry.

Article Details

Issue
Vol. 6 No. 2 (2024): https://doi.org/10.33093/jetap.2024.6.2
Section
Articles
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

References

Global Monitor, Malaysia Telecommunication Market Report, pp.1–4. 2021.

Statista, Number of internet and social media users worldwide as of July 2023, https://www.statista.com/statistics/ [Accessed 2023/10/20]

N. Y. Nhu, T. Van Ly and D. V. Truong Son, “Churn Prediction in Telecommunication Industry using Kernel Support Vector Machines,” PLoS One, vol. 17, pp. 0267935, 2022.

M. Zhu and J. Liu, “Telecom Customer Churn Prediction Based on Classification Algorithm,” in 2021 Int. Conf. Aviation Safety and Inform. Technol., pp. 268–273, 2021.

N. Alboukaey, A. Joukhadar and N. Ghneim, “Dynamic Behavior Based Churn Prediction in Mobile Telecom,” Expert Sys. Appl., vol. 162, pp. 113779, 2020.

N. I. Mohammad, S. A. Ismail, M. N. Kama, O. M. Yusop and A. Azmi, “Customer Churn Prediction in Telecommunication Industry Using Machine Learning Classifiers,” in Proc. 3rd Int. Conf. Vision, Image and Sign. Process., no. 34, pp. 1-7, 2019.

S. Saleh and S. Saha, “Customer Retention and Churn Prediction in The Telecommunication Industry: A Case Study on A Danish University,” SN Appl. Sci., vol. 5, pp. 173. 2023.

B. Do Chung, J. H. Park, Y. J. Koh and S. Lee, “User Satisfaction and Retention of Mobile Telecommunications Services in Korea,” Int. J. Hum. Comput. Interact., vol. 32, pp. 532–543, 2016.

L. Y. Chen, Y. Chen, Y. D. Kwon, Y. Kang and P. Hui, “IAN: Interpretable Attention Network for Churn Prediction in LBSNs,” in Proc. of the 2021 IEEE/ACM on Adv. in Soc. Netw. Analy. and Mining, pp. 23–30, 2021.

M. Al-Mashraie, S. H. Chung and H. W. Jeon, “Customer Switching Behavior Analysis in The Telecommunication Industry via Push-Pull-Mooring Framework: A Machine Learning Approach,” Comput. Ind. Eng., vol. 144, pp. 106476, 2020.

A. Amin, B. Shah, A. Khattak, F. Moreira, G. Ali, Á. Rocha and S. Anwar, “Cross-company Customer Churn Prediction in Telecommunication: A Comparison of Data Transformation Methods,” Int. J. Inf. Manage., vol. 46, pp. 304–319, 2019.

S. Maldonado, J. López and C. Vairetti, “Profit-based Churn Prediction Based on Minimax Probability Machines,” Eur. J. Oper. Res., vol. 284, pp. 273–284, 2020.

E. King and J. Rice, “Analysis of Churn in Mobile Telecommunications: Predicting the Timing of Customer Churn,” AIMS Int. J. of Mgt., vol. 13, pp. 127-141, 2019.

A. Amin, F. Al-Obeidat, B. Shah, A. A. J. Loo and S. Anwar, “Customer Churn Prediction in Telecommunication Industry using Data Certainty,” J. Bus. Res., vol. 94, pp. 290-301, 2019.

S. M. Kostic, M. I. Simic and M. V. Kostic, “Social Network Analysis and Churn Prediction in Telecommunications Using Graph Theory,” Entropy, vol. 22, pp. 753, 2020.

N. Gamulin, M. Stular and S. Tomazic, “Impact of Social Network to Churn in Mobile Network,” Automatika, vol. 56, pp. 252–261, 2015.

S. A. Amatare and A. K. Ojo, “Predicting Customer Churn in Telecommunication Industry Using Convolutional Neural Network Model,” IOSR J. Comput. Eng., vol. 22, pp. 54–59, 2020.

S. Li, G. Xia and X. Zhang, “Customer Churn Combination Prediction Model Based on Convolutional Neural Network and Gradient Boosting Decision Tree,” in Proc. 2022 5th Int. Conf. Algorit., Comput. and Artif. Intellig., no. 12, pp. 1-6, 2022.

V. Umayaparvathi and K. Iyakutti, “Automated Feature Selection and Churn Prediction using Deep Learning Models,” Int. Res. J. Eng. and Technol., vol. 4, pp. 1846–1854, 2017.

S. Kumar and M. Kumar, “Predicting Customer Churn using Artificial Neural Network,” Commun. in Comput. and Inform. Sci., vol. 1000, pp. 299-306, Springer, Cham, 2019.

A. Fernandez, S. Garcia, M. Galar, R. C. Prati, B. Krawczyk and F. Herrera, Learning from Imbalanced Data Sets, Springer, 2018.

R. Shwartz-Ziv and A. Armon, “Tabular data: Deep learning is not all you need,” Information Fusion, vol. 81, pp. 84–90, 2022.

L. Grinsztajn, E. Oyallon and G. Varoquaux, “Why Do Tree-Based Models Still Outperform Deep Learning on Typical Tabular Data?” Adv. Neural Inf. Process Sys., vol. 35, pp. 1-14, 2022.

Y. Gorishniy, I. Rubachev, V. Khrulkov and A. Babenko, “Revisiting Deep Learning Models for Tabular Data,” Adv. Neural Inf. Process Sys., vol. 23, pp. 18932–18943, 2021.

S. A. Fayaz, M. Zaman, S. Kaul and M. A. Butt, “Is Deep Learning on Tabular Data Enough? An Assessment,” Int. J. Adv. Comput. Sci. and Appl., vol. 13, pp. 466–473, 2022.

B. N. R. Chagas, J. Viana, O. Reinhold, F. M. F. Lobato, A. F. L. Jacob, R. Alt, “A Literature Review of The Current Applications of Machine Learning ond Their Practical Implications,” Web Intellig., vol. 18, pp. 69–83, 2020.

J. Xiao, L. Huang and L. Xie, “Cost-Sensitive Semi-Supervised Ensemble Model for Customer Churn Prediction,” in 2018 15th Int. Conf. Serv. Sys. and Serv. Manage., Hangzhou, China, pp. 1-6, 2018.

M. Panjasuchat and Y. Limpiyakorn, “Applying Reinforcement Learning for Customer Churn Prediction,” J. Phys. Conf. Ser., vol. 1619, no. 012016, 2020.

A. Amin, S. Anwar, A. Adnan, M. Nawaz, N. Howard, J. Qadir, A. Hawalah and A. Hussain, “Comparing Oversampling Techniques to Handle the Class Imbalance Problem: A Customer Churn Prediction Case Study,” IEEE Access, vol. 4, pp. 7940–7957, 2016.

X. Wang and S. Mu, “E-commerce Customer Churn Prediction Based on Improved SMOTE and AdaBoost,” in 13th Int. Conf. on Serv. Sys. and Serv. Manage., Kunming, pp. 1–5, 2016.

N. M. Mqadi, N. Naicker and T. Adeliyi, “Solving Misclassification of The Credit Card Imbalance Problem using Near Miss,” Math. Probl. Eng., vol. 2021, pp. 1–16, 2021.

M. Fathian, Y. Hoseinpoor and B. Minaei-Bidgoli, “Offering A Hybrid Approach of Data Mining to Predict The Customer Churn Based on Bagging and Boosting Methods,” Kybernetes, vol. 45, pp. 732–743, 2016.

K. Kim, C. H. Jun and J. Lee, “Improved Churn Prediction in Telecommunication Industry by Analyzing A Large Network,” Expert Sys. Appl., vol. 41, pp. 6575–6584, 2014.

A. Sniegula, A. Poniszewska-Maranda and M. Popovic, "Study of Machine Learning Methods for Customer Churn Prediction in Telecommunication Company," in Proc. 21st Int. Conf. Inform. Integr. and Web-based Appl. & Serv., pp. 640-644, 2019.

A. Chouiekh and E. H. I. El Haj, "Deep Convolutional Neural Networks for Customer Churn Prediction Analysis," Int. J. Cognit. Inform. and Nat. Intellig., vol. 14, pp. 1–16, 2020.

E. Domingos, B. Ojeme and O. Daramola, "Experimental Analysis of Hyperparameters for Deep Learning-Based Churn Prediction in The Banking Sector," Computation, vol. 9, no. 34, pp. 1-19, 2021.

Y. L. Tan, Y. H. Pang, S. Y. Ooi, W. H. Khoh and F. S. Hiew, "Stacking Ensemble Approach for Churn Prediction: Integrating CNN and Machine Learning Models with CatBoost Meta-Learner," J. Eng. Technol. and Appl. Phys., vol. 5, pp. 99–107, 2023.

J. Jepkoech, D. M. Mugo, B. K. Kenduiywo and E. C. Too, "The Effect of Adaptive Learning Rate on the Accuracy of Neural Networks," Int. J. Adv. Comput. Sci. and Appl., vol. 12, pp. 736–751, 2021.

O. V. Johnson, C. Xinying, K. W. Khaw and M. H. Lee, "ps-CALR: Periodic-Shift Cosine Annealing Learning Rate for Deep Neural Networks," IEEE Access, vol. 11, pp. 139171–139186, 2023.

D. Chen, F. Hu, G. Nian and T. Yang, "Deep Residual Learning for Nonlinear Regression," Entropy, vol. 22(2), no. 193, pp. 1-14, 2020.

M. L. Yadav and B. Roychoudhury, "Handling Missing Values: A Study of Popular Imputation Packages in R," Knowl. Based Sys., vol. 160, pp. 104–118, 2018.

D. Singh and B. Singh, "Investigating the Impact of Data Normalization on Classification Performance," Appl. Soft Comput., vol. 97, pp. 105524, 2020.

Z. Allen-Zhu and Y. Li, “What Can Resnet Learn Efficiently, Going Beyond Kernels?” Adv. in Neur. Inform. Process. Sys. 32, 2019.

F. He, T. Liu and D. Tao, “Why Resnet Works? Residuals Generalize,” IEEE Trans. Neur. Netw. and Learn. Sys., vol. 31, no. 12, pp. 5349-5362, 2020.

D. Bahdanau, K. Cho and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” in Int. Conf. Learn. Repres., 2015.

Z. Niu, G. Zhong and H. Yu, "A Review on The Attention Mechanism of Deep Learning," Neurocomputing, vol. 452, pp. 48–62, 2021.

R. Balestriero, “Neural Decision Trees,” arXiv preprint, arXiv:1702.07360, 2017.

O. V. Johnson, C. XinYing, K. W. Khaw and M. H. Lee, "A Cost-Based Dual ConvNet-Attention Transfer Learning Model for ECG Heartbeat Classification," J. Inform. and Web Eng., vol. 2, pp. 90–110, 2023.

A. Keramati, R. Jafari-Marandi, M. Aliannejadi, I. Ahmadian, M. Mozaffari and U. Abbasi, "Improved Churn Prediction in Telecommunication Industry using Data Mining Techniques," Appl. Soft Comput. J., vol. 24, pp. 994–1012, 2014.

T. W. Cenggoro, R. A. Wirastari, E. Rudianto, M. I. Mohadi, D. Ratj and B. Pardamean, "Deep Learning As A Vector Embedding Model for Customer Churn," Procedia Comput. Sci., vol. 179, pp. 624–631, 2021.

M. Moradi and M. Dass, "Applications of Artificial Intelligence in B2B Marketing: Challenges and Future Directions," Indus. Market. Manage., vol. 107, pp. 300–314, 2022.

X. Wang, M. Johansson and T. Zhang, "Generalized Polyak Step Size for First Order Optimization with Momentum," in Proc. 40th Int. Conf. Mach. Learn., no. 1489, pp. 35836–35863, 2023.