Enhancing Fraud Detection in Financial Transactions using LightGBM and Random Forest

Abstract

In recent years, the frequency and complexity of financial fraud have been rising and have become an urgent challenge for the global financial system. Traditional rule-based detection methods struggle to cope with new types of fraud, especially in terms of real-time detection, generalization ability, and accuracy. To overcome these limitations, machine learning techniques have gradually emerged as a promising solution for identifying fraudulent transactions with better flexibility and scalability. Based on the publicly available European credit card fraud transaction dataset, this study proposes a hybrid model that combines the advantages of LightGBM and Random Forest, aiming to improve the accuracy, robustness, and interpretability of fraud detection. To handle the severe data imbalance problem (fraud cases accounting for only 0.17%), this study applies a RandomUnderSampling strategy and further enhances model performance through systematic parameter tuning using RandomizedSearchCV and decision threshold optimization. Stratified K-Fold cross-validation is also used to validate model stability. In addition, the model is compared with alternative resampling methods including SMOTE and ADASYN, and the results reaffirm the effectiveness and practicality of the undersampling approach. The final model achieves an overall accuracy of 99%, a recall of 86% on the fraud class, ROC-AUC of 0.9746, and PR-AUC of 0.6639. While the precision is relatively low (13%), it reflects a deliberate strategy of prioritizing fraud detection over false positives. This hybrid approach demonstrates a good balance between detection performance and practicality, offering better interpretability and lower computational cost compared to many deep learning models.