Department of Computer Science and Engineering (Artificial Intelligence and Machine Learning), Harsha Institute of Technology, Bengaluru, Karnataka, India.
In recent years, the banking sector has faced increasing challenges from fraudulent activities in online transactions. According to survey reports, annual losses due to such frauds exceed $1 trillion. Even while financial fraud unsafe for entire organizations, it may be recovered with the help of intellectual solution like Machine Learning (ML) models, Artificial Intelligence (AI) etc. Also, leveraging big data analytics ML algorithm van improves the identification and mitigation performance of fraudulent activities efficiently. Therefore, this article has developed the hybridized algorithm for predicting financial fraud by integrating metaheuristic optimization-based ML model hyperparameter tuning with suitable classifier logics. Name of the developed model is an intelligent Gradient Boost based Whale Hawk’s Optimization with Bayesian (GB-WHOB) framework. Moreover, Banksim dataset has been collected for detecting the fraudulent transactions. This dataset includes payment transaction of numerous customers made in various time periods and amounts. Then, data pre-processing function applied on the collected dataset to messy raw data into readable and clean language formats. Here, convolution kernel function was enabled to altering the data before entering the next stage. Then, feature extraction is performed to extract the fraudulent features from the pre-processed data using. then, the developed model was enabled to analyse the anomaly actions using that Gradient Boost Tree (GBT) algorithm. This model establishes a baseline for normal transactions and detects deviations from this baseline to identify potential fraud. After that, user behavioural is important for detecting the fraud therefore Whale Optimization (WO) fitness function and Harris Hawk’s Optimization (HHO) fitness was combined the residual blocks and new decision tree was designed to trained the above residual block function then analyse the frauds accurately. In addition, Bayesian optimization function was adapted to enhance the current best observation in fraudulent activities. The proposed algorithm was modelled and implemented in the Python tool, and the proposed model achieved exceptional performance, recording 99.76% accuracy, 99.72% precision, 99.78% recall, 99.77% F-measure, 99.92% specificity, and a minimal 0.24% error rate. These results significantly outperform other optimization techniques, demonstrating its superior capability in accurately detecting fraudulent financial transactions with minimal false positives and false negatives.
N. F. Ryman-Tubb, P. Krause, and W. Garn, “How Artificial Intelligence and machine learning research impacts payment card fraud detection: A survey and industry benchmark,” Engineering Applications of Artificial Intelligence, vol. 76, pp. 130–157, Nov. 2018, doi: 10.1016/j.engappai.2018.07.008.
M. R. Kishore Mullangi, Vamsi Krishna Yarlagadda, Niravkumar Dhameliya, “Integrating AI and Reciprocal Symmetry in Financial Management: A Pathway to Enhanced Decision-Making,” Int. J. Reciprocal Symmetry Theor. Phys., vol. 5, no. 1, pp. 42–52, 2018.
S. K. R. A. Sai Charan Reddy Vennapusa, Takudzwa Fadziso, Dipakkumar Kanubhai Sachani, Vamsi Krishna Yarlagadda, “Cryptocurrency-Based Loyalty Programs for Enhanced Customer Engagement,” Technol. Manag. Rev., vol. 3, no. 1, pp. 46–62, 2018.
H. Zhou et al., “A Distributed Approach of Big Data Mining for Financial Fraud Detection in a Supply Chain,” Computers, Materials & Continua, vol. 64, no. 2, pp. 1091–1105, 2020, doi: 10.32604/cmc.2020.09834.
A. K. Rai and R. K. Dwivedi, “Fraud Detection in Credit Card Data using Unsupervised Machine Learning Based Scheme,” 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), pp. 421–426, Jul. 2020, doi: 10.1109/icesc48915.2020.9155615.
A. Keramati, H. Ghaneei, and S. M. Mirmohammadi, “Developing a prediction model for customer churn from electronic banking services using data mining,” Financial Innovation, vol. 2, no. 1, Aug. 2016, doi: 10.1186/s40854-016-0029-6.
F. Lv, J. Huang, W. Wang, Y. Wei, Y. Sun, and B. Wang, “A two-route CNN model for bank account classification with heterogeneous data,” PLOS ONE, vol. 14, no. 8, p. e0220631, Aug. 2019, doi: 10.1371/journal.pone.0220631.
I. Smeureanu, G. Ruxanda, and L. M. Badea, “CUSTOMER SEGMENTATION IN PRIVATE BANKING SECTOR USING MACHINE LEARNING TECHNIQUES,” Journal of Business Economics and Management, vol. 14, no. 5, pp. 923–939, Sep. 2013, doi: 10.3846/16111699.2012.749807.
F. N. Ogwueleka, S. Misra, R. Colomo-Palacios, and L. Fernandez, “Neural network and classification approach in identifying customer behavior in the banking sector: A case study of an international bank,” Human Factors and Ergonomics in Manufacturing & Service Industries, p. n/a-n/a, 2012, doi: 10.1002/hfm.20398.
M. M. Ismail and M. A. Haq, “Enhancing Enterprise Financial Fraud Detection Using Machine Learning,” Engineering, Technology & Applied Science Research, vol. 14, no. 4, pp. 14854–14861, Aug. 2024, doi: 10.48084/etasr.7437.
R. P. Pillai and D. P. P. Latha, “Study on Application of Artificial Intelligence and Machine Learning in the Banking Sector for Fraud Detection and Prevention,” Machine Learning for Environmental Monitoring in Wireless Sensor Networks, pp. 359–382, Sep. 2024, doi: 10.4018/979-8-3693-3940-4.ch017.
A. U. Usman, S. B. Abdullahi, Y. Liping, B. Alghofaily, A. S. Almasoud, and A. Rehman, “Financial Fraud Detection Using Value-at-Risk with Machine Learning in Skewed Data,” IEEE Access, vol. 12, pp. 64285–64299, 2024, doi: 10.1109/access.2024.3393154.
N. Damanik and C.-M. Liu, “Advanced Fraud Detection: Leveraging K-SMOTEENN and Stacking Ensemble to Tackle Data Imbalance and Extract Insights,” IEEE Access, vol. 13, pp. 10356–10370, 2025, doi: 10.1109/access.2025.3528079.
R. Sabareesh, “AI-Driven Fraud Detection in Banking: Enhancing Transaction Security,” Journal of Informatics Education and Research, vol. 4, no. 3, Nov. 2024, doi: 10.52783/jier.v4i3.1707.
A. Vashistha and A. K. Tiwari, “Building Resilience in Banking Against Fraud with Hyper Ensemble Machine Learning and Anomaly Detection Strategies,” SN Computer Science, vol. 5, no. 5, May 2024, doi: 10.1007/s42979-024-02854-w.
R. Sharma, K. Mehta, and P. Sharma, “Role of Artificial Intelligence and Machine Learning in Fraud Detection and Prevention,” Risks and Challenges of AI-Driven Finance, pp. 90–120, Apr. 2024, doi: 10.4018/979-8-3693-2185-0.ch005.
Kamble, Vitthal B., et al. "Enhancing UPI Fraud Detection: A Machine Learning Approach Using Stacked Generalization." International Journal of Multidisciplinary on Science and Management 2.1 (2025): 69-83.
X. Zheng, “Suspicious Transaction Recognition Based on KNN Model for Credit Card Fraud Prevention,” Recent Advancements in Computational Finance and Business Analytics, pp. 516–524, 2024, doi: 10.1007/978-3-031-70598-4_47.
Udeh, Ezekiel Onyekachukwu, et al. "The role of big data in detecting and preventing financial fraud in digital transactions." World Journal of Advanced Research and Reviews 22.2 (2024): 1746-1760.
“Using random forest and artificial neural network to detect fraudulent financial reporting: Data from listed companies in Vietnam,” Quality-Access to Success, vol. 25, no. 202, Sep. 2024, doi: 10.47750/qas/25.202.17.
X. Hou, “Financial Abnormal Data Detection System Based on Reinforcement Learning,” Mobile Information Systems, vol. 2022, pp. 1–8, Aug. 2022, doi: 10.1155/2022/7358383.
P. Hajek, M. Z. Abedin, and U. Sivarajah, “Fraud Detection in Mobile Payment Systems using an XGBoost-based Framework,” Information Systems Frontiers, vol. 25, no. 5, pp. 1985–2003, Oct. 2022, doi: 10.1007/s10796-022-10346-6.
A. A. Arfeen and B. M. A. Khan, “Empirical Analysis of Machine Learning Algorithms on Detection of Fraudulent Electronic Fund Transfer Transactions,” IETE Journal of Research, vol. 69, no. 11, pp. 7920–7932, Mar. 2022, doi: 10.1080/03772063.2022.2048700.
A. Vashistha, A. K. Tiwari, P. Singh, P. K. Yadav, and S. Pandey, “A Robust Framework for fraud Detection in Banking using ML and NN,” Proceedings of the National Academy of Sciences, India Section A: Physical Sciences, vol. 94, no. 2, pp. 201–212, Feb. 2024, doi: 10.1007/s40010-024-00871-1.
A. Y. Shdefat, M. Mohamed, S. Khaled, F. Hany, H. Fathi, and D. S. AbdElminaam, “Comparative Analysis of Machine Learning Models in Online Payment Fraud Prediction,” 2024 Intelligent Methods, Systems, and Applications (IMSA), pp. 243–250, Jul. 2024, doi: 10.1109/imsa61967.2024.10652861.
E. M. Al‐dahasi, R. K. Alsheikh, F. A. Khan, and G. Jeon, “Optimizing fraud detection in financial transactions with machine learning and imbalance mitigation,” Expert Systems, vol. 42, no. 2, Jul. 2024, doi: 10.1111/exsy.13682.
S. E. Sorour, K. M. AlBarrak, A. A. Abohany, and A. A. A. El-Mageed, “Credit card fraud detection using the brown bear optimization algorithm,” Alexandria Engineering Journal, vol. 104, pp. 171–192, Oct. 2024, doi: 10.1016/j.aej.2024.06.040.
G. Yu and Z. Luo, “Financial fraud detection using a hybrid deep belief network and quantum optimization approach,” Discover Applied Sciences, vol. 7, no. 5, May 2025, doi: 10.1007/s42452-025-06999-y.
Md. A. Talukder, M. Khalid, and M. A. Uddin, “An integrated multistage ensemble machine learning model for fraudulent transaction detection,” Journal of Big Data, vol. 11, no. 1, Nov. 2024, doi: 10.1186/s40537-024-00996-5.
F. Sabry Esmail, F. Kamal Alsheref, and A. Elsayed Aboutabl, “Review of Loan Fraud Detection Process in the Banking Sector Using Data Mining Techniques,” International journal of electrical and computer engineering systems, vol. 14, no. 2, pp. 229–239, Feb. 2023, doi: 10.32985/ijeces.14.2.12.
M. N. Ashtiani and B. Raahemi, “Intelligent Fraud Detection in Financial Statements Using Machine Learning and Data Mining: A Systematic Literature Review,” IEEE Access, vol. 10, pp. 72504–72525, 2022, doi: 10.1109/access.2021.3096799.
M. Nalini, B. Yamini, F. M. H. Fernandez, and P. S. Uma Priyadarsini, “Enhancing anomaly detection Efficiency: Introducing grid searchbased multi-population particle Swarm optimization algorithm based optimized Regional based Convolutional neural network for robust and scalable solutions in High-Dimensional data,” Biomedical Signal Processing and Control, vol. 96, p. 106651, Oct. 2024, doi: 10.1016/j.bspc.2024.106651.
CRediT Author Statement
The authors confirm contribution to the paper as follows:
Conceptualization: Renukadevi S, Manujakshi B C, Shashidhar T M and Sivakumar N;
Writing- Original Draft Preparation: Renukadevi S, Manujakshi B C, Shashidhar T M and Sivakumar N;
Visualization: Renukadevi S and Manujakshi B C;
Investigation: Shashidhar T M and Sivakumar N;
Supervision: Renukadevi S and Manujakshi B C;
Validation: Shashidhar T M and Sivakumar N;
Writing- Reviewing and Editing: Renukadevi S, Manujakshi B C, Shashidhar T M and Sivakumar N; All authors reviewed the results and approved the final version of the manuscript.
Acknowledgements
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Funding
No funding was received to assist with the preparation of this manuscript.
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Availability of data and materials
Data sharing is not applicable to this article as no new data were created or analysed in this study.
Author information
Contributions
All authors have equal contribution in the paper and all authors have read and agreed to the published version of the manuscript.
Corresponding author
Renukadevi S
School of Computer Science and Information Technology, Jain (Deemed-to-be-University), Bengaluru, Karnataka, India.
Open Access This article is licensed under a Creative Commons Attribution NoDerivs is a more restrictive license. It allows you to redistribute the material commercially or non-commercially but the user cannot make any changes whatsoever to the original, i.e. no derivatives of the original work. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
Cite this article
Renukadevi S, Manujakshi B C, Shashidhar T M and Sivakumar N, “Fraud Detection in Financial Transactions Using Gradient Boost with Hybrid Optimization”, Journal of Machine and Computing, vol.5, no.4, pp. 2328-2344, October 2025, doi: 10.53759/7669/jmc202505181.
