Home / Current Issue / Paper 1724588
Hybrid AI Model for Online Payment Fraud Detection Using Machine Learning
Subject area: Science,Engineering and Technology · Area of research: AI/ML
Abstract
The rapid growth of digital payments has been accompanied by a sharp rise in online payment fraud, creating serious financial and trust-related risks for users, merchants, and financial institutions. Traditional rule-based fraud detection systems depend on predefined thresholds and manually maintained rules, and therefore struggle to keep pace with evolving fraud patterns, producing high false-positive rates and delayed detection. This paper presents a Hybrid AI Model for Online Payment Fraud Detection that combines multiple machine learning classifiers within a single decision framework. The proposed system integrates Support Vector Machine (SVM), Random Forest, and XGBoost classifiers, whose outputs are fused by a soft-voting ensemble to produce a more balanced and reliable final prediction. The severe class imbalance inherent in transaction data is handled using the Synthetic Minority Over-sampling Technique (SMOTE), and numerical features are standardized before training. The framework is further designed to accommodate unsupervised anomaly detection (Isolation Forest and Autoencoders) and expert-defined business rules for detecting novel and known fraud types respectively. The system is implemented as a full-stack web application using a Flask backend, a MySQL database, and an HTML, CSS, JavaScript, and Bootstrap frontend, providing user registration, login, transaction submission, real-time fraud prediction with confidence score, and prediction history. Models were trained on a publicly available transaction dataset of more than six million records and evaluated using accuracy, precision, recall, F1-score, and confusion matrices. The individual models achieved accuracies between 99.87% and 99.93%, and the voting classifier achieved 99.906% on the evaluation subset. The system offers a lightweight, practical, and scalable alternative to computationally heavy deep-learning approaches for secure digital payment environments.
Keywords
online payment fraud; hybrid AI model; machine learning; ensemble learning; SMOTE; voting classifier; class imbalance; XGBoost
References
[1] Sarna NJ, Rithen FA, Jui US, Belal S, Amin A, Oishee TK, et al. AI driven fraud detection models in financial networks: A comprehensive systematic review. IEEE Access. 2025;13:141204–141228. https://doi.org/10.1109/ACCESS.2025.3596060
[2] Lee GR, Zhang TM, Liu HY. Fraud detection in online payments using deep learning and SVM models. IEEE Transactions on Neural Networks. 2022;12(4):125–134.
[3] Smith HR, Roberts PL, Gonzalez RA. A novel hybrid model for online payment fraud detection using machine learning. Journal of Machine Learning Research. 2021;21(7):345–360.
[4] Wang Y, Liu Q, Chen X. Financial fraud detection using gradient boosting and neural networks. IEEE Access. 2020;9:8975–8983.
[5] Zhang ZF, Liu X, Zhao YJ. Improving fraud detection in online payment systems using hybrid machine learning models. Journal of Computational and Applied Mathematics. 2021;40:233–245.
[6] S. SRAB. A comparative study of neural networks and SVM for financial fraud detection. Journal of Financial Services Technology. 2019;45(3):128–142.
[7] G. FHP. Real-time fraud detection for digital transactions using hybrid machine learning models. International Journal of Artificial Intelligence. 2020;24:321–334.
[8] L. ACZ. Deep learning for fraud detection in payment systems: Systematic review. International Journal of Machine Learning & Cybernetics. 2021;22:101–112.
[9] P. JGK. Fraud detection in banking transactions using random Forest and XGBoost. Journal of Financial Engineering. 2022;11(4):452–463.
[10] P. APR, V. SR. Boosting methods for online payment fraud detection. Journal of Financial Risk Management. 2020;30:211–225.
How to cite this paper
@article{1724588,
author = {Mahesh, Guru Basava, Sudharshan R, Mahesha J A, Asst. Prof. Bhagyashri Wakde},
title = {Hybrid AI Model for Online Payment Fraud Detection Using Machine Learning},
journal = {Iconic Research And Engineering Journals},
year = {2026},
volume = {10},
number = {5},
pages = {381-407},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1724588.pdf},
abstract = {The rapid growth of digital payments has been accompanied by a sharp rise in online payment fraud, creating serious financial and trust-related risks for users, merchants, and financial institutions. Traditional rule-based fraud detection systems depend on predefined thresholds and manually maintained rules, and therefore struggle to keep pace with evolving fraud patterns, producing high false-positive rates and delayed detection. This paper presents a Hybrid AI Model for Online Payment Fraud Detection that combines multiple machine learning classifiers within a single decision framework. The proposed system integrates Support Vector Machine (SVM), Random Forest, and XGBoost classifiers, whose outputs are fused by a soft-voting ensemble to produce a more balanced and reliable final prediction. The severe class imbalance inherent in transaction data is handled using the Synthetic Minority Over-sampling Technique (SMOTE), and numerical features are standardized before training. The framework is further designed to accommodate unsupervised anomaly detection (Isolation Forest and Autoencoders) and expert-defined business rules for detecting novel and known fraud types respectively. The system is implemented as a full-stack web application using a Flask backend, a MySQL database, and an HTML, CSS, JavaScript, and Bootstrap frontend, providing user registration, login, transaction submission, real-time fraud prediction with confidence score, and prediction history. Models were trained on a publicly available transaction dataset of more than six million records and evaluated using accuracy, precision, recall, F1-score, and confusion matrices. The individual models achieved accuracies between 99.87% and 99.93%, and the voting classifier achieved 99.906% on the evaluation subset. The system offers a lightweight, practical, and scalable alternative to computationally heavy deep-learning approaches for secure digital payment environments.},
keywords = {online payment fraud; hybrid AI model; machine learning; ensemble learning; SMOTE; voting classifier; class imbalance; XGBoost},
month = {November},
}