Voice Phishing (vishing) Detection System

References

[1] K. Chen, X. Zhang, and Y. Liu, "Voice Phishing Detection Using Machine Learning Techniques," IEEE Access, vol. 8, pp. 123456–123468, 2020.

[2] T. Nguyen, H. Le, and D. Pham, "Detecting Vishing Attacks in Real-Time Using Deep Neural Networks," Journal of Information Security and Applications, vol. 58, p. 102757, 2021.

[3] S. Sharma, R. Kumar, and A. Verma, "A Survey on Voice Phishing (Vishing) Attack Detection Methods," International Journal of Computer Applications, vol. 175, no. 35, pp. 20–27, 2020.

[4] J. Zhang, L. Li, and M. Xu, "AI-based Detection of Fraudulent Phone Calls for Vishing Prevention," Procedia Computer Science, vol. 187, pp. 210–218, 2021.

[5] M. Alshammari and A. Alharbi, "Real-Time Voice Phishing Detection Using Audio Feature Extraction and Machine Learning," IEEE International Conference on Big Data, pp. 450–457, 2022.

[6] A. Majumder, S. Pal, and P. Bhattacharya, "Voice Phishing Detection System Using NLP and Audio Signal Analysis," Computers, Materials and Continua, vol. 71, no. 3, pp. 5411–5427, 2022.

[7] H. Dai, Z. Liu, and W. Liao, "AugGPT: Leveraging ChatGPTfor Text Data Augmentation," arXiv preprint, arXiv:2302.13007, 2023.

[8] Y. Fang, C. Qian, and J. Yu, "Compositional Sentence Generation for Improved Phishing Detection," IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1234–1245, 2023.

[9] M. Boussougou and D. Park, "Hybrid FastText and CNN-BiLSTM Architecture for Voice Phishing Detection," Applied Sciences, vol. 11, no. 14, p. 6353, 2021.

[10] S. Lee, J. Kim, and H. Park, "Real-Time Voice Phishing Detection Using Speech-to-Text and Machine Learning Classification," Electronics, vol. 11, no. 6, p. 895, 2022.

[11] Z. Zhang, Y. Wang, and X. Liu, "Synthetic Speech Detection Using MFCC and Spectrogram Features with Deep Learning," IEEE Signal Processing Letters, vol. 28, pp. 1710–1714, 2021.

[12] A. Triantafyllopoulos, B. Schuller, and Z. Ping, "Foundation Models and Multitask Learning for Robust Audio Analysis," IEEE Transactions on Audio, Speech and Language Processing, vol. 30, pp. 2345–2358, 2022.

[13] A. Majumder, S. Pal, and P. Bhattacharya, "Voice Phishing Detection System Using NLP and Audio Signal Analysis," Computers, Materials and Continua, vol. 71, no. 3, pp. 5411–5427, 2022.

[14] H. Dai, Z. Liu, and W. Liao, "AugGPT: Leveraging ChatGPTfor Text Data Augmentation," arXiv preprint, arXiv:2302.13007, 2023.

[15] Y. Fang, C. Qian, and J. Yu, "Compositional Sentence Generation for Improved Phishing Detection," IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1234–1245, 2023.

[16] M. Boussougou and D. Park, "Hybrid FastText and CNN-BiLSTM Architecture for Voice Phishing Detection," Applied Sciences, vol. 11, no. 14, p. 6353, 2021.

[17] S. Lee, J. Kim, and H. Park, "Real-Time Voice Phishing Detection Using Speech-to-Text and Machine Learning Classification," Electronics, vol. 11, no. 6, p. 895, 2022.

[18] Z. Zhang, Y. Wang, and X. Liu, "Synthetic Speech Detection Using MFCC and Spectrogram Features with Deep Learning," IEEE Signal Processing Letters, vol. 28, pp. 1710–1714, 2021.

[19] A. Triantafyllopoulos, B. Schuller, and Z. Ping, "Foundation Models and Multitask Learning for Robust Audio Analysis," IEEE Transactions on Audio, Speech and Language Processing, vol. 30, pp. 2345–2358, 2022.

@article{1716820,
      author = {Koncha Harshitha, Konduru Yaswanth Raju, Harshith G H, K V V Yashwanth, Nermitha S},
      title = {Voice Phishing (vishing) Detection System},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {9},
      number = {10},
      pages = {3251-3256},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1716820.pdf},
      abstract = {With the rapid advancement of AI-powered communication tools, voice phishing — commonly known as vishing — has emerged as one of the most deceptive forms of cybercrime. Attackers impersonate trusted entities over phone calls to manipulate victims into disclosing sensitive data such as banking credentials, OTPs, and personal identification details. Conventional defenses like caller ID screening and spam number databases are no longer adequate, as they fail to analyze the actual content and emotional tone of a conversation. This paper presents a multimodal AI system that addresses this gap by examining both the acoustic properties and linguistic content of a phone call. Audio input is first transcribed using Whisper ASR, after which TF-IDF-based Naive Bayes classification is applied on the transcript for textlevel phishing detection. Simultaneously, MFCC features extracted from the raw audio are fed into a Random Forest classifier to detect emotional cues such as urgency and stress. A fusion decision engine consolidates both outputs to produce a final verdict — Vishing or Safe — along with a confidence score. The system is deployed as a Streamlit web application. Experimental results show spam detection accuracy of 98.7%, emotion detection accuracy of 69.8%, and an overall system accuracy in the range of 90–95%.},
      keywords = {Voice Phishing, Vishing Detection, Whisper ASR, Natural Language Processing, TFIDF, Naive Bayes, MFCC, Random Forest, Emotion Detection, Multimodal Fusion, Cybersecurity, Fraud Detection.},
      month = {April},
      doi = {https://doi.org/10.64388/IREV9I10-1716820}
  }