An Enhanced Siamese Neural Network with Attention-Adam Optimization for Bimodal Biometric Authentication in Mobile Financial Systems

References

[1] Abderrahmane, H., Noubeil, G., Lahcene, Z., Akhtar, Z., & Dasgupta, D. (2020). Weighted quasi-arithmetic mean based score level fusion for multi-biometric systems. IET Biometrics, 9(3), 91–99. Wiley

[2] Akhtar, Z., Micheloni, C., & Foresti, G. L. (2015). Biometric liveness detection: Challenges and research opportunities. IEEE Security & Privacy, 13(5), 63–72. IEEE

[3] Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. IEEE

[4] Bhavani, S. A., & Karthikeyan, C. (2025). An attention based deep learning with effective SVM-ConvFaceNeXt model for face recognition in unconstrained environment. Signal, Image and Video Processing, 19(12). Springer

[5] Bromley, J., et al. (1993). Signature verification using a “Siamese” time delay neural network. International Journal of Pattern Recognition and Artificial Intelligence, 7(4), 669–688. World Scientific

[6] Chopra, S., Hadsell, R., & LeCun, Y. (2005). Learning a similarity metric discriminatively, with application to face verification. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE

[7] Deng, J., Guo, J., Xue, N., & Zafeiriou, S. (2019). ArcFace: Additive angular margin loss for deep face recognition. arXiv:1801.07698. arXiv

[8] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.

[9] Hadsell, R., Chopra, S., & LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE

[10] Hall, D. L., & Llinas, J. (1997). An introduction to multisensor data fusion. Proceedings of the IEEE, 85(1), 6–23. IEEE

[11] He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. arXiv:1512.03385. arXiv

[12] Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861. arXiv

[13] Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., & Kalenichenko, D. (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE

[14] Jain, A. K., Nandakumar, K., & Nagar, A. (2008). Biometric template security. EURASIP Journal on Advances in Signal Processing, 2008, 579416. Springer

[15] Juels, A., & Sudan, M. (2006). A fuzzy vault scheme. Designs, Codes and Cryptography, 38(2), 237–257. Springer

[16] Koch, G., Zemel, R., & Salakhutdinov, R. (2015). Siamese neural networks for one-shot image recognition. Carnegie Mellon University

[17] Kumar, C. R., N., S., Priyadharshini, M., E., D. G., & M., K. R. (2023). Face recognition using CNN and siamese network. Measurement: Sensors, 27, 100800. ScienceDirect

[18] LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. Nature

[19] Liu, W., & Pan, Y. (2024). Spatio-temporal-based action face anti-spoofing detection via fusing dynamics and texture face keypoints cues. IEEE Transactions on Consumer Electronics, 70(1), 2401–2413. IEEE

[20] Ma, H., Guo, D., Wang, J., Li, P., Xu, C., & Wang, Y. (2025). Real-time face recognition algorithm based on lightweight neural network in the field of computer vision. Discover Applied Sciences.

[21] Maltoni, D., Maio, D., Jain, A. K., & Prabhakar, S. (2009). Handbook of fingerprint recognition. Springer. Springer

[22] Naji Ali, H., & Mahmood Al-Dabbagh, S. S. (2026). A systematic literature review on biometric authentication in mobile banking. F1000Research, 15, 5. F1000Research

[23] Ratha, N. K., Chikkerur, S., Connell, J. H., & Bolle, R. M. (2007). Generating cancelable fingerprint templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(4), 561–572. IEEE

[24] Ratha, N. K., Connell, J. H., & Bolle, R. M. (2001). Enhancing security and privacy in biometrics-based authentication systems. IBM Systems Journal, 40(3), 614–634. IBM Research

[25] Ross, A., & Jain, A. K. (2003). Information fusion in biometrics. Pattern Recognition Letters, 24, 2115–2125. ScienceDirect

[26] Ross, A., & Jain, A. K. (2004). Multimodal biometrics: An overview. In Proceedings of the 2004 International Symposium on Electronics in the 21st Century.

[27] Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 4510–4520). IEEE

[28] Setyawan, N., Sun, C.-C., Hsu, M.-H., Kuo, W.-K., & Hsieh, J.-W. (2025). FaceLiVT: Face recognition using linear vision transformer with structural reparameterization for mobile device. In 2025 IEEE International Conference on Image Processing (pp. 1720–1725). IEEE

[29] Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014). DeepFace: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE

[30] U. Sumalatha, K., Krishna Prakasha, K., Prabhu, S., & Nayak, V. C. (2024). A comprehensive review of unimodal and multimodal fingerprint biometric authentication systems: Fusion, attacks, and template protection. IEEE Access. IEEE

[31] Warden, P., & Situnayake, D. (2019). TinyML: Machine learning with TensorFlow Lite on Arduino and ultra-low-power microcontrollers. O’Reilly Media.

[32] Yuan, B., Du, C.-Q., & Li, Z.-T. (2026). Mobi-FaceNeXt: Research on an efficient face recognition algorithm based on lightweight convolutional neural networks. Thermal Science, 30(2 Part A), 1191–1201. Crossref

[33] Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10), 1499–1503. IEEE

@article{1723171,
      author = {Aluko, Tolulope Olugbemiga, Ayeni Josua Ayobami, Makinde Oladayo Ezekiel},
      title = {An Enhanced Siamese Neural Network with Attention-Adam Optimization for Bimodal Biometric Authentication in Mobile Financial Systems},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {10},
      number = {3},
      pages = {2024-2036},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1723171.pdf},
      abstract = {The increasing use of mobile financial services has intensified the need for authentication mechanisms that combine security, usability, and computational efficiency on resource-constrained devices. Passwords and unimodal biometric systems can expose mobile financial applications to credential compromise, presentation attacks, environmental variability, and single-modality failure. This study proposes an enhanced bimodal biometric authentication framework that combines an Enhanced Siamese Neural Network (ESNN) for facial verification with hardware-backed fingerprint authentication. The facial model uses a ResNet-18 backbone enhanced with Squeeze-and-Excitation (SE) blocks for channel-wise feature recalibration. A novel Attention-Adam (Attn-Adam) optimizer is formulated using a sliding window of historical gradients and temporal attention weights to emphasize consistent optimization directions and reduce the effect of noisy gradient updates. Facial data were obtained from the Labeled Faces in the Wild (LFW) dataset, comprising 13,233 images from 5,749 identities. The thesis reports a 70:20:10 training-validation-test split and balanced positive and negative contrastive pairs. The proposed system achieved an ROC-AUC of 0.9892, EER of 4.20%, precision of 95.86%, recall of 95.74%, and F1-score of 95.80%. In the reported ablation study, the baseline ResNet-18 with standard Adam achieved AUC = 0.9152 and EER = 11.24%; adding SE blocks improved performance to AUC = 0.9581 and EER = 7.52%; and the complete ESNN with Attn-Adam achieved AUC = 0.9892 and EER = 4.20%. The model was converted to TensorFlow Lite using post-training Int8 quantization, with the reported model size decreasing from approximately 45 MB to 11 MB. The mobile authentication workflow uses serial face-then-fingerprint verification and Android Keystore/Trusted Execution Environment support for cryptographic binding. The findings indicate that combining residual feature learning, channel attention, and temporal gradient attention can improve facial verification performance while supporting deployment in mobile financial environments.},
      keywords = {bimodal biometrics; siamese neural network; attention-adam; resnet-18; squeeze-and-excitation; facial verification; edge ai; mobile financial security},
      month = {September},
  }