Adversarial Robustness in Transfer Learning Models

References

[1] Pan, S. J., and Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.

[2] Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014). How trans- ferable are features in deep neural networks? In Advances in Neural Information Processing Systems (NeurIPS), 3320–3328.

[3] Szegedy, C., Zaremba, W., Sutskever, I., et al. (2014). Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR).

[4] Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR).

[5] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations (ICLR).

[6] Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto.

[7] Moosavi-Dezfooli, S.-M., Fawzi, A., and Frossard, P. (2016). Deep- Fool: A simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2574–2582.

[8] Deng, J., Dong, W., Socher, R., et al. (2009). ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 248–255.

[9] Wang, X., Peng, Y., Lu, L., et al. (2017). ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classifica- tion and localization of common thorax diseases. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3462–3471.

[10] He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778.

[11] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre- training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 4171–4186.

[12] Houlsby, N., Giurgiu, A., Jastrzebski, S., et al. (2019). Parameter- efficient transfer learning for NLP. In Proceedings of the International Conference on Machine Learning (ICML), 2790–2799.

[13] Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K. Q. (2017). Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4700–4708.

[14] Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR).

[15] Salman, H., Ilyas, A., Engstrom, L., Kapoor, A., and Madry, A. (2020). Adversarially robust transfer learning. In Advances in Neural Information Processing Systems (NeurIPS), 33, 15458–15470.

[16] Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A. (2019). Robustness may be at odds with accuracy. arXiv preprint arXiv:1805.12152.

[17] Shafahi, A., et al. (2019). Adversarially robust transfer learning. In Pro- ceedings of the International Conference on Learning Representations (ICLR).

[18] Hendrycks, D., et al. (2019). Using self-supervised learning can improve model robustness and uncertainty. In Advances in Neural Information Processing Systems (NeurIPS), 15637–15648.

[19] Chen, D., Hu, H., Wang, Q., et al. (2021). Cooperative adversarially- robust transfer learning. arXiv preprint arXiv:2106.06667.

[20] Paszke, A., Gross, S., Massa, F., et al. (2019). PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), 8024–8035.

[21] Rauber, J., Brendel, W., and Bethge, M. (2017). Foolbox: A Python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131.

@article{1703680,
      author = {Praveen Kumar Myakala},
      title = {Adversarial Robustness in Transfer Learning Models},
      journal = {Iconic Research And Engineering Journals},
      year = {2022},
      volume = {6},
      number = {1},
      pages = {772-779},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1703680.pdf},
      abstract = {Transfer learning has become a cornerstone tech- nique for adapting pre-trained models to diverse downstream tasks, significantly reducing data requirements. However, the extent to which adversarial robustness is retained or degraded during transfer learning remains unclear. This study systemati- cally evaluates the adversarial vulnerabilities of transfer learning models across fine-tuning strategies, such as full fine-tuning, layer freezing, and feature extraction. Our experiments, conducted on benchmark datasets, reveal that adversarial pretraining improves robustness by up to 25% under Projected Gradient Descent (PGD) and Fast Gradient Sign Method (FGSM) attacks com- pared to standard fine-tuning approaches. Additionally, freezing batch normalization layers during fine-tuning preserves robust- ness, likely due to the stabilization of learned feature distributions and prevention of gradient amplification. This study provides actionable insights for designing transfer learning pipelines that are not only accurate but also robust against adversarial threats, with implications for applications in healthcare, autonomous systems, and finance.},
      keywords = {Transfer Learning, Adversarial Robustness, Fine-tuning, Adversarial Attacks, Batch Normalization, Deep Learning},
      month = {July},
  }