Privacy-Preserving Medical Image Diagnosis using Federated Learning and Vision Transformers

References

[1] McMahan B, Moore E, Ramage D, Hampson S, Agüera y Arcas B. Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS). Proc Mach Learn Res. 2017;54:1273–1282.

[2] Li T, Sahu AK, Talwalkar A, Smith V. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine. 2020;37(3):50–60. https://doi.org/10.1109/MSP.2020.2975749

[3] Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V. Federated optimization in heterogeneous networks. In: Proceedings of Machine Learning and Systems (MLSys), vol. 2; 2020. p. 429–450.

[4] Rieke N, Hancox J, Li W, Milletarì F, Roth HR, Albarqouni S, et al. The future of digital health with federated learning. npj Digital Medicine. 2020;3:119. https://doi.org/10.1038/s41746-020-00323-1

[5] Yang Q, Liu Y, Chen T, Tong Y. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology. 2019;10(2):12. https://doi.org/10.1145/3298981

[6] Dwork C, Roth A. The algorithmic foundations of differential privacy. Hanover (MA): Now Publishers; 2014. https://doi.org/10.1561/0400000042

[7] Geyer RC, Klein T, Nabi M. Differentially private federated learning: A client level perspective. In: NeurIPS Workshop on Machine Learning on the Phone and Other Consumer Devices; 2017. arXiv:1712.07557.

[8] Abadi M, Chu A, Goodfellow I, McMahan HB, Mironov I, Talwar K, et al. Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS); 2016. p. 308–318. https://doi.org/10.1145/2976749.2978318

[9] Papernot N, Song S, Mironov I, Raghunathan A, Talwar K, Erlingsson Ú. Scalable private learning with PATE. In: International Conference on Learning Representations (ICLR); 2018.

[10] Bonawitz K, Ivanov V, Kreuter B, Marcedone A, McMahan HB, Patel S, et al. Practical secure aggregation for privacy-preserving machine learning. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS); 2017. p. 1175–1191. https://doi.org/10.1145/3133956.3133982

[11] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In: International Conference on Learning Representations (ICLR); 2021.

[12] Touvron H, Cord M, Douze M, Massa F, Sablayrolles A, Jégou H. Training data-efficient image transformers & distillation through attention. In: Proceedings of the 38th International Conference on Machine Learning (ICML). Proc Mach Learn Res. 2021;139:10347–10357.

[13] Hatamizadeh A, Tang Y, Nath V, Yang D, Myronenko A, Landman B, et al. UNETR: Transformers for 3D medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV); 2022. p. 1748–1758. https://doi.org/10.1109/WACV51458.2022.00181

[14] Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, et al. TransUNet: Transformers make strong encoders for medical image segmentation [preprint]. arXiv:2102.04306. 2021.

[15] Yang J, Shi R, Wei D, Liu Z, Zhao L, Ke B, et al. MedMNIST v2: A large-scale lightweight benchmark for 2D and 3D biomedical image classification. Scientific Data. 2023;10:41. https://doi.org/10.1038/s41597-022-01721-8

[16] Wang S, Tuor T, Salonidis T, Leung KK, Makaya C, He T, et al. Adaptive federated learning in resource constrained edge computing systems. IEEE Journal on Selected Areas in Communications. 2019;37(6):1205–1221. https://doi.org/10.1109/JSAC.2019.2904348

[17] Bonawitz K, Eichner H, Grieskamp W, Huba D, Ingerman A, Ivanov V, et al. Towards federated learning at scale: System design. In: Proceedings of Machine Learning and Systems (MLSys), vol. 1; 2019. p. 374–388.

[18] Li X, Jiang M, Zhang X, Kamp M, Dou Q. FedBN: Federated learning on non-IID features via local batch normalization. In: International Conference on Learning Representations (ICLR); 2021.

[19] Kairouz P, McMahan HB, Avent B, Bellet A, Bennis M, Bhagoji AN, et al. Advances and open problems in federated learning. Foundations and Trends in Machine Learning. 2021;14(1–2):1–210. https://doi.org/10.1561/2200000083

[20] Alistarh D, Hoefler T, Johansson M, Konstantinov N, Khirirat S, Renggli C. The convergence of sparsified gradient methods. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 31; 2018.

@article{1724791,
      author = {Ishita Awasthi, Anuj Sharma, Anurag Sharma, Ankita Sharma},
      title = {Privacy-Preserving Medical Image Diagnosis using Federated Learning and Vision Transformers},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {10},
      number = {5},
      pages = {1006-1017},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1724791.pdf},
      abstract = {Deep learning model training is a central approach, which poses risks to patient data privacy and hampers data sharing for medical image classification, essential for computer-assisted diagnosis. In this study, a privacy-preserving medical image classification system is presented that combines Vision Transformers (ViTs) with Federated Learning (FL) for the classification of four categories of white blood cells: eosinophils, lymphocytes, monocytes, and neutrophils. The framework uses model training on local data in three simulated clients and uses Federated Averaging (FedAvg) to merge model updates without sharing raw images. A conventional Convolutional Neural Network (CNN) and a centralized ViT are used as baseline models. The preliminary results indicate that the centralized ViT can attain an accuracy of 58.50% and a macro-F1 score of 51.82%, which is better than the CNN baseline (53.50% accuracy). The overall performance of the federated ViT can be seen as good with an accuracy score of 49.50% and a recall score of 96.00% for the class of monocytes, which suggests good class-specific recognition and, simultaneously, room for improvement at the overall level. An additional enhancement that can help prevent information leakage from model updates is proposed as differential privacy. Additional rounds of communication, larger sets of training data, hyperparameter optimization and formal guarantees of privacy will be explored in further work. This is a framework to give a basis for co-operative privacy-aware medical image classification.},
      keywords = {Federated learning; Vision Transformer; medical image classification; white blood cell classification; FedAvg; differential privacy; privacy-preserving machine learning},
      month = {November},
      doi = {https://doi.org/10.64388/ire1724791}
  }