Temporal and Spatial Fusion for Breast Lesion Classification in Ultrasound Videos using Hybrid UNet-LSTM Architecture

References

[1] Lin, Z., Lin, J., Zhu, L., Fu, H., Qin, J., & Wang, L. (2022) 'A New Dataset and a Baseline Model for Breast Lesion Detection in Ultrasound Videos', in *Lecture Notes in Computer Science*. Available at: https://doi.org/10.1007/978-3-031-16437-8_59.

[2] Berg, W.A., Zhang, Z., Lehrer, D., Jong, R.A., Pisano, E.D., Barr, R.G., Böhm-Vélez, M., Mahoney, M.C., Evans, W.P. 3rd, Larsen, L.H., Morton, M.J., Mendelson, E.B., Farria, D.M., Cormack, J.B., Marques, H.S., Adams, A., Yeh, N.M., Gabrielli, G., ACRIN 6666 Investigators (2012) 'Detection of breast cancer with addition of annual screening ultrasound or a single screening MRI to mammography in women with elevated breast cancer risk', *JAMA*, 307(13), pp. 1394-1404. doi: 10.1001/jama.2012.388.

[3] Giger, M.L., Chan, H.P., & Boone, J. (2008) 'Anniversary paper: History and status of CAD and quantitative image analysis: the role of Medical Physics and AAPM', *Medical Physics*, 35(12), pp. 5799-5820. doi: 10.1118/1.3013555.

[4] Jemal, A., Bray, F., Center, M.M., Ferlay, J., Ward, E., & Forman, D. (2011) 'Global cancer statistics', *CA: A Cancer Journal for Clinicians*, 61(2), pp. 69-90. doi: 10.3322/caac.20107.

[5] Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., van der Laak, J.A.W.M., van Ginneken, B., & Sánchez, C.I. (2017) 'A survey on deep learning in medical image analysis', *Medical Image Analysis*, 42, pp. 60-88. doi: 10.1016/j.media.2017.07.005.

[6] Shen, D., Wu, G., & Suk, H.I. (2017) 'Deep Learning in Medical Image Analysis', *Annual Review of Biomedical Engineering*, 19, pp. 221-248. doi: 10.1146/annurev-bioeng-071516-044442.

[7] Ronneberger, O., Fischer, P., & Brox, T. (2015) 'U-Net: Convolutional Networks for Biomedical Image Segmentation', in *Lecture Notes in Computer Science*. Available at: https://doi.org/10.1007/978-3-319-24574-4_28.

[8] Topol, E.J. (2019) 'High-performance medicine: the convergence of human and artificial intelligence', *Nature Medicine*, 25(1), pp. 44-56. doi: 10.1038/s41591-018-0300-7.

[9] European Parliament and Council (2016) *Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)*. Official Journal of the European Union, L119, pp. 1-88.

[10] Sree, S.V., Ng, E.Y., Acharya, R.U., & Faust, O. (2011) 'Breast imaging: A survey', *World Journal of Clinical Oncology*, 2(4), pp. 171-178. doi: 10.5306/wjco.v2.i4.171.

[11] Zhou, S.K., Greenspan, H., Davatzikos, C., Duncan, J.S., van Ginneken, B., Madabhushi, A., Prince, J.L., Rueckert, D., & Summers, R.M. (2021) 'A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises', *Proceedings of the IEEE*, 109(5), pp. 820-838. doi: 10.1109/JPROC.2021.3054390.

[12] Acharya, U.R., Ng, E.Y., Tan, J.H., & Sree, S.V. (2012) 'Thermography based breast cancer detection using texture features and Support Vector Machine', *Journal of Medical Systems*, 36(3), pp. 1503-1510. doi: 10.1007/s10916-010-9611-z.

[13] Horsch, K., Giger, M.L., Venta, L.A., & Vyborny, C.J. (2002) 'Computerized diagnosis of breast lesions on ultrasound', *Medical Physics*, 29(2), pp. 157-164. doi: 10.1118/1.1429239.

[14] Simonyan, K., & Zisserman, A. (2014) 'Very Deep Convolutional Networks for Large-Scale Image Recognition', *arXiv preprint*. Available at: https://arxiv.org/abs/1409.1556.

[15] He, K., Zhang, X., Ren, S., & Sun, J. (2016) 'Deep Residual Learning for Image Recognition', in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 770-778. doi: 10.1109/CVPR.2016.90.

[16] Huang, G., Liu, Z., van der Maaten, L., & Weinberger, K. (2017) 'Densely Connected Convolutional Networks', in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*. doi: 10.1109/CVPR.2017.243.

[17] Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R.M. (2016) 'Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning', *IEEE Transactions on Medical Imaging*, 35(5), pp. 1285-1298. doi: 10.1109/TMI.2016.2528162.

[18] Tajbakhsh, N., Shin, J.Y., Gurudu, S.R., Hurst, R.T., Kendall, C.B., Gotway, M.B., & Liang, J. (2016) 'Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?', *IEEE Transactions on Medical Imaging*, 35(5), pp. 1299-1312. doi: 10.1109/TMI.2016.2535302.

[19] Jetley, S., Lord, N., Lee, N., & Torr, P. (2018) 'Learn To Pay Attention', *arXiv preprint*. Available at: https://arxiv.org/abs/1804.02391.

[20] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., & Polosukhin, I. (2017) 'Attention Is All You Need', *arXiv preprint*. Available at: https://arxiv.org/abs/1706.03762.

[21] Hochreiter, S., & Schmidhuber, J. (1997) 'Long Short-term Memory', *Neural Computation*, 9(8), pp. 1735-1780. doi: 10.1162/neco.1997.9.8.1735.

[22] Zhao, T., Desjardins, A.E., Ourselin, S., Vercauteren, T., and Xia, W. (2019) 'Minimally invasive photoacoustic imaging: Current status and future perspectives', Photoacoustics, 16, p. 100146. doi: 10.1016/j.pacs.2019.100146.

[23] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017) 'MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications'. Available at: arXiv:1704.04861.

[24] Lea, C., Vidal, R., Reiter, A., and Hager, G.D. (2016) 'Temporal Convolutional Networks: A Unified Approach to Action Segmentation'. In: Hua, G., Jégou, H. (eds) Computer Vision – ECCV 2016 Workshops. Lecture Notes in Computer Science, vol 9915. Springer, Cham. doi: 10.1007/978-3-319-49409-8_7.

@article{1718691,
      author = {Sakthi Yazhini Jothi Latha},
      title = {Temporal and Spatial Fusion for Breast Lesion Classification in Ultrasound Videos using Hybrid UNet-LSTM Architecture},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {9},
      number = {12},
      pages = {552-580},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1718691.pdf},
      abstract = {This research involves the design, implementation, and evaluation of a Hybrid UNet-LSTM architecture aimed at enhancing the classification of breast lesions in ultrasound videos, which is critical for the early and accurate diagnosis of breast cancer. The model integrates Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) networks inspired from the U-Net architecture to capture both spatial and temporal features from video sequences. This approach addresses the inherent challenges of ultrasound imaging, such as speckle noise, operator dependency, and variability in lesion appearances, which often complicate diagnosis. The Hybrid UNet-LSTM model processes ultrasound videos, employing intra-video and inter-video fusion techniques to improve classification accuracy. The model is built using the latest advancements in deep learning, ensuring a robust and reliable system for medical diagnostics. The implementation of this Hybrid UNet-LSTM architecture serves as a foundational model that can be further improved upon to develop more advanced models in the future, enhancing breast cancer diagnostics by providing a more accurate and reliable method for lesion classification. This groundwork supports ongoing innovation in the field, with the potential to improve patient outcomes and assist radiologists in clinical decision-making.},
      month = {June},
      doi = {https://doi.org/10.64388/IREV9I12-1718691}
  }