Explainable AI in Medical Decision-Making: Challenges and Opportunities

References

[1] Saraswat, D., Bhattacharya, P., Verma, A., Prasad, V. K., Tanwar, S., Sharma, G., ... & Sharma,

[2] R. (2022). Explainable AI for healthcare 5.0: opportunities and challenges. IEEe Access, 10, 84486-84517.

[3] Das, A., & Rad, P. (2020). Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371.

[4] Hulsen, T. (2023). Explainable artificial intelligence (XAI): concepts and challenges in healthcare. AI, 4(3), 652-666.

[5] Wani, N. A., Kumar, R., Bedi, J., & Rida, I. (2024). Explainable AI-driven IoMT fusion: Unravelling techniques, opportunities, and challenges with Explainable AI in healthcare. Information Fusion, 102472.

[6] Korica, P., Gayar, N. E., & Pang, W. (2021, November). Explainable artificial intelligence in healthcare: Opportunities, gaps and challenges and a novel way to look at the problem space.

[7] In International conference on intelligent data engineering and automated learning (pp. 333- 342). Cham: Springer International Publishing.

[8] Hossain, M. I., Zamzmi, G., Mouton, P. R., Salekin, M. S., Sun, Y., & Goldgof, D. (2025). Explainable AI for medical data: current methods, limitations, and future directions. ACM Computing Surveys, 57(6), 1-46.

[9] Patidar, N., Mishra, S., Jain, R., Prajapati, D., Solanki, A., Suthar, R., ... & Patel, H. (2024). Transparency in AI decision making: A survey of explainable AI methods and applications. Advances of Robotic Technology, 2(1).

[10] Amann, J., Blasimme, A., Vayena, E., Frey, D., Madai, V. I., & Precise4Q Consortium. (2020). Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC medical informatics and decision making, 20, 1-9.

[11] De Bruijn, H., Warnier, M., & Janssen, M. (2022). The perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making. Government information quarterly, 39(2), 101666.

[12] Amann, J., Vetter, D., Blomberg, S. N., Christensen, H. C., Coffee, M., Gerke, S., ... & Z- Inspection Initiative. (2022). To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems. PLOS Digital Health, 1(2), e0000016.

[13] Pierce, R. L., Van Biesen, W., Van Cauwenberge, D., Decruyenaere, J., & Sterckx, S. (2022). Explainability in medicine in an era of AI-based clinical decision support systems. Frontiers in genetics, 13, 903600.

[14] Grover, V., & Dogra, M. (2024). Challenges and Limitations of Explainable AI in Healthcare. In Analyzing Explainable AI in Healthcare and the Pharmaceutical Industry (pp. 72-85). IGI Global.

[15] Abiodun, K. M., Awotunde, J. B., Aremu, D. R., & Adeniyi, E. A. (2022). Explainable AI for fighting COVID-19 pandemic: Opportunities, challenges, and future prospects. Computational Intelligence for COVID-19 and Future Pandemics: Emerging Applications and Strategies, 315-332.

[16] Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., & Zhu, J. (2019). Explainable AI: A brief survey on history, research areas, approaches and challenges. In Natural language processing and Chinese computing: 8th cCF international conference, NLPCC 2019, dunhuang, China, October 9–14, 2019, proceedings, part II 8 (pp. 563-574). Springer International Publishing.

[17] Belghachi, M. (2023). A review on explainable artificial intelligence methods, applications, and challenges. Indonesian Journal of Electrical Engineering and Informatics (IJEEI), 11(4), 1007-1024.

[18] Szymanski, M., Verbert, K., & Vanden Abeele, V. (2022, September). Designing and evaluating explainable AI for non-AI experts: challenges and opportunities. In Proceedings of the 16th ACM Conference on Recommender Systems (pp. 735-736).

[19] Erdeniz, S. P., Tran, T. N. T., Felfernig, A., Lubos, S., Schrempf, M., Kramer, D., & Rainer,

[20] P. P. (2023, December). Employing nudge theory and persuasive principles with explainable ai in clinical decision support. In 2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 2983-2989). IEEE.

[21] van Leersum, C. M., & Maathuis, C. (2025). Human centred explainable AI decision-making in healthcare. Journal of Responsible Technology, 21, 100108.

[22] Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Del Ser, J., ... & Stumpf, S. (2024). Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusion, 106, 102301.

[23] Srinivasu, P. N., Sandhya, N., Jhaveri, R. H., & Raut, R. (2022). From blackbox to explainable AI in healthcare: existing tools and case studies. Mobile Information Systems, 2022(1), 8167821.

[24] Rachha, A., & Seyam, M. (2023). Explainable AI in education: Current trends, challenges, and opportunities. SoutheastCon 2023, 232-239.

[25] Simuni, G. (2024). Explainable AI in Ml: The path to Transparency and Accountability. International Journal of Recent Advances in.

[26] Holzinger, A., Biemann, C., Pattichis, C. S., & Kell, D. B. (2017). What do we need to build explainable AI systems for the medical domain?. arXiv preprint arXiv:1712.09923.

[27] Sindiramutty, S. R., Tee, W. J., Balakrishnan, S., Kaur, S., Thangaveloo, R., Jazri, H., ... & Manchuri, A. R. (2024). Explainable AI in healthcare application. In Advances in Explainable AI Applications for Smart Cities (pp. 123-176). IGI Global Scientific Publishing.

[28] Sanwar, A. S. M. (2024). Explainable artificial intelligence into cyber-physical system architecture of smart cities: technologies, challenges, and opportunities. J Electr Syst, 20(2), 2343-2362.

[29] Panigutti, C., Beretta, A., Fadda, D., Giannotti, F., Pedreschi, D., Perotti, A., & Rinzivillo, S. (2023). Co-design of human-centered, explainable AI for clinical decision support. ACM Transactions on Interactive Intelligent Systems, 13(4), 1-35.

[30] Yang, W., Wei, Y., Wei, H., Chen, Y., Huang, G., Li, X., ... & Kang, B. (2023). Survey on explainable AI: From approaches, limitations and applications aspects. Human-Centric Intelligent Systems, 3(3), 161-188

@article{1703509,
      author = {Hassan Tanveer, Muhammad Faheem, Arbaz Haider Khan},
      title = {Explainable AI in Medical Decision-Making: Challenges and Opportunities},
      journal = {Iconic Research And Engineering Journals},
      year = {2022},
      volume = {5},
      number = {12},
      pages = {423-435},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1703509.pdf},
      abstract = {The use of artificial intelligence in medical decision-making has thus far proved beneficial. It has improved diagnostic accuracy, patient monitoring, and treatment planning. The accessibility of AI-driven systems has met with some resistance in health care mainly because of the nontransparent nature of many machine learning models, which are quite commonly dubbed black- box models. One of the intentions of explainable AI has been to enhance the interpretability and transparency of AI-driven decisions so as to form a basis of trust in them with clinicians and patients alike. Unfortunately, various challenges have curtailed the practical use of XAI in medicine, such as trading model accuracy against explainability, conflicting complexities and variability of medical data, lack of common evaluation metrics, and even ethical and regulatory issues. Then again, the active resistance of the medical professions would rather discourage large- scale adoption of this technology based on AI's unreliable clinical representations. The hybrid models for trustworthy AI, the possible design of standardized frameworks for explainability, and the enhanced emphasis on the integration of AI literacy within medical training as a means of increasing trustworthiness and usability of AI-driven health care are bright opportunities for the way forward. Furthermore, regulatory and policy reforms questioning explicability could reinforce XAI's use in the medical decision process. Based on these factors, this research shows that a balancing act is warranted to ensure AI models remain interpretable in real-time and clinically applicable in predefined medical contexts. Future efforts would be directed toward creating human-centered AI models which ensure medicolegal clarity, transparency, accountability, and ethical consideration in medical decision-making, thereby addressing the commonly held belief that AI becomes an element of patient outcomes and clinician trustworthiness.},
      keywords = {Explainable AI (XAI), Medical Decision-Making, Interpretability, Transparency, Ethical AI, Machine Learning in Healthcare, AI Trust, Regulatory Compliance, Hybrid AI Models.},
      month = {June},
  }