Autonomous Data Mining Systems for Real-Time Big Data Streams in Edge AI: A Comprehensive Survey

References

[1] Hemmati, A., Raoufi, P., & Rahmani, A. M. (2024). Edge artificial intelligence for big data: a systematic review. Neural Computing and Applications, 36(19), 11461-11494.

[2] Li, L., Shao, W., Dong, W., Tian, Y., Zhang, Q., Yang, K., & Zhang, W. (2024). Data-centric evolution in autonomous driving: A comprehensive survey of big data system, data mining, and closed-loop technologies. arXiv preprint arXiv:2401.12888.

[3] Abeyratne, D. (2024). Real-time streaming analytics and latency minimization in autonomous vehicle big data pipelines. Northern Reviews on Smart Cities, Sustainable Engineering, and Emerging Technologies, 9(11), 49-62.

[4] Veluru, S. P. (2022). Streaming Data Pipelines for AI at the Edge: Architecting for Real-Time Intelligence. International Journal of Artificial Intelligence, Data Science, and Machine Learning, 3(2), 60-68.

[5] Jihong, X. I. E., & Xiang, Z. H. O. U. (2024). Edge Computing for Real-Time Decision Making in Autonomous Driving: Review of Challenges, Solutions, and Future Trends. International Journal of Advanced Computer Science & Applications, 15(7).

[6] Zhong, Y., Chen, L., Dan, C., & Rezaeipanah, A. (2022). A systematic survey of data mining and big data analysis in internet of things. The Journal of Supercomputing, 78(17), 18405-18453.

[7] Do, T. T. T., Huynh, Q. T., Kim, K., & Nguyen, V. Q. (2025). A survey on video big data analytics: architecture, technologies, and open research challenges. Applied Sciences, 15(14), 8089.

[8] Alam, M. A., Nabil, A. R., Mintoo, A. A., & Islam, A. (2024). Real-time analytics in streaming big data: techniques and applications. Journal of Science and Engineering Research, 1(01), 104-122.

[9] Rozony, F. Z. (2024). A Comprehensive Review Of Real-Time Analytics Techniques And Applications In Streaming Big Data. Available at SSRN 5256050.

[10] Chang, Z., Liu, S., Xiong, X., Cai, Z., & Tu, G. (2021). A survey of recent advances in edge-computing-powered artificial intelligence of things. IEEE Internet of Things Journal, 8(18), 13849-13875.

[11] Vaigandla, K. K. (2025). AI and Edge Analytics for Real-Time IoT Decision-Making in SMACEnvironments: A Comprehensive Review. Journal of Sensors, IoT & Health Sciences (JSIHS, ISSN: 2584-2560), 3(4), 1-16.

[12] Gong, T., Zhu, L., Yu, F. R., & Tang, T. (2023). Edge intelligence in intelligent transportation systems: A survey. IEEE Transactions on Intelligent Transportation Systems, 24(9), 8919-8944.

[13] Duan, S., Wang, D., Ren, J., Lyu, F., Zhang, Y., Wu, H., & Shen, X. (2022). Distributed artificial intelligence empowered by end-edge-cloud computing: A survey. IEEE Communications Surveys & Tutorials, 25(1), 591-624.

[14] Shahnawaz, M., & Kumar, M. (2025). A comprehensive survey on big data analytics: Characteristics, tools and techniques. ACM Computing Surveys, 57(8), 1-33.

[15] Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019). Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proceedings of the IEEE, 107(8), 1738-1762.

[16] Mohamed, A., Najafabadi, M. K., Wah, Y. B., Zaman, E. A. K., & Maskat, R. (2020). The state of the art and taxonomy of big data analytics: view from new big data framework. Artificial intelligence review, 53(2), 989-1037.

[17] Shen, M., Gu, A., Kang, J., Tang, X., Lin, X., Zhu, L., & Niyato, D. (2023). Blockchains for artificial intelligence of things: A comprehensive survey. IEEE Internet of Things Journal, 10(16), 14483-14506.

[18] Zhang, J., & Tao, D. (2020). Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things. IEEE Internet of Things Journal, 8(10), 7789-7817.

[19] Kabir, R., Watanobe, Y., Ding, D., Islam, M. R., & Naruse, K. (2025). A Comprehensive Survey on Advanced Data Science Platforms for Cyber-Physical Systems, Digital Twins, and Robotics. IEEE Access.

[20] Arthurs, P., Gillam, L., Krause, P., Wang, N., Halder, K., & Mouzakitis, A. (2021). A taxonomy and survey of edge cloud computing for intelligent transportation systems and connected vehicles. IEEE Transactions on Intelligent Transportation Systems, 23(7), 6206-6221.

[21] Yao, J., Zhang, S., Yao, Y., Wang, F., Ma, J., Zhang, J., ... & Yang, H. (2022). Edge-cloud polarization and collaboration: A comprehensive survey for AI. IEEE Transactions on Knowledge and Data Engineering, 35(7), 6866-6886.

[22] Panduman, Y. Y. F., Funabiki, N., Fajrianti, E. D., Fang, S., & Sukaridhoto, S. (2024). A survey of AI techniques in IoT applications with use case investigations in the smart environmental monitoring and analytics in real-time IoT platform. Information, 15(3), 153.

[23] Himeur, Y., Elnour, M., Fadli, F., Meskin, N., Petri, I., Rezgui, Y., ... & Amira, A. (2023). AI-big data analytics for building automation and management systems: a survey, actual challenges and future perspectives. Artificial intelligence review, 56(6), 4929-5021.

[24] Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2021). Edge artificial intelligence for 6G: Vision, enabling technologies, and applications. IEEE journal on selected areas in communications, 40(1), 5-36.

[25] Wang, X., Han, Y., Leung, V. C., Niyato, D., Yan, X., & Chen, X. (2020). Convergence of edge computing and deep learning: A comprehensive survey. IEEE communications surveys & tutorials, 22(2), 869-904.

[26] Chen, W., Milosevic, Z., Rabhi, F. A., & Berry, A. (2023). Real-time analytics: Concepts, architectures, and ML/AI considerations. IEEE Access, 11, 71634-71657.

[27] Tang, S., He, B., Yu, C., Li, Y., & Li, K. (2020). A survey on spark ecosystem: Big data processing infrastructure, machine learning, and applications. IEEE Transactions on Knowledge and Data Engineering, 34(1), 71-91.

[28] Andreoni, M., Lunardi, W. T., Lawton, G., & Thakkar, S. (2024). Enhancing autonomous system security and resilience with generative AI: A comprehensive survey. IEEE Access, 12, 109470-109493.

[29] Awaysheh, F. M., Alazab, M., Garg, S., Niyato, D., & Verikoukis, C. (2021). Big data resource management & networks: Taxonomy, survey, and future directions. IEEE Communications Surveys & Tutorials, 23(4), 2098-2130.

[30] Syu, J. H., Lin, J. C. W., Srivastava, G., & Yu, K. (2023). A comprehensive survey on artificial intelligence empowered edge computing on consumer electronics. IEEE Transactions on Consumer Electronics, 69(4), 1023-1034.

[31] Uddin, M., Obaidat, M., Manickam, S., Laghari, S. U. A., Dandoush, A., Ullah, H., & Ullah, S. S. (2024). Exploring the convergence of Metaverse, Blockchain, and AI: A comprehensive survey of enabling technologies, applications, challenges, and future directions. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 14(6), e1556.

[32] Friha, O., Ferrag, M. A., Kantarci, B., Cakmak, B., Ozgun, A., & Ghoualmi-Zine, N. (2024). Llm-based edge intelligence: A comprehensive survey on architectures, applications, security and trustworthiness. IEEE Open Journal of the Communications Society, 5, 5799-5856.

[33] Ozkan-Okay, M., Akin, E., Aslan, Ö., Kosunalp, S., Iliev, T., Stoyanov, I., & Beloev, I. (2024). A comprehensive survey: Evaluating the efficiency of artificial intelligence and machine learning techniques on cyber security solutions. IEEe Access, 12, 12229-12256.

[34] Guo, F., Yu, F. R., Zhang, H., Li, X., Ji, H., & Leung, V. C. (2021). Enabling massive IoT toward 6G: A comprehensive survey. IEEE Internet of Things Journal, 8(15), 11891-11915.

[35] Nguyen, H. T., Nguyen, M. T., Do, H. T., Hua, H. T., & Nguyen, C. V. (2021). DRL‐based intelligent resource allocation for diverse QoS in 5G and toward 6G vehicular networks: a comprehensive survey. Wireless Communications and Mobile Computing, 2021(1), 5051328.

[36] Chen, D., Huang, C., Fan, T., Lau, H. C., & Yan, X. (2025). Predictive modelling for vessel traffic flow: A comprehensive survey from statistics to AI. Transportation Safety and Environment, 7(3), tdaf022.

[37] López Delgado, J. L., & López Ramos, J. A. (2024). A comprehensive survey on generative AI solutions in IoT security. Electronics, 13(24), 4965.

[38] Afrin, M., Jin, J., Rahman, A., Rahman, A., Wan, J., & Hossain, E. (2021). Resource allocation and service provisioning in multi-agent cloud robotics: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(2), 842-870.

[39] Fouda, M. M., Fadlullah, Z. M., Ibrahem, M. I., & Kato, N. (2024). Privacy-preserving data-driven learning models for emerging communication networks: A comprehensive survey. IEEE Communications Surveys & Tutorials, 27(4), 2505-2542.

[40] Ma, Y., Wang, Z., Yang, H., & Yang, L. (2020). Artificial intelligence applications in the development of autonomous vehicles: A survey. IEEE/CAA Journal of Automatica Sinica, 7(2), 315-329.

@article{1719005,
      author = {A. Deepa, Dr. A. S. Naveen Kumar},
      title = {Autonomous Data Mining Systems for Real-Time Big Data Streams in Edge AI: A Comprehensive Survey},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {9},
      number = {12},
      pages = {1797-1814},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1719005.pdf},
      abstract = {The exponential growth of real-time big data streams generated by IoT devices, cyber-physical systems, and distributed sensors has necessitated the evolution of autonomous data mining systems capable of operating efficiently at the network edge. Traditional cloud-centric analytics architectures suffer from high latency, bandwidth overhead, and privacy risks, limiting their suitability for time-sensitive applications. This survey presents a comprehensive review of autonomous data mining systems for real-time big data streams within Edge AI environments. The study systematically analyzes over 120 recent research contributions (2018–2025), categorizing approaches into stream mining algorithms, online learning frameworks, distributed edge intelligence models, federated mining strategies, and self-adaptive optimization mechanisms. Key statistical insights indicate that nearly 68% of recent frameworks employ deep learning–based stream processing models, while 54% integrate adaptive concept drift detection techniques to maintain model robustness in dynamic environments. Furthermore, approximately 47% of surveyed systems incorporate privacy-preserving mechanisms such as federated learning and differential privacy to address edge-level data security challenges. The survey evaluates methodologies based on latency performance, computational efficiency, scalability, autonomy level, and energy consumption. Comparative analysis reveals that hybrid adaptive mining architectures demonstrate up to 35% improvement in real-time decision latency compared to static edge models. The paper concludes by identifying open research challenges, including autonomous model orchestration, resource-aware self-optimization, explainable stream mining, and trust-aware decentralized intelligence. Future research directions emphasize integrating reinforcement learning–driven adaptability and lightweight generative models for continuous stream evolution. This survey provides a structured taxonomy, performance benchmarking synthesis, and a research roadmap to advance next-generation autonomous Edge AI data mining systems.},
      keywords = {Autonomous Data Mining, Real-Time Big Data Streams, Edge AI Stream Mining Algorithms, Concept Drift Detection, Federated Learning, Distributed Edge Intelligence.},
      month = {June},
      doi = {https://doi.org/10.64388/IREV9I12-1719005}
  }