A Review of System Monitoring Architectures Using Prometheus, ELK Stack, and Custom Dashboards

References

[1] S. Karnouskos, A. W. Colombo, F. Jammes, J. Delsing, and T. Bangemann, "Towards an architecture for service-oriented process monitoring and control," in IECON 2010-36th Annual Conference on IEEE Industrial Electronics Society, 2010: IEEE, pp. 1385-1391.

[2] M. Andreolini, M. Colajanni, and M. Pietri, "A scalable architecture for real-time monitoring of large information systems," in 2012 Second Symposium on Network Cloud Computing and Applications, 2012: IEEE, pp. 143-150.

[3] V. M. Tovarnitchi, "Cloud-based architectures for environment monitoring," in 2017 21st International Conference on Control Systems and Computer Science (CSCS), 2017: IEEE, pp. 708-714.

[4] B. Manate, F. Fortiş, and P. Moore, "Applying the Prometheus methodology for an Internet of Things architecture," in 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, 2014: IEEE, pp. 435-442.

[5] N. Sukhija and E. Bautista, "Towards a framework for monitoring and analyzing high performance computing environments using kubernetes and prometheus," in 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), 2019: IEEE, pp. 257-262.

[6] O. Mart, C. Negru, F. Pop, and A. Castiglione, "Observability in kubernetes cluster: Automatic anomalies detection using prometheus," in 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), 2020: IEEE, pp. 565-570.

[7] N. Sukhija et al., "Event management and monitoring framework for HPC environments using ServiceNow and Prometheus," in Proceedings of the 12th international conference on management of digital ecosystems, 2020, pp. 149-156.

[8] P. Amogh, G. Veeramachaneni, A. K. Rangisetti, B. R. Tamma, and A. A. Franklin, "A cloud native solution for dynamic auto scaling of MME in LTE," in 2017 IEEE 28th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC), 2017: IEEE, pp. 1-7.

[9] T. Prakash, M. Kakkar, and K. Patel, "Geo-identification of web users through logs using ELK stack," in 2016 6th international conference-cloud system and big data engineering (confluence), 2016: IEEE, pp. 606-610.

[10] S. Dharur and K. Swaminathan, "Efficient surveillance and monitoring using the ELK stack for IoT powered Smart Buildings," in 2018 2nd international conference on inventive systems and control (ICISC), 2018: IEEE, pp. 700-705.

[11] A. F. Rochim, M. A. Aziz, and A. Fauzi, "Design log management system of computer network devices infrastructures based on elk stack," in 2019 International Conference on Electrical Engineering and Computer Science (ICECOS), 2019: IEEE, pp. 338-342.

[12] A. Mikula, D. Adamová, M. Adam, J. Chudoba, and J. Švec, "Grid site monitoring and log processing using elk," in CEUR Workshop Proceedings, 2016.

[13] R. Appleyard and J. Adams, "Using the ELK stack for CASTOR application logging at RAL," in International Symposium on Grids and Clouds (ISGC), 2015, vol. 15, no. 20.

@article{1708762,
      author = {Ejielo Ogbuefi, Oyejide Timothy Odofin, Abraham Ayodeji Abayomi, Bolaji Iyanu Adekunle, Oluwademilade Aderemi Agboola; Samuel Owoade},
      title = {A Review of System Monitoring Architectures Using Prometheus, ELK Stack, and Custom Dashboards},
      journal = {Iconic Research And Engineering Journals},
      year = {2021},
      volume = {4},
      number = {12},
      pages = {408-419},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1708762.pdf},
      abstract = {System monitoring architectures play a critical role in ensuring the operational efficiency, performance, and security of modern IT infrastructures. This paper provides a comprehensive review of three prominent monitoring solutions?Prometheus, the ELK Stack (Elasticsearch, Logstash, and Kibana), and Custom Dashboards?evaluating their features, architecture, use cases, and benefits. Prometheus, an open-source time-series monitoring system, is ideal for large-scale, dynamic environments, particularly those utilizing microservices and cloud-native architectures. The ELK Stack offers a robust solution for log aggregation, search, and visualization, making it effective for security monitoring, log analysis, and performance management. Custom Dashboards, integrated with Prometheus or ELK, facilitate real-time visualization of key performance indicators, improving decision-making and responsiveness. Despite their strengths, these solutions present challenges, including scalability concerns, resource requirements, and integration complexities. The paper concludes with suggestions for future advancements in AI-driven monitoring, cross-platform integration, and automated configurations, aiming to enhance the adaptability and efficiency of monitoring systems in increasingly complex IT environments.},
      keywords = {System Monitoring, Prometheus, ELK Stack, Custom Dashboards, Real-Time Data Visualization, Log Aggregation},
      month = {June},
  }