Home / Current Issue / Paper 1722629
Automation of Log Collection and Incident Triage for Security Operations: A Review
Subject area: Science,Engineering and Technology · Area of research: Security Operations Automation
Abstract
Modern security operations centres (SOCs) ingest volumes of telemetry that far exceed the capacity of human analysts to inspect manually, and the resulting flood of alerts has made automation of log collection and incident triage a central concern for defensive practice. This paper reviews the research and engineering literature underpinning that automation, organising a fragmented body of work into a coherent picture spanning log collection and normalization, detection and correlation, automated triage and prioritization, security orchestration and automation, and the datasets used to evaluate these systems. We conduct a structured narrative review of works published up to 2019, complemented by seminal earlier contributions, and we synthesise them into a described taxonomy that maps each stage of the alert pipeline to its dominant techniques, inputs, and failure modes. The review shows that pattern-based log parsing and statistical anomaly detection have matured substantially, that alert correlation and adaptive classification can materially reduce analyst load, and that orchestration platforms increasingly encode triage as executable playbooks. Nevertheless, persistent challenges remain: high false-positive rates rooted in the base-rate fallacy, alert fatigue and analyst burnout, scarcity of labelled operational data, concept drift, and the weak external validity of widely used benchmark datasets. We argue that human factors, evaluation realism, and feedback-driven learning deserve as much attention as detection accuracy. The paper closes with future directions emphasising analyst-in-the-loop design, reproducible benchmarking, and resilience-oriented triage.
Keywords
security operations centre; log analysis; incident triage; SIEM; alert correlation; anomaly detection; alert fatigue; SOAR
References
[1] Bhatt, S., Manadhata, P. K., & Zomlot, L. (2014). The operational role of security information and event management systems. IEEE Security & Privacy, 12(5), 35–41. https://doi.org/10.1109/MSP.2014.103
[2] Axelsson, S. (2000). The base-rate fallacy and the difficulty of intrusion detection. ACM Transactions on Information and System Security, 3(3), 186–205. https://doi.org/10.1145/357830.357849
[3] Sundaramurthy, S. C., Bardas, A. G., Case, J., Ou, X., Wesch, M., McHugh, J., & Rajagopalan, S. R. (2015). A human capital model for mitigating security analyst burnout. In Proceedings of the Eleventh Symposium on Usable Privacy and Security (SOUPS 2015) (pp. 347–359). USENIX Association. https://www.usenix.org/conference/soups2015/proceedings/presentation/sundaramurthy
[4] Ejofodomi, O. A., Gideon, E. N., Oladipo, G. O., & Oshomah, E. R. (2014). Automated detection of architectural detection in mammograms using template matching. International Journal of Biomedical Science and Engineering, 2(1), 1–6.
[5] Ilodigwe, L., & Adesemoye, A. C. (2019a). A critical review of health insurance financing mechanisms and provider payment reform strategies for balancing coverage expansion and financial protection in emerging markets. International Journal of Health and Pharmaceutical Research, 5(2), 31-55. https://doi.org/10.56201/ijhpr.vol.5.no2.2019.pg31.55
[6] Ilodigwe, L., & Adesemoye, A. C. (2019b). From molecules to health systems: A conceptual model explaining why scientific innovation fails to improve patient outcomes without effective healthcare delivery infrastructure. International Journal of Health and Pharmaceutical Research, 5(2), 56-79. https://doi.org/10.56201/ijhpr.vol.5.no2.2019.pg56.79
[7] Denning, D. E. (1987). An intrusion-detection model. IEEE Transactions on Software Engineering, SE-13(2), 222–232. https://doi.org/10.1109/TSE.1987.232894
[8] Roesch, M. (1999). Snort: Lightweight intrusion detection for networks. In Proceedings of the 13th USENIX Conference on System Administration (LISA ’99) (pp. 229–238). USENIX Association. https://www.usenix.org/legacy/publications/library/proceedings/lisa99/full_papers/roesch/roesch.pdf
[9] D’Amico, A., & Whitley, K. (2008). The real work of computer network defense analysts. In J. R. Goodall, G. Conti, & K.-L. Ma (Eds.), VizSEC 2007: Proceedings of the Workshop on Visualization for Computer Security (pp. 19–37). Springer. https://doi.org/10.1007/978-3-540-78243-8_2
[10] Compaine, B. M. (Ed.). (2001). The digital divide: Facing a crisis or creating a myth? MIT Press.
[11] Mossberger, K., Tolbert, C. J., & Stansbury, M. (2003). Virtual inequality: Beyond the digital divide. Georgetown University Press.
[12] Attewell, P. (2001). The first and second digital divides. Sociology of Education, 74(3), 252–259. https://doi.org/10.2307/2673277
[13] Helsper, E. J. (2012). A corresponding fields model for the links between social and digital exclusion. Communication Theory, 22(4), 403–426. https://doi.org/10.1111/j.1468-2885.2012.01416.x
[14] van Deursen, A. J. A. M., & Helsper, E. J. (2015). The third-level digital divide: Who benefits most from being online? Communication and Information Technologies Annual, 10, 29–52. https://doi.org/10.1108/S2050-206020150000010002
[15] Scheerder, A., van Deursen, A., & van Dijk, J. (2017). Determinants of Internet skills, uses and outcomes: A systematic review of the second- and third-level digital divide. Telematics and Informatics, 34(8), 1607–1624. https://doi.org/10.1016/j.tele.2017.07.007
[16] Heeks, R. (2010). Do information and communication technologies (ICTs) contribute to development? Journal of International Development, 22(5), 625–640. https://doi.org/10.1002/jid.1716
[17] Walsham, G. (2017). ICT4D research: Reflections on history and future agenda. Information Technology for Development, 23(1), 18–41. https://doi.org/10.1080/02681102.2016.1246406
[18] Gunkel, D. J. (2003). Second thoughts: Toward a critique of the digital divide. New Media & Society, 5(4), 499–522. https://doi.org/10.1177/146144480354003
[19] DiMaggio, P., Hargittai, E., Neuman, W. R., & Robinson, J. P. (2001). Social implications of the Internet. Annual Review of Sociology, 27, 307–336. https://doi.org/10.1146/annurev.soc.27.1.307
[20] Wei, K.-K., Teo, H.-H., Chan, H. C., & Tan, B. C. Y. (2011). Conceptualizing and testing a social cognitive model of the digital divide. Information Systems Research, 22(1), 170–187. https://doi.org/10.1287/isre.1090.0273
[21] Hilbert, M. (2011). The end justifies the definition: The manifold outlooks on the digital divide and their practical usefulness for policy-making. Telecommunications Policy, 35(8), 715–736. https://doi.org/10.1016/j.telpol.2011.06.012
[22] Selwyn, N. (2016). Is technology good for education? Polity Press.
[23] Sharples, M., Taylor, J., & Vavoula, G. (2010). A theory of learning for the mobile age. In Medienbildung in neuen Kulturräumen (pp. 87–99). VS Verlag. https://doi.org/10.1007/978-3-531-92133-4_6
[24] Traxler, J. (2007). Defining, discussing and evaluating mobile learning. International Review of Research in Open and Distributed Learning, 8(2). https://doi.org/10.19173/irrodl.v8i2.346
[25] Kukulska-Hulme, A. (2009). Will mobile learning change language learning? ReCALL, 21(2), 157–165. https://doi.org/10.1017/S0958344009000202
[26] Donner, J. (2008). Research approaches to mobile use in the developing world: A review of the literature. The Information Society, 24(3), 140–159. https://doi.org/10.1080/01972240802019970
[27] Heeks, R. (2008). ICT4D 2.0: The next phase of applying ICT for international development. Computer, 41(6), 26–33. https://doi.org/10.1109/MC.2008.192
[28] Brown, J. S., & Adler, R. P. (2008). Minds on fire: Open education, the long tail, and Learning 2.0. EDUCAUSE Review, 43(1), 16–32.
[29] Kozma, R. B. (2003). Technology and classroom practices: An international study. Journal of Research on Technology in Education, 36(1), 1–14. https://doi.org/10.1080/15391523.2003.10782399
[30] Pittinsky, M. S. (2003). The wired tower: Perspectives on the impact of the Internet on higher education. Financial Times Prentice Hall.
[31] Box, G. E. P., & Jenkins, G. M. (1976). Time series analysis: Forecasting and control (Rev. ed.). Holden-Day.
[32] Armstrong, J. S. (Ed.). (2001). Principles of forecasting: A handbook for researchers and practitioners. Kluwer Academic. https://doi.org/10.1007/978-0-306-47630-3
[33] De Gooijer, J. G., & Hyndman, R. J. (2006). 25 years of time series forecasting. International Journal of Forecasting, 22(3), 443–473. https://doi.org/10.1016/j.ijforecast.2006.01.001
[34] Gardner, E. S. (2006). Exponential smoothing: The state of the art: Part II. International Journal of Forecasting, 22(4), 637–666. https://doi.org/10.1016/j.ijforecast.2006.03.005
[35] Holt, C. C. (2004). Forecasting seasonals and trends by exponentially weighted moving averages. International Journal of Forecasting, 20(1), 5–10. https://doi.org/10.1016/j.ijforecast.2003.09.015
[36] Fildes, R., Nikolopoulos, K., Crone, S. F., & Syntetos, A. A. (2008). Forecasting and operational research: A review. Journal of the Operational Research Society, 59(9), 1150–1172. https://doi.org/10.1057/palgrave.jors.2602597
[37] Syntetos, A. A., Boylan, J. E., & Croston, J. D. (2005). On the categorization of demand patterns. Journal of the Operational Research Society, 56(5), 495–503. https://doi.org/10.1057/palgrave.jors.2601841
[38] Ahmed, N. K., Atiya, A. F., Gayar, N. E., & El-Shishiny, H. (2010). An empirical comparison of machine learning models for time series forecasting. Econometric Reviews, 29(5–6), 594–621. https://doi.org/10.1080/07474938.2010.481556
[39] Crone, S. F., Hibon, M., & Nikolopoulos, K. (2011). Advances in forecasting with neural networks? Empirical evidence from the NN3 competition. International Journal of Forecasting, 27(3), 635–660. https://doi.org/10.1016/j.ijforecast.2011.04.001
[40] Makridakis, S., Wheelwright, S. C., & Hyndman, R. J. (1998). Forecasting: Methods and applications (3rd ed.). Wiley.
[41] Fildes, R., & Goodwin, P. (2007). Against your better judgment? How organizations can improve their use of management judgment in forecasting. Interfaces, 37(6), 570–576. https://doi.org/10.1287/inte.1070.0309
[42] Chatfield, C. (2000). Time-series forecasting. Chapman & Hall/CRC.
[43] Hyndman, R. J., Koehler, A. B., Snyder, R. D., & Grose, S. (2002). A state space framework for automatic forecasting using exponential smoothing methods. International Journal of Forecasting, 18(3), 439–454. https://doi.org/10.1016/S0169-2070(01)00110-8
[44] Norris, P. (2001). Digital divide: Civic engagement, information poverty, and the Internet worldwide. Cambridge University Press. https://doi.org/10.1017/CBO9781139164887
[45] Badmus, O., Dosunmu, A. A., & Ozowara, D. E. (2018). A systematic review of CI/CD pipeline strategies in Salesforce DevOps: Tools, practices, and deployment outcomes. Iconic Research and Engineering Journals, 2(6).
[46] Badmus, O., Dosunmu, A. A., & Ozowara, D. E. (2019a). A conceptual model for ETL design and data integration in Salesforce-centric enterprise architectures. Iconic Research and Engineering Journals, 3(5).
[47] Badmus, O., Dosunmu, A. A., & Ozowara, D. E. (2019b). A governance framework for Salesforce platform management in regulated healthcare environments. Iconic Research and Engineering Journals, 3(6).
[48] Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009). Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles (SOSP ’09) (pp. 117–132). ACM. https://doi.org/10.1145/1629575.1629587
[49] Vaarandi, R., & Pihelgas, M. (2015). LogCluster: A data clustering and pattern mining algorithm for event logs. In Proceedings of the 2015 11th International Conference on Network and Service Management (CNSM) (pp. 1–7). IEEE. https://doi.org/10.1109/CNSM.2015.7367331
[50] He, P., Zhu, J., Zheng, Z., & Lyu, M. R. (2017). Drain: An online log parsing approach with fixed depth tree. In Proceedings of the 2017 IEEE International Conference on Web Services (ICWS) (pp. 33–40). IEEE. https://doi.org/10.1109/ICWS.2017.13
[51] Zhu, J., He, S., Liu, J., He, P., Xie, Q., Zheng, Z., & Lyu, M. R. (2019). Tools and benchmarks for automated log parsing. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (pp. 121–130). IEEE. https://doi.org/10.1109/ICSE-SEIP.2019.00021
[52] Dosunmu, A. A., & Ogundele, P. O. (2019). Security audit and enterprise risk assessment frameworks for resilient information systems. Iconic Research and Engineering Journals, 3(5), 434-447. https://doi.org/10.64388/IREV3I5-1713225
[53] Agbabiaka, J., Okonkwo, C. S., Ogunwole, O., Mayo, W., & Okeke, O. T. (2019). Supply chain risk management model for EPC and gas processing projects. Iconic Research and Engineering Journals, 3(2), 968-980. https://doi.org/10.64388/IREV3I2-1713124
[54] Okonkwo, C. S., Ogunwole, O., & Okeke, O. T. (2018b). Framework for strategic procurement optimization in oil and gas operations. Iconic Research and Engineering Journals, 1(7), 153-168. https://doi.org/10.64388/IREV1I7-1713119
[55] Okonkwo, C. S., Ogunwole, O., & Okeke, O. T. (2018c). Model for inventory availability and plant uptime improvement in energy facilities. Iconic Research and Engineering Journals, 2(4), 160-172. https://doi.org/10.64388/IREV2I4-1713120
[56] Okonkwo, C. S., Ogunwole, O., Okeke, O. T., & Mayo, W. (2019). Conceptual framework for cost reduction through contract negotiation and vendor governance. Iconic Research and Engineering Journals, 2(9), 468-482. https://doi.org/10.64388/IREV2I9-1713121
[57] García-Teodoro, P., Díaz-Verdejo, J., Maciá-Fernández, G., & Vázquez, E. (2009). Anomaly-based network intrusion detection: Techniques, systems and challenges. Computers & Security, 28(1–2), 18–28. https://doi.org/10.1016/j.cose.2008.08.003
[58] Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3), Article 15. https://doi.org/10.1145/1541880.1541882
[59] Sperotto, A., Schaffrath, G., Sadre, R., Morariu, C., & Pras, A. (2010). An overview of IP flow-based intrusion detection. IEEE Communications Surveys & Tutorials, 12(3), 343–356. https://doi.org/10.1109/SURV.2010.032210.00054
[60] Du, M., Li, F., Zheng, G., & Srikumar, V. (2017). DeepLog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17) (pp. 1285–1298). ACM. https://doi.org/10.1145/3133956.3134015
[61] He, S., Zhu, J., He, P., & Lyu, M. R. (2016). Experience report: System log analysis for anomaly detection. In Proceedings of the 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE) (pp. 207–218). IEEE. https://doi.org/10.1109/ISSRE.2016.21
[62] Sommer, R., & Paxson, V. (2010). Outside the closed world: On using machine learning for network intrusion detection. In Proceedings of the 2010 IEEE Symposium on Security and Privacy (pp. 305–316). IEEE. https://doi.org/10.1109/SP.2010.25
[63] Buczak, A. L., & Guven, E. (2016). A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Communications Surveys & Tutorials, 18(2), 1153–1176. https://doi.org/10.1109/COMST.2015.2494502
[64] Valeur, F., Vigna, G., Kruegel, C., & Kemmerer, R. A. (2004). A comprehensive approach to intrusion detection alert correlation. IEEE Transactions on Dependable and Secure Computing, 1(3), 146–169. https://doi.org/10.1109/TDSC.2004.21
[65] Ning, P., Cui, Y., & Reeves, D. S. (2002). Constructing attack scenarios through correlation of intrusion alerts. In Proceedings of the 9th ACM Conference on Computer and Communications Security (CCS ’02) (pp. 245–254). ACM. https://doi.org/10.1145/586110.586144
[66] Julisch, K. (2003). Clustering intrusion detection alarms to support root cause analysis. ACM Transactions on Information and System Security, 6(4), 443–471. https://doi.org/10.1145/950191.950192
[67] Mbonu, I. S., Aliliele, C., Iwuanyanwu, U., & Oluoha, O. M. (2018). A conceptual framework for legal and ethical risk modeling in enterprise data protection governance systems. Iconic Research and Engineering Journals, 2(2), 207-226. https://doi.org/10.64388/IREV2I2-1714911
[68] Mbonu, I. S., Aliliele, C., Uzoka, E., & Oluoha, O. M. (2019a). A review of comparative data protection regulations and secure cloud implementation strategies across jurisdictions. Iconic Research and Engineering Journals, 2(9), 482-501. https://doi.org/10.64388/IREV2I9-1714912
[69] Mbonu, I. S., Iwuanyanwu, U., Uzoka, E., & Oluoha, O. M. (2019b). Advances in enterprise log analytics and automated incident response architectures using Python and SIEM platforms. Iconic Research and Engineering Journals, 3(2), 1000-1019. https://doi.org/10.64388/IREV3I2-1714915
[70] Jardine, A. K. S., Lin, D., & Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 20(7), 1483–1510. https://doi.org/10.1016/j.ymssp.2005.09.012
[71] Susto, G. A., Schirru, A., Pampuri, S., McLoone, S., & Beghi, A. (2015). Machine learning for predictive maintenance: A multiple classifier approach. IEEE Transactions on Industrial Informatics, 11(3), 812–820. https://doi.org/10.1109/TII.2014.2349359
[72] Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018). Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48, 144–156. https://doi.org/10.1016/j.jmsy.2018.01.003
[73] Tao, F., Zhang, H., Liu, A., & Nee, A. Y. C. (2019). Digital twin in industry: State-of-the-art. IEEE Transactions on Industrial Informatics, 15(4), 2405–2415. https://doi.org/10.1109/TII.2018.2873186
[74] Stouffer, K., Lightman, S., Pillitteri, V., Abrams, M., & Hahn, A. (2015). Guide to Industrial Control Systems (ICS) Security. NIST SP 800-82.
[75] Dragos, & Inc. (2017). TRISIS Malware: Analysis of Safety System Targeted Attack. Dragos Intelligence.
[76] Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018). SoK: Security and privacy in machine learning. in Proc. IEEE EuroS&P, 399-414.
[77] Sproul, W. D., Christie, D. J., & Carter, D. C. (2005). Control of reactive sputtering processes. Thin Solid Films, 491(1), 1–17. https://doi.org/10.1016/j.tsf.2005.05.022
[78] Strijckmans, K., Schelfhout, R., & Depla, D. (2018). Tutorial: Hysteresis during the reactive magnetron sputtering process. Journal of Applied Physics, 124(24), 241101. https://doi.org/10.1063/1.5042084
[79] Musil, J., Baroch, P., Vlček, J., Nam, K. H., & Han, J. G. (2005). Reactive magnetron sputtering of thin films: present status and trends. Thin Solid Films, 475(1), 208–218. https://doi.org/10.1016/j.tsf.2004.07.041
[80] Sarakinos, K., Alami, J., & Konstantinidis, S. (2010). High power pulsed magnetron sputtering: A review on scientific and engineering state of the art. Surface and Coatings Technology, 204(11), 1661–1684. https://doi.org/10.1016/j.surfcoat.2009.11.013
[81] Anders, A. (2017). Tutorial: Reactive high power impulse magnetron sputtering (R-HiPIMS). Journal of Applied Physics, 121(17), 171101. https://doi.org/10.1063/1.4978350
[82] Anders, A. (2014). A review comparing cathodic arcs and high power impulse magnetron sputtering (HiPIMS). Surface and Coatings Technology, 257, 308–325. https://doi.org/10.1016/j.surfcoat.2014.08.043
[83] Negri, E., Fumagalli, L., & Macchi, M. (2017). A Review of the Roles of Digital Twin in CPS-based Production Systems. Procedia Manufacturing, 11, 939–948. https://doi.org/10.1016/j.promfg.2017.07.198
[84] Kritzinger, W., Karner, M., Traar, G., Henjes, J., & Sihn, W. (2018). Digital Twin in manufacturing: A categorical literature review and classification. IFAC-PapersOnLine, 51(11), 1016–. https://doi.org/10.1016/j.ifacol.2018.08.474
[85] Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686–707. https://doi.org/10.1016/j.jcp.2018.10.045
[86] Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences, 113(15), 3932–3937. https://doi.org/10.1073/pnas.1517384113
[87] Benner, P., Gugercin, S., & Willcox, K. (2015). A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems. SIAM Review, 57(4), 483–531. https://doi.org/10.1137/130932715
[88] Evensen, G. (2003). The Ensemble Kalman Filter: theoretical formulation and practical implementation. Ocean Dynamics, 53(4), 343–367. https://doi.org/10.1007/s10236-0030036-9
[89] Mayne, D. Q. (2014). Model predictive control: Recent developments and future promise. Automatica, 50(12), 2967–2986. https://doi.org/10.1016/j.automatica.2014.10.128
[90] Odejobi, O. D., Hammed, N. I., & Ahmed, K. S. (2019). Approximation complexity model for cloud-based database optimization problems. IRE Journals, 2(9), 1–10.
[91] Ahmed, K. S., Odejobi, O. D., & Oshoba, T. O. (2019). Algorithmic model for constraint satisfaction in cloud network resource allocation. IRE Journals, 2(12), 516–532.
[92] Dagodzo, D. (2018a). A conceptual framework for UAV integration into national power grid inspection programs. IRE Journals, 2(5), 391–412. https://doi.org/10.64388/IREV2I51716082
[93] Pflug, A., Siemers, M., Melzig, T., Schäfer, L., & Bräuer, G. (2014). Simulation of linear magnetron discharges in 2D and 3D. Surface and Coatings Technology, 260, 411–416. https://doi.org/10.1016/j.surfcoat.2014.09.042
[94] Nyberg, T., Högberg, H., Greczynski, G., & Berg, S. (2019). A simple model for non-saturated reactive sputtering processes. Thin Solid Films, 688, 137413. https://doi.org/10.1016/j.tsf.2019.137413
[95] Berg, S., Särhammar, E., & Nyberg, T. (2014). Upgrading the ‘Berg-model’ for reactive sputtering processes. Thin Solid Films, 565, 186–192. https://doi.org/10.1016/j.tsf.2014.02.063
[96] Elebe, O. (2018). Conceptual model for insider threat classification and risk modeling in complex digital systems. and digital identity verification framework for.
[97] Srinidhi, B., Yan, J., & Bhargava, H. K. (2015). Effect of information security investments on firm performance. Decision Support Systems, 74, 1–15.
[98] McKone, K. E., Schroeder, R. G., & Cua, K. O. (2001). The impact of total productive maintenance practices on manufacturing performance. Journal of Operations Management, 19(1), 39–58. https://doi.org/10.1016/S0272-6963(00)00030-9
[99] Dal, B., Tugwell, P., & Greatbanks, R. (2000). Overall equipment effectiveness as a measure of operational improvement: A practical analysis. International Journal of Operations & Production Management, 20(12), 1488–1502. https://doi.org/10.1108/01443570010355750
[100] Bamber, D., Sharp, C. J., & Hides, M. T. (1999). Factors affecting successful implementation of total productive maintenance: A UK manufacturing case study perspective. Journal of Quality in Maintenance Engineering, 5(3), 162–181. https://doi.org/10.1108/13552519910282601
[101] McKone, K. E., Schroeder, R. G., & Cua, K. O. (1999). Total productive maintenance: A contextual view. Journal of Operations Management, 17(2), 123–144. https://doi.org/10.1016/S0272-6963(98)00039-4
[102] Tezel, A., Koskela, L., & Tzortzopoulos, P. (2016). Visual management in production management: A literature synthesis. Journal of Manufacturing Technology Management, 27(6), 766–799. https://doi.org/10.1108/JMTM-08-2015-0071
[103] Bagavathiappan, S., Lahiri, B. B., Saravanan, T., Philip, J., & Jayakumar, T. (2013). Infrared thermography for condition monitoring: A review. Infrared Physics & Technology, 60, 35–55. https://doi.org/10.1016/j.infrared.2013.03.006
[104] Lee, J., Wu, F., Zhao, W., Ghaffari, M., Liao, L., & Siegel, D. (2014b). Prognostics and health management design for rotary machinery systems: Reviews, methodology and applications. Mechanical Systems and Signal Processing, 42(1), 314–334. https://doi.org/10.1016/j.ymssp.2013.06.004
[105] Lei, Y., Li, N., Guo, L., Li, N., Yan, T., & Lin, J. (2018). Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mechanical Systems and Signal Processing, 104, 799–834. https://doi.org/10.1016/j.ymssp.2017.11.016
[106] Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008). Damage propagation modeling for aircraft engine run-to-failure simulation. in Proc. Int. Conf. Prognostics and Health Management (PHM), 1–9. https://doi.org/10.1109/PHM.2008.4711414
[107] Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019). Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing, 115, 213–237. https://doi.org/10.1016/j.ymssp.2018.05.050
[108] Lee, J., Kao, H.-A., & Yang, S. (2014a). Service innovation and smart analytics for Industry 4.0 and big data environment. Procedia CIRP, 16, 3–8. https://doi.org/10.1016/j.procir.2014.02.001
[109] Tao, F., Zhang, M., Liu, Y., & Nee, A. Y. C. (2018b). Digital twin driven prognostics and health management for complex equipment. CIRP Annals, 67(1), 169–172. https://doi.org/10.1016/j.cirp.2018.04.055
[110] Selcuk, S. (2017). Predictive maintenance, its implementation and latest trends. Proceedings of the Institution of Mechanical Engineers, 231(9), 1670–1679. https://doi.org/10.1177/0954405415601640
[111] Okonkwo, C. S., Ogunwole, O., & Okeke, O. T. (2018a). Model for inventory availability and plant uptime improvement in energy facilities. IRE Journals, 2(4), 160–172. https://doi.org/10.64388/IREV2I4-1713120
[112] Peng, Y., Dong, M., & Zuo, M. J. (2010). Current status of machine prognostics in condition-based maintenance: A review. The International Journal of Advanced Manufacturing Technology, 50(1), 297–313. https://doi.org/10.1007/s00170-009-2482-0
[113] Liu, H.-C., Liu, L., & Liu, N. (2013). Risk evaluation approaches in failure mode and effects analysis: A literature review. Expert Systems with Applications, 40(2), 828–838. https://doi.org/10.1016/j.eswa.2012.08.010
[114] Antoni, J. (2007). Fast computation of the kurtogram for the detection of transient faults. Mechanical Systems and Signal Processing, 21(1), 108–124. https://doi.org/10.1016/j.ymssp.2005.12.002
[115] Lei, Y., Lin, J., Zuo, M. J., & He, Z. (2014). Condition monitoring and fault diagnosis of planetary gearboxes: A review. Measurement, 48, 292–305. https://doi.org/10.1016/j.measurement.2013.11.012
[116] Lei, Y., Jia, F., Lin, J., Xing, S., & Ding, S. X. (2016). An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data. IEEE Transactions on Industrial Electronics, 63(5), 3137–3147. https://doi.org/10.1109/TIE.2016.2519325
[117] Wen, L., Li, X., Gao, L., & Zhang, Y. (2018). A new convolutional neural network-based data-driven fault diagnosis method. IEEE Transactions on Industrial Electronics, 65(7), 5990–. https://doi.org/10.1109/TIE.2017.2774777
[118] Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H., & Sui, F. (2018a). Digital twin-driven product design, manufacturing and service with big data. The International Journal of Advanced Manufacturing Technology, 94(9), 3563–3576. https://doi.org/10.1007/s00170-0170233-1
[119] Pietraszek, T. (2004). Using adaptive alert classification to reduce false positives in intrusion detection. In E. Jonsson, A. Valdes, & M. Almgren (Eds.), Recent Advances in Intrusion Detection (RAID 2004) (Lecture Notes in Computer Science, Vol. 3224, pp. 102–124). Springer. https://doi.org/10.1007/978-3-540-30143-1_6
[120] Strom, B. E., Applebaum, A., Miller, D. P., Nickels, K. C., Pennington, A. G., & Thomas, C. B. (2018). MITRE ATT&CK: Design and philosophy (Technical Report MP180360). The MITRE Corporation. https://attack.mitre.org/docs/ATTACK_Design_and_Philosophy_March_2020.pdf
[121] Cichonski, P., Millar, T., Grance, T., & Scarfone, K. (2012). Computer security incident handling guide (NIST Special Publication 800-61 Revision 2). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-61r2
[122] Ladapo, O. O., Dosunmu, A. A., Jooda, D., & Abolaji, T. O. (2018). Lessons learned from offline assessment of security-critical systems: The case of Microsoft Active Directory. Iconic Research and Engineering Journals, 2(6), 277-299. https://doi.org/10.64388/IREV2I6-1717205
[123] Ladapo, O. O., Jooda, D., Dosunmu, A. A., & Abolaji, T. O. (2019). Implementation of Active Directory for efficient management of enterprise networks. Iconic Research and Engineering Journals, 3(4), 608-627. https://doi.org/10.64388/IREV3I4-1717206
[124] Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP) (pp. 108–116). SciTePress. https://doi.org/10.5220/0006639801080116
[125] Tavallaee, M., Bagheri, E., Lu, W., & Ghorbani, A. A. (2009). A detailed analysis of the KDD CUP 99 data set. In Proceedings of the 2009 IEEE Symposium on Computational Intelligence for Security and Defence Applications (CISDA) (pp. 1–6). IEEE. https://doi.org/10.1109/CISDA.2009.5356528
[126] Bola-Sadipe, D., Aliliele, C., & Eboh, E. E. (2019). Conceptual model for strategic accounting systems strengthening financial transparency and regulatory compliance. IRE Journals, 3(6), 535-564. https://doi.org/10.64388/IREV3I6-1715327
[127] Akeju, B., Edivri, J., Ogbole, J. I., Okoruwa, P. O., Fadayomi, O., & Abolaji, T. O. (2018). Conceptual model for insider threat classification and risk modeling in complex digital systems. Iconic Research and Engineering Journals, 1(9), 476–492. https://doi.org/10.64388/IREV1I9-1713778
[128] Fadayomi, O., Abolaji, T. O., Edivri, J., Ogbole, J. I., Okoruwa, P. O., & Akeju, B. (2019). Risk-based cybersecurity assurance and data availability limitations, advances and future research opportunities. Iconic Research and Engineering Journals, 2(12), 602–617. https://doi.org/10.64388/IREV2I12-1713779
[129] Aminu-Ibrahim, A. Y., Ogbete, J. C., & Ambali, K. B. (2018). Developing sustainable diagnostic laboratory infrastructure models for emerging and resource constrained health systems. Iconic Research and Engineering Journals, 1(8), 118-132. https://doi.org/10.64388/IREV1I8-1713586
[130] Aminu-Ibrahim, A. Y., Ogbete, J. C., & Ambali, K. B. (2019). Capital project delivery models for high risk healthcare infrastructure in developing national health systems. Iconic Research and Engineering Journals, 2(10), 626-649. https://doi.org/10.64388/IREV2I10-1713588
[131] Ogbete, J. C., Aminu-Ibrahim, A. Y., & Ambali, K. B. (2018). Optimizing laboratory spatial planning strategies to improve diagnostic accuracy, safety, and clinical throughput. Iconic Research and Engineering Journals, 2(1), 87-113. https://doi.org/10.64388/IREV1I7-1713587
[132] Ogbete, J. C., Aminu-Ibrahim, A. Y., & Ambali, K. B. (2019). Regulatory compliant design systems for molecular and pathology laboratories in highly controlled environments. Iconic Research and Engineering Journals, 3(4), 607-631. https://doi.org/10.64388/IREV3I4-1713589
[133] Arumosoye, O. M., & Obriki, O. D. (2018). Development of an integrated heat stress risk conceptual model for industrial operations in extreme environments. Iconic Research and Engineering Journals, 1(12), 141-160. https://doi.org/10.64388/IREV1I12-1714415
[134] Arumosoye, O. M., & Obriki, O. D. (2019). Systematic review of near-miss and hazard observation data utilization in industrial safety management. Iconic Research and Engineering Journals, 3(2), 981-999. https://doi.org/10.64388/IREV3I2-1714417
[135] Obogo, S. F., Ozobu, C. O., & Uduokhai, D. O. (2019a). Advances in leadership driven safety culture transformation in large construction workforces. Iconic Research and Engineering Journals, 3(5), 507-523. https://doi.org/10.64388/IREV3I5-1715497
[136] Obogo, S. F., Ozobu, C. O., & Uduokhai, D. O. (2019b). Development of a construction safety risk governance model for multi contractor high rise projects. Iconic Research and Engineering Journals, 2(9), 502-522. https://doi.org/10.64388/IREV2I9-1715495
[137] Obogo, S. F., Uduokhai, D. O., & Ozobu, C. O. (2019c). Systematic review of hazard identification and risk control practices in urban construction projects. Iconic Research and Engineering Journals, 2(12), 666-681. https://doi.org/10.64388/IREV2I12-1715496
[138] Obriki, O. D., & Arumosoye, O. M. (2018). Conceptual modeling of data-driven occupational safety risk control in large-scale energy infrastructure projects. Iconic Research and Engineering Journals, 1(7), 169-189. https://doi.org/10.64388/IREV1I7-1714414
[139] Obriki, O. D., & Arumosoye, O. M. (2019). A conceptual framework linking management safety walkthrough frequency and coverage to safety culture outcomes in mega projects. Iconic Research and Engineering Journals, 2(8), 355-374. https://doi.org/10.64388/IREV2I8-1714416
[140] Dagodzo, D. (2018b). A conceptual framework for UAV integration into national power grid inspection programs. Iconic Research and Engineering Journals, 2(5), 391-412. https://doi.org/10.64388/IREV2I5-1716082
[141] Dagodzo, D. (2018c). A review of UAV applications in electrical transmission line inspection: Methods, technologies, and challenges. Iconic Research and Engineering Journals, 2(6), 234-254. https://doi.org/10.64388/IREV2I6-1716083
[142] Sunday, E. A., & Omoegun, G. O. (2018). Integrating solar power solutions in small-scale manufacturing industries in Nigeria. International Journal of Scientific Research in Science, Engineering and Technology, 4(8), 832-853.
[143] Sunday, E. A., & Omoegun, G. O. (2019). Optimizing electrical load distribution for hybrid solar installations in developing economies. International Journal of Scientific Research in Mechanical and Materials Engineering, 3(6), 27-47.
[144] Sunday, E. A., Omoegun, G. O., Essien, M. A., & Oluokun, O. A. (2019). Thermodynamic efficiency and control strategies in residential air conditioning systems. International Journal of Scientific Research in Civil Engineering, 3(3), 52-76.
[145] Bobga, M. A., Boakye, K., Ogbona, C. S., & Yeboah, T. J. (2018). Pedagogical strategies for teaching students with learning difficulties in resource-constrained schools. Iconic Research and Engineering Journals, 2(4), 173-194. https://doi.org/10.64388/IREV2I4-1714176
[146] Yeboah, T. J., Bobga, M. A., Boakye, K., & Ogbona, C. S. (2019). Professional development and teacher competence in managing exceptional learners: A Ghanaian perspective. Iconic Research and Engineering Journals, 2(12), 618-636. https://doi.org/10.64388/IREV2I12-1714177
[147] Amayo, E. B. (2017). Multi algorithm of loss objective function of a single phase induction motor. International Journal of Development Research, 7(5), 12805-12810.
[148] Amayo, E. B., & Popoola, T. T. (2017a). Multi algorithm of weight objective function of a single phase induction motor. International Journal of Trend in Research and Development, 4(2), 184-188.
[149] Amayo, E. B., & Popoola, T. T. (2017b). Scientific study of telecommunication deployment in achieving quality network. International Journal of Trend in Research and Development, 4(5), 311-314.
[150] Amayo, E. B., Ubeku, E., & Oshevire, P. (2015). Multi algorithm of a single objective function of a single phase induction motor. Journal of Multidisciplinary Engineering Science and Technology, 2(12), 3400-3403.
[151] Oshevire, P., Eyenubo, O. J., & Amayo, B. (2017). Voltage control in the presence of distributed generation. ATBU Journal of Science, Technology and Education, 5(2), 165-173.
[152] Adeyelu, O. O. (2019). An adaptive safety management system implementation model for aerodromes in developing economy contexts. Iconic Research and Engineering Journals, 3(4), 628-654. https://doi.org/10.64388/IREV3I4-1718479
[153] Akomolafe, O., & Agu, M. U. (2018). A conceptual model for enhancing internal audit quality through technology-enabled risk assessment frameworks. Iconic Research and Engineering Journals, 1(9), 458-475.
[154] Akomolafe, O., & Agu, M. U. (2019a). A conceptual framework for developing risk-based internal control models in the insurance and banking sectors. Iconic Research and Engineering Journals, 2(8), 335-354.
[155] Akomolafe, O., & Agu, M. U. (2019b). A review of data-driven risk evaluation models for emerging market financial institutions. Iconic Research and Engineering Journals, 3(6), 433-448.
[156] Akomolafe, O., & Agu, M. U. (2019c). Advances in financial resilience through integrated governance and compliance strategies. Iconic Research and Engineering Journals, 2(10), 607-620.
[157] Lawal, O. A., & Oduleye, T. E. (2018a). A conceptual model for financial analytics driven enterprise value creation in technology firms. Iconic Research and Engineering Journals, 2(2).
[158] Lawal, O. A., & Oduleye, T. E. (2018b). A review and conceptual framework for tax governance and cross-border compliance analytics. Iconic Research and Engineering Journals, 2(5).
[159] Lawal, O. A., & Oduleye, T. E. (2019a). A conceptual risk assessment model for transfer pricing in multinational corporations. Iconic Research and Engineering Journals, 2(12).
[160] Lawal, O. A., & Oduleye, T. E. (2019b). Conceptualizing data driven executive decision systems for strategic financial planning. Iconic Research and Engineering Journals, 3(3).
[161] Dogbatsey, E. A., & Ebhojie, O. (2018). Budget compliance and statutory reporting in Sub-Saharan Africa: A systematic review of frameworks, gaps, and reform pathways. Zenodo. https://doi.org/10.5281/zenodo.20446859
[162] Dogbatsey, E. A., Ebhojie, O., & Oyeleye, A. O. (2019). Reconciliation control and financial workflow redesign in emerging market organizations: A conceptual framework. Zenodo. https://doi.org/10.5281/zenodo.20447103
How to cite this paper
@article{1722629,
author = {Ifeanyichukwu Jeffrey Okwesa, Daniel Ajiga, Uzoamaka Iwuanyanwu},
title = {Automation of Log Collection and Incident Triage for Security Operations: A Review},
journal = {Iconic Research And Engineering Journals},
year = {2019},
volume = {3},
number = {5},
pages = {627-650},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1722629.pdf},
abstract = {Modern security operations centres (SOCs) ingest volumes of telemetry that far exceed the capacity of human analysts to inspect manually, and the resulting flood of alerts has made automation of log collection and incident triage a central concern for defensive practice. This paper reviews the research and engineering literature underpinning that automation, organising a fragmented body of work into a coherent picture spanning log collection and normalization, detection and correlation, automated triage and prioritization, security orchestration and automation, and the datasets used to evaluate these systems. We conduct a structured narrative review of works published up to 2019, complemented by seminal earlier contributions, and we synthesise them into a described taxonomy that maps each stage of the alert pipeline to its dominant techniques, inputs, and failure modes. The review shows that pattern-based log parsing and statistical anomaly detection have matured substantially, that alert correlation and adaptive classification can materially reduce analyst load, and that orchestration platforms increasingly encode triage as executable playbooks. Nevertheless, persistent challenges remain: high false-positive rates rooted in the base-rate fallacy, alert fatigue and analyst burnout, scarcity of labelled operational data, concept drift, and the weak external validity of widely used benchmark datasets. We argue that human factors, evaluation realism, and feedback-driven learning deserve as much attention as detection accuracy. The paper closes with future directions emphasising analyst-in-the-loop design, reproducible benchmarking, and resilience-oriented triage.},
keywords = {security operations centre; log analysis; incident triage; SIEM; alert correlation; anomaly detection; alert fatigue; SOAR},
month = {November},
doi = {https://doi.org/10.64388/IREV3I5-1722629}
}