Big Data Analytics for Cross-Domain Anomaly Detection to Identify Hidden Service-Impacting Patterns in Fiber Access Broadband Networks

References

[1] [1] Sun, W., Zhang, F., Guo, L. and Liu, X. (2025). A multi-feature domain interaction learning framework for anomalous network detection. Electronics, 14, 3729. https://doi.org/10.3390/electronics14183729.

[2] [2] Hwang, R.H., Peng, M.C., Huang, C.W., Lin, P.C. and Nguyen, V.L. (2020). An unsupervised deep learning model for early network traffic anomaly detection. IEEE Access, 8, 30387-30399.

[3] [3] Zhao, H., Wang, Y., Duan, J., Huang, C., Cao, D., Tong, Y., Xu, B., Bai, J., Tong, J. and Zhang, Q. (2020). Multivariate time-series anomaly detection via graph attention network. Proceedings of the IEEE International Conference on Data Mining, 841-850. https://doi.org/10.1109/ICDM50108.2020.00093.

[4] [4] Audibert, J., Michiardi, P., Guyard, F., Marti, S. and Zuluaga, M.A. (2020). USAD: UnSupervised anomaly detection on multivariate time series. Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 3395-3404. https://doi.org/10.1145/3394486.3403392.

[5] [5] Deng, A. and Hooi, B. (2021). Graph neural network-based anomaly detection in multivariate time series. Proceedings of the AAAI Conference on Artificial Intelligence, 35(5), 4027-4035. https://doi.org/10.1609/aaai.v35i5.16523.

[6] [6] Chen, Z., Chen, D., Zhang, X., Yuan, Z. and Cheng, X. (2022). Learning graph structures with transformer for multivariate time-series anomaly detection in IoT. IEEE Internet of Things Journal, 9(12), 9179-9190. https://doi.org/10.1109/JIOT.2021.3100509.

[7] [7] Xu, J., Wu, H., Wang, J. and Long, M. (2022). Anomaly Transformer: Time series anomaly detection with association discrepancy. International Conference on Learning Representations.

[8] [8] Tuli, S., Casale, G. and Jennings, N.R. (2022). TranAD: Deep transformer networks for anomaly detection in multivariate time series data. Proceedings of the VLDB Endowment, 15(6), 1201-1214.

[9] [9] Zheng, W., Zhong, J., Zhang, Q. and Zhao, G. (2022). MTT: An efficient model for encrypted network traffic classification using multi-task transformer. Applied Intelligence, 52, 10741-10756.

[10] [10] Sarhan, M., Layeghy, S. and Portmann, M. (2022). Towards a standard feature set for network intrusion detection system datasets. Mobile Networks and Applications, 27, 357-370.

[11] [11] Huoh, T.L., Luo, Y., Li, P. and Zhang, T. (2023). Flow-based encrypted network traffic classification with graph neural networks. IEEE Transactions on Network and Service Management, 20, 1224-1237.

[12] [12] Mayer, K.S., Soares, J.A., Pinto, R.P., Rothenberg, C.E., Arantes, D.S. and Mello, D.A.A. (2021). Machine-learning-based soft-failure localization with partial software-defined networking telemetry. Journal of Optical Communications and Networking, 13(10), E122-E131.

[13] [13] Babbar, J., Triki, A., Ayassi, R. and Laye, M. (2022). Machine learning models for alarm classification and failure localization in optical transport networks. Journal of Optical Communications and Networking, 14(8), 621-628.

[14] [14] Khan, L.Z., Pedro, J., Costa, N., De Marinis, L., Napoli, A. and Sambo, N. (2023). Data augmentation to improve performance of neural networks for failure management in optical networks. Journal of Optical Communications and Networking, 15(1), 57-67. https://doi.org/10.1364/JOCN.472605.

[15] [15] Abdelli, K., Tropschug, C., Griesser, H. and Pachnicke, S. (2023). Faulty branch identification in passive optical networks using machine learning. Journal of Optical Communications and Networking, 15(4), 187-196. https://doi.org/10.1364/JOCN.475882.

[16] [16] Zeng, C., Zhang, J., Wang, R., Zhang, B. and Ji, Y. (2023). Multiple attention mechanisms-driven component fault location in optical networks with network-wide monitoring data. Journal of Optical Communications and Networking, 15(7), C9-C19.

[17] [17] Fosic, I., Zagar, D., Grgic, K. and Krizanovic, V. (2023). Anomaly detection in NetFlow network traffic using supervised machine learning algorithms. Journal of Industrial Information Integration, 33, 100466.

[18] [18] Han, D., Zhou, H., Weng, T.H., Wu, Z., Han, B., Li, K.C. and Pathan, A.S.K. (2023). LMCA: A lightweight anomaly network traffic detection model integrating adjusted MobileNet and coordinate attention mechanism for IoT. Telecommunication Systems, 84, 549-564.

[19] [19] Ren, J., Zhang, F., Wang, Z., Li, J., Zhang, Y., Pan, S. and Yu, P.S. (2023). Graph learning for anomaly analytics: Algorithms, applications, and challenges. ACM Transactions on Knowledge Discovery from Data, 17. https://doi.org/10.1145/3570906.

[20] [20] Kruse, L.E., Kuhl, S., Dochhan, A. and Pachnicke, S. (2024). Experimental investigation of machine-learning-based soft-failure management using the optical spectrum. Journal of Optical Communications and Networking, 16(2), 94-103. https://doi.org/10.1364/JOCN.500930.

[21] [21] Straub, M., Reber, J., Saier, T., Borkowski, R., Li, S., Khomchenko, D., Richter, A., Farber, M., Kafer, T. and Bonk, R. (2024). ML approaches for OTDR diagnoses in passive optical networks-event detection and classification: Ways for ODN branch assignment. Journal of Optical Communications and Networking, 16(7), C43-C50.

[22] [22] Han, X., Xu, G., Zhang, M., Yang, Z., Yu, Z., Huang, W. and Meng, C. (2024). DE-GNN: Dual embedding with graph neural network for fine-grained encrypted traffic classification. Computer Networks, 245, 110372.

[23] [23] Yang, Z., Ma, Z., Zhao, W., Li, L. and Gu, F. (2024). HRNN: Hypergraph recurrent neural network for network intrusion detection. Journal of Grid Computing, 22, 52.

[24] [24] Wang, T., Xie, X., Wang, W., Wang, C., Zhao, Y. and Cui, Y. (2024). NetMamba: Efficient network traffic classification via pre-training unidirectional Mamba. Proceedings of the IEEE International Conference on Network Protocols, 1-11.

[25] [25] Azab, A., Khasawneh, M., Alrabaee, S., Choo, K.K.R. and Sarsour, M. (2024). Network traffic classification: Techniques, datasets, and challenges. Digital Communications and Networks, 10, 676-692.

[26] [26] Liu, Z., Huang, X., Zhang, J., Hao, Z., Sun, L. and Peng, H. (2024). Multivariate time-series anomaly detection based on enhancing graph attention networks with topological analysis. arXiv:2408.13082.

[27] [27] Zhang, C., Chen, Y., Zhang, M., Liu, Z. and Wang, D. (2025). SHAP-assisted EE-LightGBM model for explainable fault diagnosis in practical optical networks. Journal of Optical Communications and Networking, 17(2), 81-94. https://doi.org/10.1364/JOCN.527872.

[28] [28] Musumeci, F. and Tornatore, M. (2025). Failure management in optical networks with ML: A tutorial on applications, challenges, and pitfalls. Journal of Optical Communications and Networking, 17(8), C144-C155. https://doi.org/10.1364/JOCN.551910.

[29] [29] Chen, X., Chen, X. and Zhu, Z. (2025). Interpretable optical network fault detection and localization with multi-task graph prototype learning. Journal of Optical Communications and Networking, 17(9), D73-D82. https://doi.org/10.1364/JOCN.562633.

[30] [30] Karunakaran, V., Reyes, R.R., Shariati, B., Fischer, J.K., Autenrieth, A. and Bauschert, T. (2025). Dynamic network-aware soft failure localization using machine learning in optical networks. Journal of Optical Communications and Networking, 17(11), 1032-1046. https://doi.org/10.1364/JOCN.564177.

@article{1722266,
      author = {Rizwan Farooq Khan},
      title = {Big Data Analytics for Cross-Domain Anomaly Detection to Identify Hidden Service-Impacting Patterns in Fiber Access Broadband Networks},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {10},
      number = {2},
      pages = {2854-2865},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1722266.pdf},
      abstract = {Fibre-to-the-x networks are observed through several operational systems, yet service-impacting degradation often remains hidden because each system describes only a fragment of the end-to-end service. Optical power, line errors, dynamic bandwidth allocation, IP-session behaviour, customer-premises telemetry, topology, alarms and complaints may each remain within local thresholds while their joint movement signals a developing fault. This review examines how cross-domain data correlation and anomaly detection can expose such weak, distributed evidence before it becomes a widespread outage. It synthesises research published from 2020 to 2025 on multivariate time-series modelling, graph learning, feature-domain interaction, optical-network failure management and explainable detection. The review proposes an evidence-centred architecture that aligns heterogeneous telemetry by service, topology and time; learns both within-domain signatures and between-domain dependencies; models propagation across shared network resources; and converts anomaly scores into service-impact hypotheses that engineers can verify. Particular attention is given to incomplete labels, changing baselines, class imbalance, encrypted traffic, topology reconfiguration and the distinction between statistical abnormality and operational harm. The analysis argues that effective FTTx assurance requires neither a single universal model nor indiscriminate data fusion. It requires a layered design in which deterministic rules, unsupervised detectors, graph-temporal models and human validation cooperate under explicit data-quality and governance controls. The resulting framework supports earlier detection, sharper fault-domain isolation and more defensible prioritisation while preserving operational explainability.},
      keywords = {FTTx networks; passive optical networks; cross-domain correlation; anomaly detection; service assurance; graph learning; multivariate time series; root-cause analysis},
      month = {August},
      doi = {https://doi.org/10.64388/IREV10I2-1722266}
  }