Unified Insights into Graph Neural Networks: From Connectivity to Linear Architectures, and Time Series Applications

References

[1] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE TNNLS, vol. 32, no. 1, pp. 4–24, 2020.

[2] Z. A. Sahili and M. Awad, “Spatio-temporal graph neural networks: A survey,” arXiv preprint, vol. abs/2301.10569, 2023.

[3] W. Jiang and J. Luo, “Graph neural network for traffic forecasting: A survey,” Expert Syst. Appl., p. 117921, 2022.

[4] X. Zhang, M. Zeman, T. Tsiligkaridis, and M. Zitnik, “Graph-guided network for irregularly sampled multivariate time series,” in ICLR, 2022.

[5] H. Zhao, Y. Wang, J. Duan, C. Huang, D. Cao, Y. Tong, B. Xu, J. Bai, J. Tong, and Q. Zhang, “Multivariate time-series anomaly detection via graph attention network,” in ICDM, 2020, pp. 841– 850.

[6] A. Cini, I. Marisca, and C. Alippi, “Filling the g ap s: Multivariate time series imputation by graph neural networks,” in ICLR, 2022.

[7] Q. Wen, L. Yang, T. Zhou, and L. Sun, “Robust time series analysis and applications: An industrial perspective,” in KDD, 2022, pp. 4836–4837.

[8] B. Lim and S. Zohren, “Time-series forecasting with deep learning: a survey,” Philos. Trans. Royal Soc. A PHILOS T R SOC A, vol. 379, no. 2194, p. 20200209, 2021.

[9] H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller, “Deep learning for time series classification: a review,” DMKD, vol. 33, no. 4, pp. 917–963, 2019.

[10] A. Blazquez-Garc ´ ´ıa, A. Conde, U. Mori, and J. A. Lozano, “A review on outlier/anomaly detection in time series data,” ACM Computing Surveys, vol. 54, no. 3, pp. 1–33, 2021.

[11] C. Fang and C. Wang, “Time series data imputation: A survey on deep learning approaches,” arXiv preprint, vol. abs/2011.11347, 2020.

[12] A. A. Cook, G. Mısırlı, and Z. Fan, “Anomaly detection for iot time-series data: A survey,” IEEE IoT Journal, vol. 7, no. 7, pp. 6481–6494, 2019.

[13] S. Wang, J. Cao, and S. Y. Philip, “Deep learning for spatiotemporal data mining: A survey,” IEEE TKDE, vol. 34, no. 8, pp. 3681–3700, 2020.

[14] J. Ye, J. Zhao, K. Ye, and C. Xu, “How to build a graph-based deep learning architecture in traffic domain: A survey,” IEEE TITS, vol. 23, no. 5, pp. 3904–3924, 2020.

[15] G. Jin, Y. Liang, Y. Fang, J. Huang, J. Zhang, and Y. Zheng, “Spatio-temporal graph neural networks for predictive learning in urban computing: A survey,” arXiv preprint, vol. abs/2303.14483, 2023.

[16] S. Rahmani, A. Baghbani, N. Bouguila, and Z. Patterson, “Graph neural networks for intelligent transportation systems: A survey,” IEEE TITS, 2023.

[17] K. Hundman, V. Constantinou, C. Laporte, I. Colwell, and T. Soderstr ¨ om, “Detecting spacecraft anomalies using lstms and ¨ nonparametric dynamic thresholding,” in KDD, 2018, pp. 387– 395.

[18] Y. Su, Y. Zhao, C. Niu, R. Liu, W. Sun, and D. Pei, “Robust anomaly detection for multivariate time series through stochastic recurrent neural network,” in KDD, 2019, pp. 2828–2837.

[19] J. Xu, H. Wu, J. Wang, and M. Long, “Anomaly transformer: Time series anomaly detection with association discrepancy,” in ICLR, 2022.

[20] Z. Chen, D. Chen, X. Zhang, Z. Yuan, and X. Cheng, “Learning graph structures with transformer for multivariate time-series anomaly detection in iot,” IEEE IoT Journal, vol. 9, no. 12, pp. 9179–9189, 2021.

[21] S. Moritz and T. Bartz-Beielstein, “imputets: time series missing value imputation in r.” R J., vol. 9, no. 1, p. 207, 2017.

[22] M. Saad, M. Chaudhary, F. Karray, and V. Gaudet, “Machine learning based approaches for imputation in time series data and their impact on forecasting,” in SMC, 2020, pp. 2621–2627.

[23] Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu, “Recurrent neural networks for multivariate time series with missing values,” Scientific reports, vol. 8, no. 1, p. 6085, 2018.

[24] M. Jin, Y. Zheng, Y.-F. Li, S. Chen, B. Yang, and S. Pan, “Multivariate time series forecasting with dynamic graph neural odes,” IEEE TKDE, 2022.

[25] G. Lai, W. Chang, Y. Yang, and H. Liu, “Modeling long- and shortterm temporal patterns with deep neural networks,” in SIGIR, 2018, pp. 95–104.

[26] S.-Y. Shih, F.-K. Sun, and H.-y. Lee, “Temporal pattern attention for multivariate time series forecasting,” Machine Learning, vol. 108, pp. 1421–1441, 2019

[27] A. D. Richardson, M. Aubinet, A. G. Barr, D. Y. Hollinger, A. Ibrom, G. Lasslop, and M. Reichstein, “Uncertainty quantification,” Eddy covariance: A practical guide to measurement and data analysis, pp. 173–209, 2012.

[28] M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya et al., “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” Information Fusion, vol. 76, pp. 243–297, 2021.

[29] Y. Li, B. Qian, X. Zhang, and H. Liu, “Knowledge guided diagnosis prediction via graph spatial-temporal network,” in SDM, 2020, pp. 19–27.

[30] H. Wen, Y. Lin, X. Mao, F. Wu, Y. Zhao, H. Wang, J. Zheng, L. Wu, H. Hu, and H. Wan, “Graph2route: A dynamic spatial-temporal graph neural network for pick-up and delivery route prediction,” in KDD, 2022, pp. 4143–4152.

[31] S. Guo, Y. Lin, N. Feng, C. Song, and H. Wan, “Attention based spatial-temporal graph convolutional networks for traffic flow forecasting,” in AAAI, 2019, pp. 922–929.

[32] D. Huang, J. Bartel, and J. Palowitch, “Recurrent graph neural networks for rumor detection in online forums,” arXiv preprint, vol. abs/2108.03548, 2021.

[33] L. Wu, P. Cui, J. Pei, L. Zhao, and L. Song, Graph neural networks. Springer, 2022.

[34] W. L. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in NeurIPS, 2017, pp. 1024–1034.

[35] J. Chen, J. Zhu, and L. Song, “Stochastic training of graph convolutional networks with variance reduction,” in ICML, vol. 80, 2018, pp. 941–949.

[36] J. Chen, T. Ma, and C. Xiao, “Fastgcn: Fast learning with graph convolutional networks via importance sampling,” in ICLR, 2018.

[37] W. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, and C. Hsieh, “Clustergcn: An efficient algorithm for training deep and large graph convolutional networks,” in KDD, 2019, pp. 257–266.

[38] X. Wang, D. Wang, L. Chen, and Y. Lin, “Building transportation foundation model via generative graph transformer,” arXiv preprint, vol. abs/2305.14826, 2023.

[39] S. Pan, L. Luo, Y. Wang, C. Chen, J. Wang, and X. Wu, “Unifying large language models and knowledge graphs: A roadmap,” arXiv preprint, vol. abs/2306.08302, 2023.

[40] G. Panagopoulos, G. Nikolentzos, and M. Vazirgiannis, “Transfer graph neural networks for pandemic forecasting,” in AAAI, 2021, pp. 4838–4845.

[41] X. Wang, G. Chen, G. Qian, P. Gao, X.-Y. Wei, Y. Wang, Y. Tian, and W. Gao, “Large-scale multi-modal pre-trained models: A comprehensive survey,” arXiv preprint, vol. abs/2302.10035, 2023.

[42] W. Xi, A. Jain, L. Zhang, and J. Lin, “Lb-simtsc: An efficient similarity-aware graph neural network for semi-supervised time series classification,” arXiv preprint, vol. abs/2301.04838, 2023.

[43] D. Zha, K.-H. Lai, K. Zhou, and X. Hu, “Towards similarity-aware time-series classification,” in SDM, 2022, pp. 199–207.

[44] K. Chen, M. Feng, and T. S. Wirjanto, “Multivariate time series anomaly detection via dynamic graph forecasting,” arXiv preprint, vol. abs/2302.02051, 2023.

[45] Y. Zheng, H. Y. Koh, M. Jin, L. Chi, K. T. Phan, S. Pan, Y.-P. P. Chen, and W. Xiang, “Correlation-aware spatial-temporal graph learning for multivariate time-series anomaly detection,” arXiv preprint, vol. abs/2307.08390, 2023.

[46] W. Zhang, C. Zhang, and F. Tsung, “Grelen: Multivariate time series anomaly detection from the perspective of graph relational learning,” in IJCAI, 2022, pp. 2390–2397.

[47] G. Shen, W. Zhou, W. Zhang, N. Liu, Z. Liu, and X. Kong, “Bidirectional spatial-temporal traffic data imputation via graph attention recurrent neural network,” Neurocomputing, vol. 531, pp. 151–162, 2023.

[48] X. Kong, W. Zhou, G. Shen, W. Zhang, N. Liu, and Y. Yang, “Dynamic graph convolutional recurrent imputation network for spatiotemporal traffic missing data,” KBS, vol. 261, p. 110188, 2023.

[49] M. Jin, G. Shi, Y.-F. Li, Q. Wen, B. Xiong, T. Zhou, and S. Pan, “How expressive are spectral-temporal graph neural networks for time series forecasting?” arXiv preprint, vol. abs/2305.06587, 2023.

[50] A. Cini*, I. Marisca*, F. Bianchi, and C. Alippi, “Scalable spatiotemporal graph neural networks,” in AAAI, 2023.

[51] H. Yu, T. Li, W. Yu, J. Li, Y. Huang, L. Wang, and A. Liu, “Regularized graph structure learning with semantic knowledge for multi-variates time-series forecasting,” in IJCAI, 2022, pp. 2362–2368.

@article{1705196,
      author = {Pratham Taneja, Sanskar Saxena, Divyansh Singh, Aditya Chauhan, Bhaumik Tyagi; Garv Kalia},
      title = {Unified Insights into Graph Neural Networks: From Connectivity to Linear Architectures, and Time Series Applications},
      journal = {Iconic Research And Engineering Journals},
      year = {2023},
      volume = {7},
      number = {5},
      pages = {71-82},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1705196.pdf},
      abstract = {The predominant data modality employed for the documentation of dynamic system measurements is time series, which emanates prolifically from physical sensors and online processes (virtual sensors). The imperative role of time series analytics in extracting the inherent wealth of information from available data is underscored by recent advancements in graph neural networks (GNNs). These networks have witnessed a notable surge in their application for time series analysis, owing to their capacity to explicitly model inter-temporal and inter-variable relationships?attributes that traditional and other deep neural network-based methodologies find challenging. This review endeavors to comprehensively review graph neural networks for time series analysis (GNN4TS), encompassing four fundamental dimensions: forecasting, classification, anomaly detection, and imputation. The overarching objective is to serve as a guiding resource for designers and practitioners, facilitating an enhanced understanding, application development, and progression of research within the domain of GNN4TS.Linear Architectures are also discussed in this very research. Neural Architecture Search (NAS) has exhibited notable advancements in optimizing Graph Neural Networks (GNNs), denoted as NAS-GNNs, outperforming manually designed GNN architectures. Nevertheless, challenges inherited from conventional NAS methods, including elevated computational costs and optimization complexities, persist. Notably, prior NAS approaches have tended to overlook the inherent characteristics of GNNs, which inherently possess expressive power without the necessity for training. Notably, NAC achieves up to a 200? acceleration in computational efficiency and a 19.9% enhancement in accuracy compared to robust baseline methods.},
      keywords = {Graph neural networks, Linear architecture search, Time series Applications.},
      month = {November},
  }