Research on Spatio-Temporal Data Prediction based on Deep Learning = 딥러닝 기반 시공간 데이터 예측 연구

국문 초록 (Abstract)

시공간 데이터(spatio-temporal data) 예측은 교통, 기상학 의료 분야에서 중요한 응용 가치를 갖는 시공간 데이터 마이닝 분야의 핵심 연구문제 중 하나이다. 데이터에 적합한 학습 모델을 설계하면, 실제 응용 시나리오에서 모든 종류의 예측 작업을 수행할 수 있으며 다양한 분야에서 시스템 지능을 실현하는 데 도움이 될 수 있다. 데이터 예측에 대한 수요가 계속 증가함에 따라, ‘정확한 시공간 데이터 예측을 어떻게 구현하는가’가 다양한 분야에서 중요한 문제가 되고 있다.
전통적인 예측 방법은 일반적으로 손으로 설계한 특징(hand-designed features)과 모델에 의존하는데, 이는 특징 추출 능력과 모델의 복잡성으로 인해 제한된다. 그러나 딥러닝 기술은 강력한 특징 학습 및 모델 표현 기능을 갖고 있어 시공간 데이터 예측 작업에서 강한 잠재력을 보여주고 있다. 시공간 데이터 예측에 딥러닝이 광범위하게 적용되고 있음에도 불구하고, 기존 연구 방법에는 몇 가지 문제가 남아있다. 예를 들어, 시간적, 공간적 종속성을 모두 고려하면 주기성 및 추세 등 시공간 데이터의 특성이 약화된다. 단일 모델은 다중 뷰 관점에서 시공간 데이터의 복잡성에 적응할 수 없다. 단일 로컬 특징의 추출은 시공간 데이터의 오랜 특징을 포착하지 못하고, 단일 전역 특징의 추출은 시공간 데이터의 세부 특징을 포착하지 못한다. 고정된 그래프 구조는 그래프의 유연성에 영향을 미친다.
본 논문은 시공간 데이터의 주기적 및 추세적 특징 추출, Multiview 학습의 관점에서 본 multi-module hybrid model 구축, MGTC(Multigate time convolution) 메커니즘 구축, MAPGL(Mixed Hop Attention Propagation Graph Learning) 메커니즘 구축 및 시공간 데이터 융합을 포함하는 다양한 시공간 데이터 응용 시나리오에 대한 딥 러닝 기반의 일련의 혁신적인 시공간 데이터 예측 모델을 제안합니다.
논문의 주요 혁신 연구는 다음과 같다.
1. 해석 가능한 신경망 모듈을 구축하여 시간에 따른 특징 추출을 개선했다. 해석 가능한 모듈은 잔차 연결(Residual connection) 링크를 사용하여 해석 가능성이 있는 추세 및 주기적 시계열 특징의 추출을 도입하여 시공간 데이터의 주기적 특징과 추세적 특징의 추출을 향상시킨다.
2. 시공간 의존성 추출을 심층적으로 포착하기 위해 새로운 하이브리드 모델을 제안한다. 다시점 학습 모델(Multi-View Learning Model) 관점에서 서로 다른 모델을 혼합하여 사용함으로써 시공간 데이터 학습에 대한 시야를 높여 단일 뷰를 가진 단일 모델의 한계를 보완한다.
3. 시공간 데이터의 로컬 및 글로벌 특징 캡처를 위한 MGTC 메커니즘을 제안한다. 각 컨볼루션 커널의 초기 계층에 게이팅 메커니즘을 적용하고 나중에 여러 컨볼루션 커널에 게이팅 메커니즘을 다시 적용하여 여러 컨볼루션 커널의 시간적 컨볼루션을 활용한다. 이를 통해 각 컨볼루션 커널은 입력 기능을 선택적으로 적응적이고 동적으로 필터링할 수 있으므로 다른 시간 규모에서 정보의 학습과 활용을 향상시킨다. 다중 게이팅 시간 컨볼루션 메커니즘은 다양한 수준에서 특징 추출을 향상시킨다.
4. 시공간 데이터에서 그래프 데이터의 특징 표현 학습 기능을 향상시키는 MAPGL 메커니즘을 제안한다. 또한, 이 메커니즘은 그래프 내 노드 간 관계를 정확하게 포착하도록 최적화되어 시공간 데이터의 공간 상관 관계를 향상시킨다.
5. 시공간 상관관계를 보다 포괄적으로 포착하기 위해 시간 컨볼루션과 그래프 컨볼루션이 서로 협력하는 시공간 융합에 대한 새로운 접근 방식을 제안한다. Fusion Model 의 이점을 활용하여, 여러 모듈이 융합되어 시공간 데이터의 시공간 종속성을 향상시킨다. 융합된 모듈 사이에 홉 연결과 잔차 연결이 추가되어 모델을 시공간 데이터의 역학에 더 잘 적응시킨다.

다국어 초록 (Multilingual Abstract)

Spatio-temporal data prediction is one of the core research problems in the field of spatio-temporal data mining, which has an important application value in the fields of transportation, meteorology, and medical treatment. Designing suitable learning models for different types of spatio-temporal data can serve all kinds of prediction tasks in practical application scenarios and assist in realizing the intelligence of systems in various fields. With the growing demand for data prediction, how to realize accurate spatio-temporal data prediction has become an important problem in various fields. Traditional prediction methods usually rely on hand-designed features and models, which are limited by the ability of feature extraction and the complexity of models. Deep learning techniques, on the other hand, have powerful feature learning and model representation capabilities and show great potential in spatio-temporal data prediction tasks. Although deep learning has been widely applied in spatio-temporal data prediction, there are still some problems in the existing research methods, for example, while considering both temporal and spatial dependencies, it weakens the characteristics of spatio-temporal data such as periodicity and trend. A single model cannot adapt to the complexity of spatio-temporal data from a multi-view perspective. The extraction of a single local feature fails to capture the long-time features of spatio- temporal data, and the extraction of a single global feature fails to capture the detailed features of spatio-temporal data. The fixed graph structure affects the flexibility of the graph. Our work proposes a series of innovative spatio-temporal data prediction models based on deep learning for different spatio-temporal data application scenarios, which are used to include the extraction of periodic and trending features of spatio- temporal data, the construction of a multi-model hybrid model from the perspective of multi-view learning, the construction of a multi-gated temporal convolution (MGTC) mechanism, the construction of a mixed-hop attention propagation graph learning (MAPGL) mechanism, and the fusion of spatio-temporal data. The main innovative research of the paper includes:
1. In order to enhance the extraction of time-dependent features, we constructed the interpretable neural network module. The interpretable module employs residual connection linkage to introduce the extraction of trending and periodic time series features with interpretability, which enhances the extraction of periodic and trending features of spatio-temporal data.
2. In order to be able to capture spatio-temporal dependency extraction more deeply, we propose a new hybrid model. It increases the view of spatio-temporal data learning by using a mixture of different models from a multi-view learning model perspective, which makes up for the limitation of a single model with a single view.
3. We propose an MGTC mechanism for local and global feature capture of spatio-temporal data. It utilizes temporal convolution of multiple convolutional kernels by applying a gating mechanism to the initial layer of each convolutional kernel, and later applying the gating mechanism again to multiple convolutional kernels. Through the gating mechanism, each convolutional kernel is able to adaptively and dynamically filter the input features selectively, which enhances the learning and utilization of information at different time scales. The multiple gated temporal convolution mechanism enhances the extraction of features at different levels.
4. We propose a MAPGL mechanism that enhances the capability of feature representation learning of graph data in spatio-temporal data. The graph is also optimized to accurately capture the relationships between nodes in the graph, which enhances the spatial correlation in spatio-temporal data.
5. We propose a new approach to spatio-temporal fusion, where temporal convolution and graph convolution collaborate with each other for a more comprehensive capture of spatio-temporal correlations. Taking advantage of the fusion model, multiple modules are fused to enhance the spatio-temporal dependencies of spatio-temporal data. Hop connections and residual connections are added between the fused modules to better adapt the model to the dynamics of the spatio-temporal data.
Key words: spatio-temporal prediction, interpretable neural network, Multi-model hybrid model, Multi-gated time convolution, mixed-hop attention propagation graph learning, spatio-temporal fusion

목차 (Table of Contents)

• 1 Introduction 1
• 1.1 Research background and significance 1
• 1.2 Related Works 10
• 1.2.1 Spatio-Temporal Data Type 10
• 1.2.2 Traditional Machine Learning Models and Spatio-Temporal Data Modeling 12
• 1.2.3 Deep Learning and Spatio-Temporal Data Modeling 16
• 1.2.4 Spatio-Temporal Data Prediction 25
• 1.3 Main Contents and Contributions of the Research 29
• 1.4 Structure of the Dissertation 33
• 1.5 Chapter Summary 34
• 2 Relevant Theoretical Basis 36
• 2.1 Spatio-Temporal Data 36
• 2.1.1 Spatio-Temporal Data Characteristics 37
• 2.1.2 Air Quality Data 42
• 2.1.3 Traffic Flow Data 43
• 2.2 Related Theories of Spatio-Temporal Data Modeling 45
• 2.2.1 Deep Neural Networks 46
• 2.2.2 Interpretable Neural Networks 47
• 2.2.3 Recurrent Neural Networks 49
• 2.2.4 Attention Mechanism 52
• 2.2.5 Graph Neural Networks 55
• 2.2.6 Temporal Convolutional Networks 60
• 2.3 Chapter Summary 63
• 3 A Hybrid Model for Spatio-temporal Air Quality Prediction Based on Inter-
• pretable Neural Networks and a Graph Neural Network 65
• 3.1 Chapter Introduction 65
• 3.2 Problem Definition 66
• 3.3 A Hybrid Model Based on Interpretable Neural Networks and Graph
• Neural Network 68
• 3.3.1 Framework Overview 68
• 3.3.2 Temporal dependency modeling 69
• 3.3.3 Spatial dependency modeling 74
• 3.4 Experiments 77
• 3.4.1 Data Description 77
• 3.4.2 Experimental Settings 78
• 3.4.3 Baseline Comparison Analysis 79
• 3.4.4 Comparative Analysis: Individual Module vs. Hybrid Model 82
• 3.4.5 Ablation Studies 83
• 3.4.6 Display and Analysis 84
• 3.5 Chapter Summary 85
• 4 GLTD:Graph-based Learning and Temporal Dynamics for Spatio-Temporal
• Traffic Flow Prediction 87
• 4.1 Chapter Introduction 87
• 4.2 Problem Definition 88
• 4.3 Spatio-Temporal Prediction Modeling with Graph-based Learning and
• Temporal Dynamics 89
• 4.3.1 Framework Overview 89
• 4.3.2 Mix-hop Attention Propagation Graph Learning 90
• 4.3.3 Multiple gated temporal convolution 94
• 4.3.4 LSTM 97
• 4.4 Experiments 98
• 4.4.1 Data Description 99
• 4.4.2 Experimental setup 99
• 4.4.3 Baselines 100
• 4.4.4 Experimental Results 101
• 4.4.5 Ablation Studies 103
• 4.4.6 Study of the Graph Learning Layer 105
• 4.4.7 Case Study 109
• 4.5 Chapter Summary 110
• 5 Summary and Prospect 111
• 5.1 Summary of Research Work 111
• 5.2 Prospect 113
• References 116
• Appendix 139
• 1. Data Set 139
• 2. Experimental Setup 139
• 3. Parameter Study 140
• Acknowledgements 143
• 국문초록 146

참고문헌 (Reference)

1. Long short-term memory, J. Schmidhuber, S. Hochreiter and, Neural Comput., vol. 9, no. 8, pp. 1735–1780, doi: 10.1162/neco.1997.9.8.1735, , 1997

2. Graph Attention Networks, P. Liò and, A. Romero, A. Casanova, P. Veličković, G. Cucurull, Y. Bengio, arXiv AccessedOnline Available http//arxiv org/abs/1710.10903, , 2018

3. Attention Is All You Need, A. Vaswani et al., arXiv, 2023. AccessedOnline Available http//arxiv. org/abs/1706.03762, , 2023

4. The Graph Neural Network Model, G. Monfardini, M. Hagenbuchner and, F. Scarselli, A. C. Tsoi, M. Gori, IEEE Trans. Neural Netw., vol. 20, no. 1, pp. 61–80, doi: 10.1109/TNN.2008.2005605, , 2009

5. Crime Rate Inference with Big Data, C. Graif and, D. Kifer, H. Wang, Z. Li, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD 16. New York, NY, USA: Association for Computing Machinery, 13 2016, pp. 635–644. doi: 10.1145/2939672.2939736., , 2016

6. WaveNetA Generative Model for Raw Audio, A. van den Oord et al., arXiv, Sep. 19, 2016. doi: 10.48550/arXiv.1609.03499, , 2016

7. Adam: A Method for Stochastic Optimization, D. P. Kingma and, J. Ba, arXiv Jan. 29 Accessed Jul 27, 2023Online Available http//arxiv. org/abs/ 1412.6980, , 2017

8. XGBoost: A Scalable Tree Boosting System in, C. Guestrin, T. Chen and, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. doi: 10.1145/2939672.2939785, , 2016

9. DeeperGCN: All You Need to Train Deeper GCNs, A. Thabet and, B. Ghanem, G. Li, C. Xiong, doi: 10.48550/arXiv.2006.07739, , 2020

10. Focal-Test-Based Spatial Decision Tree Learning, S. Shekhar, J. Corcoran, Z. Jiang, J. Knight and, X. Zhou, IEEE Trans. Knowl. Data Eng., vol. 27, no. 6, pp. 1547–1559, doi: 10.1109/TKDE.2014.2373383, , 2015

11. MFAS: Multimodal Fusion Architecture Search 2024, F. Jurie, V. Vielzeuf, J.-M. Perez-Rua, M. Baccouche and, S. Pateux, presented at the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 6966–6975 AccessedOnline Available https, , 2019

12. Inductive Representation Learning on Large Graphs, J. Leskovec, Z. Ying and, W. Hamilton, in Advances in Neural Information Processing Systems, Curran Associates, Inc. Accessed Dec. 2023Online Available https//proceedings neurips cc/paper_files/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract. html., , 2017

13. Sequence to Sequence Learning with Neural Networks, I. Sutskever, O. Vinyals and, Q. V. Le, in Advances in Neural Information Processing Systems, Curran Associates, Inc. Accessed: 2024Online Available https//proceedings. neurips. cc/paper/2014/hash/a14ac55a4f27472c5d894ec1c3c743d2-Abstract. html., , 2014

14. Learning representations by back-propagating errors, R. J. Williams, D. E. Rumelhart, G. E. Hinton and, vol. 323, no. 6088, pp. 533–536, doi: 10.1038/323533a0, , 1986

15. Graph Deep Learning: State of the Art and Challenges, S. Georgousis, X. Xie, M. P. Kenning and, IEEE Access, vol. 9, pp. 22106–22140, 2021, doi: 10.1109/ACCESS.2021.3055280, , 2021

16. Bayesian-Based Ensemble Source Apportionment of PM2.5, J. A. Mulholland and, H. A. Holmes, S. Balachandran, H. H. Chang, J. E. Pachon, A. G. Russell, vol. 47, no. 23, pp. 13511–13518, doi: 10.1021/ es4020647., , 2013

17. Deep Learning for Spatio-Temporal Data Mining: A Survey, J. Cao and, S. Wang, P. S. Yu, IEEE Trans. Knowl. Data Eng., vol. 34, no. 8, pp. 3681–3700, doi: 10.1109/TKDE.2020.3025580, , 2022

18. Graph WaveNet for Deep Spatial-Temporal Graph Modeling,, C. Zhang, G. Long, J. Jiang and, S. Pan, Z. Wu, arXiv, May 31, 2019. doi: 10.48550/arXiv.1906. 00121, , 2019

19. FCCF: forecasting citywide crowd flows based on big data, M. X. Hoang, A. K. Singh, Y. Zheng and, in Proceedings of the 24th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, in SIGSPACIAL 16. New York, NY, USA: Association for Computing Machinery, pp. 1–10. doi: 10.1145/2996913.2996934., , 2016

20. PM-25 forecasting use reconstruct phase space LS-SVM, in, Z. Li and, J. Yang, 2010 The 2nd Conference on Environmental Science and Information Application Technology, pp. 143–146. doi: 10.1109/ESIAT.2010.5568607, , 2010

21. Forecasting Fine-Grained Air Quality Based on Big Data in, Y. Zheng et al., Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD 15. New York, NY, USA: Association for Computing Machinery, 10 2015, pp. 2267–2276. doi: 10.1145/2783258.2788573, , 2015

22. Multi-Task Learning for Spatio-Temporal Event Forecasting, C.-T. Lu and, N. Ramakrishnan, L. Zhao, F. Chen, Q. Sun, J. Ye, in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD 15. New York, NY, USA: Association for Computing Machinery, 10 2015, pp. 1503–1512. doi: 10.1145/2783258.2783377., , 2015

23. U-Air: when urban air quality inference meets big data, in, Y. Zheng, H.-P. Hsieh, F. Liu and, Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, in KDD 13. New York, NY, USA: Association for Computing Machinery, pp. 1436–1444. doi: 10.1145/2487575.2488188., , 2013

24. Spectral Networks and Locally Connected Networks on Graphs,, A. Szlam and, J. Bruna, W. Zaremba, Y. LeCun, arXiv, doi: 10.48550/ arXiv.1312.6203, , 2014

25. Backpropagation Applied to Handwritten Zip Code Recognition,, Y. LeCun et al., Neural Comput., vol. 1, no. 4, pp. 541–551, doi: 10.1162/ neco.1989.1.4.541, , 1989

26. GMAN: A Graph Multi-Attention Network for Traffic Prediction, C. Wang and, X. Fan, J. Qi, C. Zheng, Proc. AAAI Conf. Artif. Intell., vol. 34, no. 01, Art. no. 01, doi: 10.1609/aaai. v34i01.5477, , 2020

27. Spatio-Temporal Data Mining: A Survey of Problems and Methods, G. Atluri, A. Karpatne and, V. Kumar, ACM Comput. Surv., vol. 51, no. 4, p. 83:1-83:41, 22 2018, doi: 10.1145/3161602, , 2018

28. Deep Distributed Fusion Network for Air Quality Prediction, in, Y. Zheng, Z. Wang, T. Li and, X. Yi, J. Zhang, Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, in KDD 18. New York, NY, USA: Association for Computing Machinery, pp. 965–973. doi: 10.1145/3219819.3219822., , 2018

29. Imagenet classification with deep convolutional neural networks, G. E. Hinton, I. Sutskever and, A. Krizhevsky, Commun. ACM, vol. 60, no. 6, pp. 84–90, 24 2017, doi: 10.1145/3065386, , 2017

30. Semi-Supervised Classification with Graph Convolutional Networks, M. Welling, T. N. Kipf and, arXiv. org. Accessed:Online Available https//arxiv. org/abs/1609.02907v4, , 2023

31. Deep Air Quality Forecasting Using Hybrid Deep Learning Framework, S.-J. Horng, Y. Yang and, S. Du, T. Li, IEEE Trans. Knowl. Data Eng., vol. 33, no. 6, pp. 2412–2424, doi: 10.1109/TKDE.2019.2954510, , 2021

32. Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction, H. Yao et al., Proc. AAAI Conf. Artif. Intell., vol. 32, no. 1, Art. no. 1, doi: 10.1609/aaai. v32i1.11836, , 2018

33. Graphs, Entities, and Step Mixture for Enriching Graph Representation, S. Kwon, W. Shin, K. Shin, J.-W. Ha and, IEEE Access, vol. 9, pp. 144025–144034, 2021, doi: 10.1109/ACCESS.2021.3121708, , 2021

34. Non-Stationary Model for Crime Rate Inference Using Modern Urban Data, C. Graif and, Z. Li, H. Wang, D. Kifer, H. Yao, IEEE Trans. Big Data, vol. 5, no. 2, pp. 180–194, doi: 10.1109/TBDATA.2017.2786405, , 2019

35. Estimating Ground-Level PM2.5 in China Using Satellite Remote Sensing,, Y. Liu, J. Bi and, X. Hu, L. Huang, Z. Ma, vol. 48, no. 13, pp. 7436–7444, doi: 10.1021/es5009399, , 2014

36. Neural Machine Translation by Jointly Learning to Align and Translate,, Y. Bengio, K. Cho and, D. Bahdanau, arXiv, doi: 10.48550/ arXiv.1409.0473, , 2016

37. Urban Water Quality Prediction based on Multi-task Multi-view Learning, D. S. Rosenblum, Y. Liang, Y. Liu, Y. Zheng, S. Liu and, presented at the Proceedings of the 25th International Joint Conference on Artificial Intelligence, Accessed 2024Online Available https//www microsoft com/en-us/research/publication/urban-water-quality-prediction-based-multi-task-multi-view-learning-2., , 2016

38. Prediction of Ambient Air Quality Based on Neural Network Technique, in, R. Jailani and, R. L. A. Shauri, M. M. Kamal, 2006 4th Student Conference on Research and Development, pp. 115–119. doi: 10.1109/SCORED.2006. 4339321, , 2006

39. AccuAir: Winning Solution to Air Quality Prediction for KDD Cup 2018, in, X. Li and, Z. Luo, K. Hu, J. Huang, P. Zhang, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, in KDD 19. New York, NY, USA: Association for Computing Machinery, 25 2019, pp. 1842–1850. doi: 10.1145/3292500.3330787., , 2019

40. MS-Net: Multi-Source Spatio-Temporal Network for Traffic Flow Prediction, S. Fang et al., IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 7142–7155, doi: 10.1109/TITS.2021.3067024, , 2022

41. Using LSTM and GRU neural network methods for traffic flow prediction, in, L. Li, Z. Zhang and, R. Fu, 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), pp. 324–328. doi: 10.1109/YAC.2016.7804912, , 2016

42. A hybrid framework for multivariate long-sequence time series forecasting,, J. Peng, Y. Wang, X. Wang, Z. Zhang and, X. Tang, vol. 53, no. 11, pp. 13549–13568,, doi: 10.1007/s10489-022-04110-1, , 2023

43. Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting, in, L. BAI, C. Li, C. Wang, L. Yao, X. Wang and, Advances in Neural Information Processing Systems, Curran Associates, Inc., pp. 17804–17815. Accessed: 2024Online Available https//proceedings. neurips. cc/ paper_files/paper/2020/hash/ce1aad92b939420fc17005e5461e6f48-Abstract. html., , 2020

44. Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction, J. Zhang, D. Qi, Y. Zheng and, Proc. AAAI Conf. Artif. Intell., vol. 31, no. 1, Art. no. 1, doi: 10.1609/aaai. v31i1.10735, , 2017

45. Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting, M. Li and, Z. Zhu, Proc. AAAI Conf. Artif. Intell., vol. 35, no. 5, Art. no. 5, doi: 10.1609/aaai. v35i5.16542, , 2021

46. Prediction of air pollution index (API) using support vector machine (SVM),, W. C. Leong, Z. Ahmad, R. O. Kelani and, vol. 8, no. 3, p. 103208, doi: 10.1016/j. jece.2019.103208, , 2020

47. DeepCrime: Attentive Hierarchical Recurrent Networks for Crime Prediction in, C. Huang, N. V. Chawla, J. Zhang, Y. Zheng and, Proceedings of the 27th ACM International Conference on Information and Knowledge Management, in CIKM 18. New York, NY, USA: Association for Computing Machinery,, pp. 1423–1432. doi: 10.1145/3269206.3271793., , 2018

48. Empirical evaluation of gated recurrent neural networks on sequence modeling, K. Cho and, Y. Bengio, J. Chung, C. Gulcehre, arXiv, Dec. 11, 2014. doi: 10.48550/arXiv.1412.3555, , 2014

49. Group-Aware Graph Neural Network for Nationwide City Air Quality Forecasting, L. Chen et al., arXiv, doi: 10.48550/arXiv.2108.12238, , 2021

50. MSSTN: Multi-Scale Spatial Temporal Network for Air Pollution Prediction, in, L. Zhang, Y. Wang and, Z. Wu, 2019 IEEE International Conference on Big Data (Big Data), pp. 1547–1556. doi: 10.1109/BigData47090.2019. 9005574, , 2019

51. Predicting citywide crowd flows using deep spatio-temporal residual networks, J. Zhang, R. Li, T. Li, Y. Zheng, D. Qi, X. Yi and, Artif. Intell., vol. 259, pp. 147–166, doi: 10.1016/j. artint.2018.03.002, , 2018

52. Utilizing Real-World Transportation Data for Accurate Traffic Prediction, in, C. Shahabi, B. Pan, U. Demiryurek and, 2012 IEEE 12th International Conference on Data Mining, pp. 595–604. doi: 10.1109/ICDM.2012.52, , 2012

53. GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction, J. Zhang, S. Ke, Y. Liang, Y. Zheng, X. Yi and, p. 3434. doi: 10.24963/ijcai.2018/476, , 2018

54. Temporal Convolutional Networks: A Unified Approach to Action Segmentation in, G. D. Hager, A. Reiter and, R. Vidal, C. Lea, Computer Vision ECCV 2016 Workshops G. Hua and H. Jégou Eds. Cham: Springer International Publishing, pp. 47–54. doi: 10.1007/978-3-319-49409-8_7., , 2016

55. Air quality prediction using CNN+LSTM-based hybrid deep learning architecture,, A. Ozmen, A. Gilik, A. S. Ogrenci and, vol. 29, no. 8, pp. 11920–11938, doi: 10.1007/s11356-021-16227-w, , 2022

56. Combining kohonen maps with arima time series models to forecast traffic flow,, M. Van Der Voort, M. Dougherty and, S. Watson, Transp. Res. Part C Emerg. Technol., vol. 4, no. 5, pp. 307–318, doi: 10.1016/S0968-090X(97)82903-8, , 1996

57. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering, P. Vandergheynst, M. Defferrard, X. Bresson and, in Advances in Neural Information Processing Systems, Curran Associates, Inc., Accessed: Mar. 20, 2024Online Available https://proceedings. neurips. cc/paper_files/paper/2016/hash/ 04df4d434d481c5bb723be1b6df1ee65-Abstract. html., , 2016

58. GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs, J. Xie, X. Shi, J. Zhang, D.-Y. Yeung, H. Ma, I. King and, arXiv, 20 doi: 10.48550/arXiv.1803.07294, , 2018

59. Predicting Citywide Road Traffic Flow Using Deep Spatiotemporal Neural Networks, P. Yan, T. Jia and, IEEE Trans. Intell. Transp. Syst., vol. 22, no. 5, pp. 3101–3111, doi: 10.1109/TITS.2020.2979634, , 2021

60. Urban Traffic Prediction from Spatio-Temporal Data Using Deep Meta Learning, in, J. Zhang, Y. Yu, Z. Pan, Y. Zheng and, Y. Liang, W. Wang, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, in KDD 19. New York, NY, USA: Association for Computing Machinery, 25 pp. 1720–1730. doi: 10.1145/3292500.3330884., , 2019

61. AST-MTL: An Attention-Based Multi-Task Learning Strategy for Traffic Forecasting, G. Bontempi, B. Lebichot and, G. Buroni, IEEE Access, vol. 9, pp. 77359–77370, 2021, doi: 10.1109/ACCESS.2021.3083412, , 2021

62. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, J. Devlin, K. Lee and, K. Toutanova, M.-W. Chang, arXiv, doi: 10.48550/arXiv.1810.04805, , 2019

63. FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling, T. Ma and, C. Xiao, J. Chen, arXiv, Jan. 30, 2018. doi: 10.48550/arXiv.1801.10247, , 2018

64. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting, H. Zhou et al., Proc. AAAI Conf. Artif. Intell., vol. 35, no. 12, Art. no. 12,, doi: 10.1609/aaai. v35i12.17325, , 2021

65. Air quality forecasting with hybrid LSTM and extended stationary wavelet transform, X. Jin and, Y. Du, Y. Zeng, N. Jin, J. Chen, Build. Environ., vol. 213, p. 108822, doi: 10.1016/j. buildenv.2022.108822, , 2022

66. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting,, C. Shahabi and, Y. Liu, R. Yu, Y. Li, arXiv, Feb. 22, 2018. doi: 10.48550/arXiv.1707.01926, , 2018

67. Ensemble of Deep Neural Networks for Estimating Particulate Matter from Images, in, R. T. Gutta, N. Rijal, J. Zhang, T. Cao, Q. Bo and, J. Lin, 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC), pp. 733–738. doi: 10.1109/ICIVC.2018.8492790, , 2018

68. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting, D. Carpov, Y. Bengio, B. N. Oreshkin, N. Chapados and, arXiv, Feb. 20, 2020. doi: 10.48550/arXiv.1905.10437, , 2020

69. Spatiotemporal Multi-Graph Convolution Network for Ride-Hailing Demand Forecasting, X. Geng et al., Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, Art. no. 01, doi: 10.1609/aaai. v33i01.33013656, , 2019

70. A novel Encoder-Decoder model based on read-first LSTM for air pollutant prediction,, B. Zhang, D. Qin, H. Wang, Y. Jin and, G. Zou, Y. Lu, vol. 765, p. 144507, doi: 10.1016/j. scitotenv.2020.144507, , 2021

71. MICN: Multi-scale Local and Global Context Modeling for Long-term Series Forecasting, H. Wang, Y. Xiao, J. Wang, J. Peng, J. Chen and, F. Huang, presented at the The Eleventh International Conference on Learning Representations, Accessed 2024Online Available https//openreview net/ forum?id=zt53IDUR1U., , 2022

72. Multi-output support vector machine for regional multi-step-ahead PM2.5 forecasting,, C.-C. Kang, I.-F. Kao, L.-C. Chang, Y. Zhou, F.-J. Chang, Y.-S. Wang and, vol. 651, pp. 230–240, doi: 10.1016/ j. scitotenv.2018.09.111., , 2019

73. Deep Spatial–Temporal 3D Convolutional Neural Networks for Traffic Data Forecasting, H. Wan, S. Guo, Y. Lin, Z. Chen and, S. Li, IEEE Trans. Intell. Transp. Syst., vol. 20, no. 10, pp. 3913–3926, doi: 10.1109/TITS. 2019.2906365, , 2019

74. Multi-Range Attentive Bicomponent Graph Convolutional Network for Traffic Forecasting, L. Chen, Y. Gao and, W. Chen, W. Cao, X. Feng, Y. Xie, vol. 34, no. 04, Art. no. 04, doi: 10.1609/ aaai. v34i04.5758., , 2020

75. ADGCN: An Asynchronous Dilation Graph Convolutional Network for Traffic Flow Prediction, T. Qi, Y. Xue, L. Chen and, G. Li, vol. 9, no. 5, pp. 4001–4014, doi: 10.1109/JIOT.2021.3102238, , 2022

76. Automatic Sleep Scoring Using Intrinsic Mode Based on Interpretable Deep Neural Networks, J. Baek et al., IEEE Access, vol. 10, pp. 36895–36906, 2022, doi: 10.1109/ACCESS.2022.3163250, , 2022

77. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks, in, S. Pan, C. Zhang, Z. Wu, J. Jiang, G. Long, X. Chang and, Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, in KDD 20. New York, NY, USA: Association for Computing Machinery, 20 pp. 753–763. doi: 10.1145/3394486.3403118., , 2020

78. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting, in, W. Wong and, W. WOO, Z. Chen, H. Wang, D.-Y. Yeung, X. SHI, Advances in Neural Information Processing Systems, Curran Associates, Inc., Accessed 2024Online Available https, , 2015

79. Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction, X. Tang, H. Wei, G. Zheng and, Z. Li, H. Yao, Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, Art. no. 01, doi: 10.1609/aaai. v33i01.33015668, , 2019

80. A hybrid model for spatial–temporal prediction of PM2.5 based on a time division method,, M. Wang and, B. Liu, H. W. Guesgen, vol. 20, no. 11, pp. 12195–12206,, doi: 10.1007/s13762-023-04813-z, , 2023

81. Hierarchical Traffic Flow Prediction Based on Spatial-Temporal Graph Convolutional Network, L. Yang, H. Wang, X. Cheng and, R. Zhang, IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 16137–16147, doi: 10.1109/ TITS.2022.3148105, , 2022

82. Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks, X. Liu, Y. Li, W.-L. Chiang, C.-J. Hsieh, S. Si, S. Bengio and, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, in KDD 19. New York, NY, USA: Association for Computing Machinery, 25 2019, pp. 257–266. doi: 10.1145/3292500.3330925., , 2019

83. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing in, S. Abu-El-Haija et al., Proceedings of the 36th International Conference on Machine Learning, PMLR, pp. 21–29. Accessed: Nov. 22, 2023Online Available https://proceedings. mlr. press/v97/abu-el-haija19a. html., , 2019

84. Prediction of PM2.5 concentration based on the similarity in air quality monitoring network,, M. Li, H. He, Y. Xue, W. Wang, Z. Wang and, Build. Environ., vol. 137, pp. 11–17, doi: 10.1016/j. buildenv.2018.03.058, , 2018

85. Attention-based parallel networks (APNet) for PM2.5 spatiotemporal prediction - ScienceDirect, J. Zhu, J. Zhao and, H. Zheng, F. Deng, doi: 10.1016/j. scitotenv.2021.145082, , 2021

86. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling,, V. Koltun, S. Bai, J. Z. Kolter and, arXiv, Apr. 19, 2018. doi: 10.48550/arXiv.1803.01271, , 2018

87. AST-GCN: Attribute-Augmented Spatiotemporal Graph Convolutional Network for Traffic Forecasting, Q. Wang, H. Li, J. Zhu, H. Deng, L. Zhao and, C. Tao, IEEE Access, vol. 9, pp. 35973–35983, 2021, doi: 10.1109/ACCESS. 2021.3062114, , 2021

88. Exploiting spatiotemporal patterns for accurate air quality forecasting using deep learning, in, Y. Lin et al., Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, in SIGSPATIAL 18. New York, NY, USA: Association for Computing Machinery,, pp. 359–368. doi: 10.1145/3274895.3274907., , 2018

89. Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting, Z. Zhu, H. Yin and, B. Yu, in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pp. 3634–3640. doi: 10.24963/ijcai.2018/505, , 2018

90. Developing an early-warning system for air quality prediction and assessment of cities in China,, J. Wang, Z. Guo and, H. Lu, X. Zhang, Expert Syst. Appl., vol. 84, pp. 102–116, doi: 10.1016/j. eswa.2017.04.059, , 2017

91. FTPG: A Fine-Grained Traffic Prediction Method With Graph Attention Network Using Big Trace Data, M. Fang, X. Yang, C. Li and, L. Tang, Y. Chen, Q. Li, IEEE Trans. Intell. Transp. Syst., vol. 23, no. 6, pp. 5163–5175, doi: 10.1109/TITS.2021.3049264., , 2022

92. A novel spatiotemporal convolutional long short-term neural network for air pollution prediction,, C. Wen et al., vol. 654, pp. 1091–1099, doi: 10.1016/j. scitotenv.2018.11.086, , 2019

93. Technical Assistance Document for the Reporting of Daily Air Quality – the Air Quality Index (AQI), undefined U. S. E. P. Agency, Environ. Prot., p. undefined-undefined, , 2018