Proceedings

SaSDim:Self-Adaptive Noise Scaling Diffusion Model for Spatial Time Series Imputation

Download BibTeX Cite

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. Main Track. Pages 2561-2569.

https://doi.org/10.24963/ijcai.2024/283

Abstract

Spatial time series imputation is of great importance to various real-world applications. As the state-of-the-art generative models, diffusion models (e.g. CSDI) have outperformed statistical and autoregressive based models in time series imputation. However, diffusion models may introduce unstable noise owing to the inherent uncertainty in sampling, leading to the generated noise deviating from the intended Gaussian distribution. Consequently, the imputed data may deviate from the real data. To this end, we propose a Self-adaptive noise Scaling Diffusion Model named SaSDim for spatial time series imputation. Specifically, we introduce a novel Probabilistic High-Order SDE Solver Module to scale the noise following the standard Gaussian distribution. The noise scaling operation helps the noise prediction module of the diffusion model to more accurately estimate the variance of noise. To effectively learn the spatial and temporal features, a Spatial guided Global Convolution Module (SgGConv) for multi-periodic temporal dependencies learning with the Fast Fourier Transformation and dynamic spatial dependencies learning with dynamic graph convolution is also proposed. Extensive experiments conducted on three real-world spatial time series datasets verify the effectiveness of SaSDim.

  1. [Alcaraz and Strodthoff, 2022] Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series impu- tation and forecasting with structured state space models. arXiv preprint arXiv:2208.09399, 2022.
  2. [Cao et al., 2018] Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent im- putation for time series. Advances in neural information processing systems, 31, 2018.
  3. [Che et al., 2018] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific reports, 8(1):6085, 2018.
  4. [Cini et al., 2022] Andrea Cini, Ivan Marisca, and Cesare Alippi. Filling the g ap s: Multivariate time series im- putation by graph neural networks. In ICLR, 2022.
  5. [Du et al., 2023] Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023.
  6. [Goel et al., 2022] Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with state- space models. In Proceedings of International Conference on Machine Learning, pages 7616-7633. PMLR, 2022.
  7. [Gu et al., 2021] Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00396, 2021.
  8. [Hewage et al., 2020] Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. Temporal con- volutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, 24:16453-16482, 2020.
  9. [Li et al., 2022] Yuhong Li, Tianle Cai, Yi Zhang, Deming Chen, and Debadeepta Dey. What makes convolutional models great on long sequence modeling? arXiv preprint arXiv:2210.09298, 2022.
  10. [Lim and Zohren, 2021] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a sur- vey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.
  11. [Luo et al., 2018] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018.
  12. [Marisca et al., 2022] Ivan Marisca, Andrea Cini, and Ce- sare Alippi. Learning to reconstruct missing data from spa- tiotemporal graphs with sparse observations. Advances in Neural Information Processing Systems, 35:32069-32082, 2022.
  13. [Nelson, 1998] Brian K Nelson. Time series analysis using autoregressive integrated moving average (arima) models. Academic emergency medicine, 5(7):739-744, 1998.
  14. [Nguyen et al., 2022] Eric Nguyen, Karan Goel, Albert Gu, Gordon Downs, Preey Shah, Tri Dao, Stephen Baccus, and Christopher Ré. S4nd: Modeling images and videos as multidimensional signals with state spaces. Advances in neural information processing systems, 35:2846-2861, 2022.
  15. [Song et al., 2020] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.
  16. [Tashiro et al., 2021] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score- based diffusion models for probabilistic time series impu- tation. Advances in Neural Information Processing Sys- tems, 34:24804-24816, 2021.
  17. [Wang et al., 2020] Senzhang Wang, Jiannong Cao, and S Yu Philip. Deep learning for spatio-temporal data min- ing: A survey. IEEE transactions on knowledge and data engineering, 34(8):3681-3700, 2020.
  18. [Wang et al., 2021] Qinfen Wang, Siyuan Ren, Yong Xia, and Longbing Cao. Bicmts: Bidirectional coupled mul- tivariate learning of irregular time series with missing val- ues. In Proceedings of the 30th ACM International Con- ference on Information & Knowledge Management, pages 3493-3497, 2021.
  19. [Wang et al., 2022a] Senzhang Wang, Jiyue Li, Hao Miao, Junbo Zhang, Junxing Zhu, and Jianxin Wang. Generative- free urban flow imputation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edge Management, pages 2028-2037, 2022.
  20. [Wang et al., 2022b] Senzhang Wang, Jiyue Li, Hao Miao, Junbo Zhang, Junxing Zhu, and Jianxin Wang. Generative- free urban flow imputation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edge Management, pages 2028-2037, 2022.
  21. [Wu et al., 2022] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Representations, 2022.
  22. [Xu et al., 2022] Xin Xu, Liangliang Zhang, Qi Kong, Chen- guang Gui, and Xing Zhang. Enhanced-historical aver- age for long-term prediction. In 2022 2nd International Conference on Computer, Control and Robotics (ICCCR), pages 115-119. IEEE, 2022.
  23. [Yang et al., 2022] Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applica- tions. arXiv preprint arXiv:2209.00796, 2022.
  24. [Yi et al., 2016] Xiuwen Yi, Yu Zheng, Junbo Zhang, and Tianrui Li. St-mvl: filling missing values in geo-sensory time series data. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016. IJCAI
  25. [Yoon et al., 2017] Jinsung Yoon, William R Zame, and Mi- haela van der Schaar. Multi-directional recurrent neural networks: A novel method for estimating missing data. In Proceedings of Time series workshop in international con- ference on machine learning, 2017.
  26. [Yoon et al., 2018] Jinsung Yoon, James Jordon, and Mi- haela Schaar. Gain: Missing data imputation using genera- tive adversarial nets. In Proceedings of International con- ference on machine learning, pages 5689-5698. PMLR, 2018.
  27. [Yu et al., 2016] Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. Advances in neu- ral information processing systems, 29, 2016.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards