Proceedings

Reconstructing Missing Variables for Multivariate Time Series Forecasting via Conditional Generative Flows

Download BibTeX Cite

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence. Main Track. Pages 2063-2071.

https://doi.org/10.24963/ijcai.2024/228

Abstract

The Variable Subset Forecasting (VSF) problem, where the majority of variables are unavailable in the inference stage of multivariate forecasting, has been an important but under-explored task with broad impacts in many real-world applications. Missing values, absent inter-correlation, and the impracticality of retraining largely hinder the ability of multivariate forecasting models to capture inherent relationships among variables, impacting their performance. However, existing approaches towards these issues either heavily rely on local temporal correlation or face limitations in fully recovering missing information from the unavailable subset, accompanied by notable computational expenses. To address these problems, we propose a novel density estimation solution to recover the information of missing variables via flows-based generative framework. In particular, a novel generative network for time series, namely Time-series Reconstruction Flows (TRF), is proposed to estimate and reconstruct the missing variable subset. In addition, a novel meta-training framework, Variable-Agnostic Meta Learning, has been developed to enhance the generalization ability of TRF, enabling it to adapt to diverse missing variables situations. Finally, extensive experiments are conducted to demonstrate the superiority of our proposed method compared with baseline methods.

  1. [Bai et al., 2018] Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271, 2018.
  2. [Bai et al., 2020] Lei Bai, Lina Yao, Can Li, Xianzhi Wang, and Can Wang. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in neural infor- mation processing systems, 33:17804-17815, 2020.
  3. [Bartholomew, 1971] David J Bartholomew. Time series analysis forecasting and control., 1971.
  4. [Byrne, 2019] Ruth MJ Byrne. Counterfactuals in explain- able artificial intelligence (xai): Evidence from human rea- soning. In IJCAI, pages 6276-6282, 2019. IJCAI
  5. [Chauhan et al., 2022] Jatin Chauhan, Aravindan Raghu- veer, Rishi Saket, Jay Nandy, and Balaraman Ravindran. Multi-variate time series forecasting on variable subsets. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 76-86, 2022.
  6. [Dinh et al., 2014] Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent compo- nents estimation. arXiv preprint arXiv:1410.8516, 2014.
  7. [Dinh et al., 2016] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.
  8. [Domeniconi and Yan, 2004] Carlotta Domeniconi and Bo- jun Yan. Nearest neighbor ensemble. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., volume 1, pages 228-231. IEEE, 2004.
  9. [Du et al., 2019] Shengdong Du, Tianrui Li, Yan Yang, and Shi-Jinn Horng. Deep air quality forecasting using hybrid deep learning framework. IEEE Transactions on Knowl- edge and Data Engineering, 33(6):2412-2424, 2019.
  10. [Du et al., 2023] Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023.
  11. [Du Preez and Witt, 2003] Johann Du Preez and Stephen F Witt. Univariate versus multivariate time series forecast- ing: an application to international tourism demand. Inter- national Journal of Forecasting, 19(3):435-451, 2003.
  12. [Fan et al., 2022] Wei Fan, Shun Zheng, Xiaohan Yi, Wei Cao, Yanjie Fu, Jiang Bian, and Tie-Yan Liu. Depts: Deep expansion learning for periodic time series forecast- ing. arXiv preprint arXiv:2203.07681, 2022.
  13. [Fan et al., 2023] Wei Fan, Pengyang Wang, Dongkun Wang, Dongjie Wang, Yuanchun Zhou, and Yanjie Fu. Dish-ts: a general paradigm for alleviating distribution shift in time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7522-7529, 2023.
  14. [Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1126-1135. PMLR, 2017.
  15. [Germain et al., 2015] Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. Made: Masked autoen- coder for distribution estimation. In International confer- ence on machine learning, pages 881-889. PMLR, 2015.
  16. [Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. Advances in neural information processing systems, 27, 2014.
  17. [Guo et al., 2019] Shengnan Guo, Youfang Lin, Ning Feng, Chao Song, and Huaiyu Wan. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 922-929, 2019.
  18. [Hamid and Wibowo, 2018] Nur Hamid and Wahyu Catur Wibowo. Wind speed forecasting using multivariate time- series radial basis function neural network. In 2018 Inter- national Conference on Advanced Computer Science and Information Systems (ICACSIS), pages 423-428. IEEE, 2018.
  19. [Hamzac ¸ebi, 2008] Cos ¸kun Hamzac ¸ebi. Improving artificial neural networks' performance in seasonal time series fore- casting. Information Sciences, 178(23):4550-4559, 2008.
  20. [Hu et al., 2023] Xuanming Hu, Wei Fan, Kun Yi, Pengfei Wang, Yuanbo Xu, Yanjie Fu, and Pengyang Wang. Boost- ing urban prediction via addressing spatial-temporal dis- tribution shift. In 2023 IEEE International Conference on Data Mining (ICDM), pages 160-169. IEEE, 2023.
  21. [Hu et al., 2024] Xuanming Hu, Wei Fan, Dongjie Wang, Pengyang Wang, Yong Li, and Yanjie Fu. Dual-stage flows-based generative modeling for traceable urban plan- ning. In Proceedings of the 2024 SIAM International Con- ference on Data Mining (SDM), pages 370-378. SIAM, 2024.
  22. [Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In International conference on machine learning, pages 448-456. pmlr, 2015.
  23. [Kingma and Dhariwal, 2018] Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 con- volutions. Advances in neural information processing sys- tems, 31, 2018.
  24. [Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  25. [Li et al., 2017] Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural net- work: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926, 2017.
  26. [Luo et al., 2018] Yonghong Luo, Xiangrui Cai, Ying Zhang, Jun Xu, et al. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems, 31, 2018.
  27. [Miao et al., 2021] Xiaoye Miao, Yangyang Wu, Jun Wang, Yunjun Gao, Xudong Mao, and Jianwei Yin. Generative semi-supervised learning for multivariate time series im- putation. In Proceedings of the AAAI conference on artifi- cial intelligence, volume 35, pages 8983-8991, 2021.
  28. [Nichol and Schulman, 2018] Alex Nichol and John Schul- man. Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999, 2(3):4, 2018.
  29. [Oreshkin et al., 2019] Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series fore- casting. arXiv preprint arXiv:1905.10437, 2019.
  30. [Papamakarios et al., 2017] George Papamakarios, Theo Pavlakou, and Iain Murray. Masked autoregressive flow for density estimation. Advances in neural information processing systems, 30, 2017.
  31. [Ren et al., 2022] Weijieying Ren, Pengyang Wang, Xiaolin Li, Charles E Hughes, and Yanjie Fu. Semi-supervised drifted stream learning with short lookback. In Proceed- ings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1504-1513, 2022.
  32. [Tang et al., 2020] Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Charu Aggarwal, Prasenjit Mitra, and Suhang Wang. Joint modeling of local and global temporal dynamics for mul- tivariate time series forecasting with missing values. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 34, pages 5956-5963, 2020.
  33. [Tashiro et al., 2021] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score- based diffusion models for probabilistic time series impu- tation. Advances in Neural Information Processing Sys- tems, 34:24804-24816, 2021.
  34. [Van Buuren and Groothuis-Oudshoorn, 2011] Stef Van Bu- uren and Karin Groothuis-Oudshoorn. mice: Multivariate imputation by chained equations in r. Journal of statistical software, 45:1-67, 2011.
  35. [Vanschoren, 2018] Joaquin Vanschoren. Meta-learning: A survey. arXiv preprint arXiv:1810.03548, 2018.
  36. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  37. [Wu et al., 2020] Zonghan Wu, Shirui Pan, Guodong Long, Jing Jiang, Xiaojun Chang, and Chengqi Zhang. Con- necting the dots: Multivariate time series forecasting with graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discov- ery & data mining, pages 753-763, 2020.
  38. [Yi et al., 2024a] Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhen- dong Niu. Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. Advances in Neural Information Processing Systems, 36, 2024.
  39. [Yi et al., 2024b] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mlps are more effective learners in time series forecast- ing. Advances in Neural Information Processing Systems, 36, 2024.
  40. [Yick et al., 2008] Jennifer Yick, Biswanath Mukherjee, and Dipak Ghosal. Wireless sensor network survey. Computer networks, 52(12):2292-2330, 2008.
  41. [Yu et al., 2016] Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. Temporal regularized matrix factorization for high-dimensional time series prediction. Advances in neu- ral information processing systems, 29, 2016.
  42. [Zeng et al., 2023] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series fore- casting? In Proceedings of the AAAI conference on artifi- cial intelligence, volume 37, pages 11121-11128, 2023.
  43. [Zhao et al., 2019] Ling Zhao, Yujiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. T-gcn: A temporal graph convolutional network for traffic prediction. IEEE transactions on intelligent transportation systems, 21(9):3848-3858, 2019.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards