Proceedings

General Incomplete Time Series Analysis via Patch Dropping Without Imputation

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 6597-6605.

https://doi.org/10.24963/ijcai.2025/734

Abstract

Missing values in multivariate time series data present significant challenges to effective analysis. Existing methods for multivariate time series analysis either ignore missing data, sacrificing performance, or follow the impute-then-analyze paradigm, which suffers from redundant training and error accumulation, leading to biased results and suboptimal performance. In this paper, we propose INTER, a novel end-to-end framework for incomplete multivariate time series analysis, which bypasses imputation by leveraging pre-trained language models to learn the distribution of incomplete time series data. INTER incorporates two novel components: the missing-rate-aware time series patch-dropping (MPD) strategy and the missing-aware Transformer block, both of which we propose to enhance model generalization, robustness, and the ability to capture underlying patterns in the observed incomplete time series. Moreover, we theoretically prove that the MPD strategy exhibits lower sample variance for time series with the same dropout rate compared to other dropping strategies. Extensive experiments on 11 public real-world time series datasets demonstrate that INTER improves accuracy by over 20% compared to state-of-the-art methods, while maintaining competitive computational efficiency.

  1. [Bagnall et al., 2018] Anthony Bagnall, Hoang Anh Dau, Ja- son Lines, Michael Flynn, James Large, Aaron Bostrom, Paul Southam, and Eamonn Keogh. The uea multivari- ate time series classification archive, 2018. arXiv preprint arXiv:1811.00075, 2018.
  2. [Bartholomew, 1971] David J Bartholomew. Time series analysis forecasting and control. Journal of the Opera- tional Research Society, 22(2):199-201, 1971.
  3. [Buuren and Groothuis-Oudshoorn, 2010] S van Buuren and Karin Groothuis-Oudshoorn. mice: Multivariate imputa- tion by chained equations in r. Journal of Statistical Soft- ware, pages 1-68, 2010.
  4. [Cao et al., 2018] Wei Cao, Dong Wang, Jian Li, Hao Zhou, Lei Li, and Yitan Li. Brits: Bidirectional recurrent imputa- tion for time series. In NeurIPS, pages 6775-6785, 2018.
  5. [Che et al., 2018] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific Reports, 8(1):6085, 2018.
  6. [Coletta et al., 2024] Andrea Coletta, Sriram Gopalakrish- nan, Daniel Borrajo, and Svitlana Vyetrenko. On the con- strained time-series generation problem. Advances in Neu- ral Information Processing Systems, 36, 2024.
  7. [Du et al., 2023] Wenjie Du, David Côté, and Yan Liu. Saits: Self-attention-based imputation for time series. Expert Systems with Applications, 219:119619, 2023.
  8. [Ekambaram et al., 2023] Vijay Ekambaram, Arindam Jati, Nam Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. Tsmixer: Lightweight mlp-mixer model for multivariate time series forecasting. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 459-469, 2023.
  9. [Feng et al., 2023] Raymond Feng, Flavio P Calmon, and Hao Wang. Adapting fairness interventions to missing val- ues. In NeurIPS, pages 59388-59409, 2023.
  10. [Feng et al., 2024] Yu Feng, Zhen Tian, Yifan Zhu, Zongfu Han, Haoran Luo, Guangwei Zhang, and Meina Song. Cp- prompt: Composition-based cross-modal prompting for domain-incremental continual learning. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 2729-2738, 2024.
  11. [Feng et al., 2025] Yu Feng, Yangli-ao Geng, Yifan Zhu, Zongfu Han, Xie Yu, Kaiwen Xue, Haoran Luo, Mengyang Sun, Guangwei Zhang, and Meina Song. Pm- moe: Mixture of experts on private model parameters for personalized federated learning. In Proceedings of the ACM on Web Conference 2025, pages 134-146, 2025.
  12. [Franceschi et al., 2019] Jean-Yves Franceschi, Aymeric Dieuleveut, and Martin Jaggi. Unsupervised scalable representation learning for multivariate time series. In NeurIPS, volume 32, pages 1-12, 2019.
  13. [Gasparin et al., 2022] Alberto Gasparin, Slobodan Lukovic, and Cesare Alippi. Deep learning for time series forecast- ing: The electric load case. CAAI Transactions on Intelli- gence Technology, 7(1):1-25, 2022.
  14. [Goel et al., 2022] Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré. It's raw! audio generation with state- space models. In ICML, pages 7616-7633, 2022.
  15. [Goldberger et al., 2000] Ary L Goldberger, Luis AN Ama- ral, Leon Glass, Jeffrey M Hausdorff, Plamen Ch Ivanov, Roger G Mark, Joseph E Mietus, George B Moody, Chung-Kang Peng, and H Eugene Stanley. Physiobank, physiotoolkit, and physionet: components of a new re- search resource for complex physiologic signals. circu- lation, 101(23):e215-e220, 2000.
  16. [Kraskov et al., 2004] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical Review E-Statistical, Nonlinear, and Soft Matter Physics, 69(6):066138, 2004.
  17. [Li et al., 2022] Yan Li, Xinjiang Lu, Yaqing Wang, and De- jing Dou. Generative time series forecasting with diffu- sion, denoise, and disentanglement. Advances in Neural Information Processing Systems, 35:23009-23022, 2022.
  18. [Liang et al., 2024] Yuxuan Liang, Haomin Wen, Yuqi Nie, Yushan Jiang, Ming Jin, Dongjin Song, Shirui Pan, and Qingsong Wen. Foundation models for time series anal- ysis: A tutorial and survey. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 6555-6565, 2024.
  19. [Liao et al., 2024] Weibin Liao, Yifan Zhu, Yanyan Li, Qi Zhang, Zhonghong Ou, and Xuesong Li. Revgnn: Negative sampling enhanced contrastive graph learning for academic reviewer recommendation. ACM Transactions on Information Systems, 43(1):1-26, 2024.
  20. [Lim and Zohren, 2021] Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a sur- vey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.
  21. [Liu et al., 2024] Yong Liu, Haoran Zhang, Chenyu Li, Xi- angdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative pre-trained transformers are large time series models. In ICML, 2024.
  22. [Makridakis, 2018] Spyros Makridakis. M4 dataset, 2018. https://github.com/M4Competition/M4-methods/ tree/master/Dataset (Accessed: 2025-01-24). Link
  23. [Miao et al., 2021] Xiaoye Miao, Yangyang Wu, Jun Wang, Yunjun Gao, Xudong Mao, and Jianwei Yin. Generative semi-supervised learning for multivariate time series im- putation. In AAAI, pages 8983-8991, 2021.
  24. [Miao et al., 2022] Xiaoye Miao, Yangyang Wu, Lu Chen, Yunjun Gao, Jun Wang, and Jianwei Yin. Efficient and effective data imputation with influence functions. Pro- ceedings of the VLDB Endowment, 15(3):624-632, 2022.
  25. [Moody et al., 2011] George B Moody, Roger G Mark, and Ary L Goldberger. Physionet: Physiologic signals, time series and related open source software for basic, clinical, and applied research. In 2011 Annual International Con- ference of the IEEE Engineering in Medicine and Biology Society, pages 8327-8330. IEEE, 2011.
  26. [Nie et al., 2022] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In ICLR, pages 1-24, 2022.
  27. [Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9, 2019.
  28. [Silva et al., 2012] Ikaro Silva, George Moody, Daniel J Scott, Leo A Celi, and Roger G Mark. Predict- ing in-hospital mortality of icu patients: The phys- ionet/computing in cardiology challenge 2012. In CINC, pages 245-248, 2012.
  29. [Tashiro et al., 2021] Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. In NeurIPS, volume 34, pages 24804-24816, 2021.
  30. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 1-11, 2017.
  31. [Wang et al., 2022] Heyuan Wang, Tengjiao Wang, Shun Li, Jiayi Zheng, Shijie Guan, and Wei Chen. Adaptive long- short pattern transformer for stock investment selection. In IJCAI, pages 3970-3977, 2022.
  32. [Wen et al., 2022] Qingsong Wen, Linxiao Yang, Tian Zhou, and Liang Sun. Robust time series analysis and applica- tions: An industrial perspective. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4836-4837, 2022.
  33. Wu et al., ] Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. In The Eleventh International Conference on Learning Rep- resentations.
  34. [Wu et al., 2022] Yangyang Wu, Jun Wang, Xiaoye Miao, Wenjia Wang, and Jianwei Yin. Differentiable and scalable generative adversarial models for data imputation. ArXiv Preprint ArXiv:2201.03202, 2022.
  35. [Wu et al., 2023] Yangyang Wu, Xiaoye Miao, Xinyu Huang, and Jianwei Yin. Jointly imputing multi-view data with optimal transport. In AAAI, volume 37, pages 4747- 4755, 2023.
  36. [Yi et al., 2023a] Kun Yi, Qi Zhang, Wei Fan, Hui He, Liang Hu, Pengyang Wang, Ning An, Longbing Cao, and Zhen- dong Niu. Fouriergnn: Rethinking multivariate time series forecasting from a pure graph perspective. Advances in neural information processing systems, 36:69638-69660, 2023.
  37. [Yi et al., 2023b] Kun Yi, Qi Zhang, Wei Fan, Shoujin Wang, Pengyang Wang, Hui He, Ning An, Defu Lian, Longbing Cao, and Zhendong Niu. Frequency-domain mlps are more effective learners in time series forecast- ing. Advances in Neural Information Processing Systems, 36:76656-76679, 2023.
  38. [Yi et al., 2024] Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, and Wei Fan. Filternet: Harness- ing frequency filters for time series forecasting. Advances in Neural Information Processing Systems, 37:55115- 55140, 2024.
  39. [Zerveas et al., 2021] George Zerveas, Srideepika Jayara- man, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 2114-2124, 2021.
  40. [Zhang and Berardi, 2001] Guoqiang Peter Zhang and Vic- tor L Berardi. Time series forecasting with neural network ensembles: An application for exchange rate prediction. Journal of the Operational Research Society, 52:652-664, 2001.
  41. [Zhang et al., 2023] Kai Zhang, Chao Li, and Qinmin Yang. Trid-mae: A generic pre-trained model for multivariate time series with missing values. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 3164-3173, 2023.
  42. [Zhou et al., 2021] Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long se- quence time-series forecasting. In AAAI, volume 35, pages 11106-11115, 2021.
  43. [Zhou et al., 2023a] Linqi Zhou, Michael Poli, Winnie Xu, Stefano Massaroli, and Stefano Ermon. Deep latent state space models for time-series generation. In ICML, pages 42625-42643, 2023.
  44. [Zhou et al., 2023b] Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series anal- ysis by pretrained lm. NeurIPS, 36:43322-43355, 2023.
  45. [Zhu et al., 2023] Yifan Zhu, Fangpeng Cong, Dan Zhang, Wenwen Gong, Qika Lin, Wenzheng Feng, Yuxiao Dong, and Jie Tang. Wingnn: dynamic graph neural networks with random gradient aggregation window. In Proceed- ings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, pages 3650-3662, 2023.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards