Proceedings

TrajCogn: Leveraging LLMs for Cognizing Movement Patterns and Travel Purposes from Trajectories

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 3698-3706.

https://doi.org/10.24963/ijcai.2025/411

Abstract

Spatio-temporal trajectories are crucial for data mining tasks, requiring versatile learning methods that can accurately extract movement patterns and travel purposes. While large language models (LLMs) have shown remarkable versatility through training on extensive datasets, and trajectories share similarities with natural language, standard LLMs cannot directly handle spatio-temporal features or extract trajectory-specific information.

We propose TrajCogn, a model that effectively adapts LLMs for trajectory learning. TrajCogn incorporates a novel trajectory semantic embedder to process spatio-temporal features and extract movement patterns and travel purposes, along with a trajectory prompt that integrates this information into LLMs for various downstream tasks. Experiments on three real-world datasets and four representative tasks demonstrate TrajCogn's effectiveness.

  1. [Brown et al., 2020] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Win- ter, Christopher Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few- shot learners. In NeurIPS, 2020.
  2. [Chang et al., 2023] Yanchuan Chang, Jianzhong Qi, Yux- uan Liang, and Egemen Tanin. Contrastive trajectory sim- ilarity learning with dual-feature attention. In ICDE, pages 2933-2945, 2023.
  3. [Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey E. Hinton. A simple frame- work for contrastive learning of visual representations. In ICML, volume 119, pages 1597-1607, 2020.
  4. [Chen et al., 2021] Yile Chen, Xiucheng Li, Gao Cong, Zhifeng Bao, Cheng Long, Yiding Liu, Arun Kumar Chan- dran, and Richard Ellison. Robust road network represen- tation learning: When traffic patterns meet traveling se- mantics. In CIKM, pages 211-220, 2021.
  5. [Chen et al., 2022a] Wei Chen, Huaiyu Wan, Shengnan Guo, Haoyu Huang, Shaojie Zheng, Jiamu Li, Shuohao Lin, and Youfang Lin. Building and exploiting spatial- temporal knowledge graph for next poi recommendation. Knowledge-Based Systems, 258:109951, 2022.
  6. [Chen et al., 2022b] Zebin Chen, Xiaolin Xiao, Yue-Jiao Gong, Jun Fang, Nan Ma, Hua Chai, and Zhiguang Cao. Interpreting trajectories from multiple views: A hierarchi- cal self-attention network for estimating the time of arrival. In KDD, pages 2771-2779, 2022.
  7. [Chen et al., 2025] Wei Chen, Haoyu Huang, Zhiyu Zhang, Tianyi Wang, Youfang Lin, Liang Chang, and Huaiyu Wan. Next-poi recommendation via spatial-temporal knowledge graph contrastive learning and trajectory prompt. IEEE Transactions on Knowledge and Data En- gineering, 2025.
  8. [Dai and Le, 2015] Andrew M. Dai and Quoc V. Le. Semi- supervised sequence learning. In NeurIPS, pages 3079- 3087, 2015.
  9. [Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understand- ing. In NAACL, pages 4171-4186, 2019.
  10. [Du et al., 2022] Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. GLM: gen- eral language model pretraining with autoregressive blank infilling. In ACL, pages 320-335, 2022.
  11. [Fang et al., 2022a] Ziquan Fang, Yuntao Du, Xinjun Zhu, Danlei Hu, Lu Chen, Yunjun Gao, and Christian S. Jensen. Spatio-temporal trajectory similarity learning in road net- works. In KDD, pages 347-356, 2022.
  12. [Fang et al., 2022b] Ziquan Fang, Yuntao Du, Xinjun Zhu, Danlei Hu, Lu Chen, Yunjun Gao, and Christian S. Jensen. Spatio-temporal trajectory similarity learning in road net- works. In KDD, pages 347-356, 2022.
  13. [Feng et al., 2018] Jie Feng, Yong Li, Chao Zhang, Funing Sun, Fanchao Meng, Ang Guo, and Depeng Jin. Deep- move: Predicting human mobility with attentional recur- rent networks. In WWW, pages 1459-1468, 2018.
  14. [Fu and Lee, 2020] Tao-Yang Fu and Wang-Chien Lee. Trembr: Exploring road networks for trajectory repre- sentation learning. ACM Trans. Intell. Syst. Technol., 11(1):10:1-10:25, 2020.
  15. [Han et al., 2022] Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. PTR: prompt tuning with rules for text classification. AI Open, 3:182-192, 2022.
  16. [Hinton and Salakhutdinov, 2006] Geoffrey E Hinton and Ruslan R Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504-507, 2006.
  17. [Hu et al., 2022] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022.
  18. [Jiang et al., 2023] Jiawei Jiang, Dayan Pan, Houxing Ren, Xiaohan Jiang, Chao Li, and Jingyuan Wang. Self- supervised trajectory representation learning with tempo- ral regularities and travel semantics. In ICDE, pages 843- 855, 2023.
  19. [Jin et al., 2023] Ming Jin, Shiyu Wang, Lintao Ma, Zhix- uan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728, 2023.
  20. [Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.
  21. [Kong and Wu, 2018] Dejiang Kong and Fei Wu. HST- LSTM: A hierarchical spatial-temporal long-short term memory network for location prediction. In IJCAI, pages 2341-2347, 2018. IJCAI
  22. [Li et al., 2018a] Xiucheng Li, Kaiqi Zhao, Gao Cong, Christian S. Jensen, and Wei Wei. Deep representation learning for trajectory similarity computation. In ICDE, pages 617-628, 2018.
  23. [Li et al., 2018b] Yaguang Li, Kun Fu, Zheng Wang, Cyrus Shahabi, Jieping Ye, and Yan Liu. Multi-task representa- tion learning for travel time estimation. In KDD, pages 1695-1704, 2018.
  24. [Li et al., 2024a] Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin, and Chao Huang. Urbangpt: Spatio-temporal large language models. In KDD, pages 5351-5362. ACM, 2024.
  25. [Li et al., 2024b] Zhonghang Li, Lianghao Xia, Yong Xu, and Chao Huang. Flashst: A simple and universal prompt- tuning framework for traffic prediction. In ICML, 2024.
  26. [Liang et al., 2022] Yuxuan Liang, Kun Ouyang, Yiwei Wang, Xu Liu, Hongyang Chen, Junbo Zhang, Yu Zheng, and Roger Zimmermann. Trajformer: Efficient trajectory classification with transformers. In CIKM, pages 1229- 1237, 2022.
  27. [Lin et al., 2021] Yan Lin, Huaiyu Wan, Shengnan Guo, and Youfang Lin. Pre-training context and time aware lo- cation embeddings from spatial-temporal trajectories for user next location prediction. In AAAI, pages 4241-4248, 2021.
  28. [Lin et al., 2023] Yan Lin, Huaiyu Wan, Shengnan Guo, Jilin Hu, Christian S Jensen, and Youfang Lin. Pre-training gen- eral trajectory embeddings with maximum multi-view en- tropy coding. IEEE Trans. Knowl. Data Eng., 2023.
  29. [Liu et al., 2020] Yiding Liu, Kaiqi Zhao, Gao Cong, and Zhifeng Bao. Online anomalous trajectory detection with deep generative sequence modeling. In ICDE, pages 949- 960, 2020.
  30. [Miao et al., 2020] Congcong Miao, Jilong Wang, Heng Yu, Weichen Zhang, and Yinyao Qi. Trajectory-user linking with attentive recurrent network. In AAMAS, pages 878- 886, 2020.
  31. [Pang et al., 2023] Ziqi Pang, Ziyang Xie, Yunze Man, and Yu-Xiong Wang. Frozen transformers in language mod- els are effective visual encoder layers. arXiv preprint arXiv:2310.12973, 2023.
  32. [Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In NeurIPS, pages 8024-8035, 2019.
  33. [Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  34. [Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67, 2020.
  35. [Shen et al., 2025] Zekai Shen, Haitao Yuan, Xiaowei Mao, Congkang Lv, Shengnan Guo, Youfang Lin, and Huaiyu Wan. Towards an efficient and effective en route travel time estimation framework. arXiv preprint arXiv:2504.04086, 2025.
  36. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.
  37. [Wang et al., 2018] Dong Wang, Junbo Zhang, Wei Cao, Jian Li, and Yu Zheng. When will you arrive? estimat- ing travel time based on deep neural networks. In AAAI, pages 2500-2507, 2018.
  38. [Yang et al., 2021a] Peilun Yang, Hanchen Wang, Ying Zhang, Lu Qin, Wenjie Zhang, and Xuemin Lin. T3S: effective representation learning for trajectory similarity computation. In ICDE, pages 2183-2188, 2021.
  39. [Yang et al., 2021b] Sean Bin Yang, Chenjuan Guo, Jilin Hu, Jian Tang, and Bin Yang. Unsupervised path representa- tion learning with curriculum negative sampling. In IJCAI, pages 3286-3292, 2021. IJCAI
  40. [Yang et al., 2023] Sean Bin Yang, Jilin Hu, Chenjuan Guo, Bin Yang, and Christian S. Jensen. Lightpath: Lightweight and scalable path representation learning. In KDD, pages 2999-3010, 2023.
  41. [Yao et al., 2017] Di Yao, Chao Zhang, Zhihua Zhu, Jian- Hui Huang, and Jingping Bi. Trajectory clustering via deep representation learning. In IJCNN, pages 3880-3887, 2017.
  42. [Yao et al., 2019] Di Yao, Gao Cong, Chao Zhang, and Jing- ping Bi. Computing trajectory similarity in linear time: A generic seed-guided neural metric learning approach. In ICDE, pages 1358-1369, 2019.
  43. [Zheng et al., 2008] Yu Zheng, Longhao Wang, Ruochi Zhang, Xing Xie, and Wei-Ying Ma. Geolife: Manag- ing and understanding your past life over maps. In MDM, pages 211-212, 2008.
  44. [Zhou et al., 2018] Fan Zhou, Qiang Gao, Goce Tra- jcevski, Kunpeng Zhang, Ting Zhong, and Fengli Zhang. Trajectory-user linking via variational autoencoder. In IJ- CAI, pages 3212-3218, 2018. IJCAI
  45. [Zhou et al., 2023] Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One fits all: Power general time series analysis by pretrained LM. In NeurIPS, 2023.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards