Proceedings

MoTRa: Motion-Aware Target Representation Learning for End-to-End Multi-Object Tracking

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Main Track. Pages 1224-1232.

https://doi.org/10.24963/ijcai.2026/137

Abstract

Multi-object tracking (MOT) has long faced challenges with identity switches, especially for targets with low appearance discriminability and complex motion. Existing end-to-end trackers typically enhance robustness by modeling long-term temporal information across target-level representations, yet these representations remain insufficiently discriminative for spatially and visually similar targets. In this paper, we present MoTRa, a Motion-aware Target Representation learning framework for end-to-end MOT that enriches target-level representations with adaptive motion cues. As a result, each representation adaptively balances motion cues and appearance-related content cues under varying conditions, enhancing its discriminability for tracking. To achieve this, we propose a feature fusion and alignment module that extracts motion cues and adaptively fuses them with content features into target representations. To further regularize the dynamically fused representations, we propose a target-specific contrastive learning strategy that promotes intra-trajectory consistency and inter-target separability. Experimental results demonstrate the effectiveness of our method, achieving competitive performance on multiple benchmarks.

  1. [Bernardin and Stiefelhagen, 2008] Keni Bernardin and Rainer Stiefelhagen. Evaluating multiple object tracking performance: the clear mot metrics. EURASIP Journal on Image and Video Processing, 2008:1-10, 2008.
  2. [Bewley et al., 2016] Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. Simple online and realtime tracking. In 2016 IEEE International Conference on Image Processing, pages 3464-3468. IEEE, 2016.
  3. [Carion et al., 2020] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision, pages 213-229. Springer, 2020.
  4. [Cui et al., 2023] Yutao Cui, Chenkai Zeng, Xiaoyu Zhao, Yichun Yang, Gangshan Wu, and Limin Wang. Sportsmot: A large multi-object tracking dataset in multiple sports scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9921-9931, 2023.
  5. [Gao and Wang, 2023] Ruopeng Gao and Limin Wang. Memotr: Long-term memory-augmented transformer for multi-object tracking. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 9901- 9910, 2023.
  6. [Gao et al., 2025] Ruopeng Gao, Ji Qi, and Limin Wang. Multiple object tracking as id prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 27883-27893, 2025.
  7. [Ge et al., 2021] Zheng Ge, Songtao Liu, Feng Wang, Zem- ing Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430, 2021.
  8. [He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for un- supervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 9729-9738, 2020.
  9. [Huang et al., 2025a] Cheng Huang, Shoudong Han, Mengyu He, Wenbo Zheng, and Yuhao Wei. Dftrack: Deconfused data association framework for multi-object tracking. IEEE Transactions on Circuits and Systems for Video Technology, 35(10):10367-10381, 2025.
  10. [Huang et al., 2025b] Yuanzhou Huang, Songwei Pei, and Rui Zeng. Dqformer: Transformer with decoupled query augmentations for end-to-end multi-object tracking. ACM Transactions on Multimedia Computing, Communications and Applications, 21(6):1-23, 2025.
  11. [Li et al., 2024] Haijun Li, Zhuye Xu, Changxi Ma, and Xiao Tang. Multi-object tracking algorithm for unmanned vehicle autonomous driving scene based on online spa- tiotemporal feature correlation. IEEE Access, 12:116489- 116497, 2024.
  12. [Liu et al., 2022] Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, and Lei Zhang. Dab-detr: Dynamic anchor boxes are better queries for detr. arXiv preprint arXiv:2201.12329, 2022.
  13. [Liu et al., 2025] Yue Liu, Shijie Zhang, Huayi Li, and Chao Zhang. Satsort: Online multi-spacecraft visual tracking algorithm. Acta Astronautica, 232:103-113, 2025.
  14. [Luiten et al., 2021] Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixé, and Bastian Leibe. Hota: A higher order metric for evalu- ating multi-object tracking. International journal of com- puter vision, 129:548-578, 2021.
  15. [Meinhardt et al., 2022] Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer. Track- former: Multi-object tracking with transformers. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8844-8854, 2022.
  16. [Mu and Lin, 2025] Siyuan Mu and Sen Lin. A comprehen- sive survey of mixture-of-experts: Algorithms, theory, and applications. arXiv preprint arXiv:2503.07137, 2025.
  17. [Qin et al., 2024] Zheng Qin, Le Wang, Sanping Zhou, Pan- pan Fu, Gang Hua, and Wei Tang. Towards generalizable multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18995-19004, 2024.
  18. [Ristani et al., 2016] Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance mea- sures and a data set for multi-target, multi-camera track- ing. In Computer Vision -ECCV 2016 Workshops, pages 17-35, Cham, 2016. Springer International Publishing.
  19. [Segu et al., 2024] Mattia Segu, Luigi Piccinelli, Siyuan Li, Yung-Hsu Yang, Bernt Schiele, and Luc Van Gool. Samba: Synchronized set-of-sequences modeling for multiple ob- ject tracking. arXiv preprint arXiv:2410.01806, 2024.
  20. [Seidenschwarz et al., 2023] Jenny Seidenschwarz, Guillem Brasó, Victor Castro Serrano, Ismail Elezi, and Laura Leal-Taixé. Simple cues lead to a strong multi-object tracker. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13813- 13823, 2023.
  21. [Shao et al., 2018] Shuai Shao, Zijian Zhao, Boxun Li, Tete Xiao, Gang Yu, Xiangyu Zhang, and Jian Sun. Crowdhu- man: A benchmark for detecting human in a crowd. arXiv preprint arXiv:1805.00123, 2018.
  22. [Shim et al., 2025] Kyujin Shim, Kangwook Ko, Yujin Yang, and Changick Kim. Focusing on tracks for online multi-object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11687-11696, 2025.
  23. [Sun et al., 2022] Peize Sun, Jinkun Cao, Yi Jiang, Zehuan Yuan, Song Bai, Kris Kitani, and Ping Luo. Dancetrack: Multi-object tracking in uniform appearance and diverse motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20993- 21002, 2022.
  24. [Wojke et al., 2017] Nicolai Wojke, Alex Bewley, and Diet- rich Paulus. Simple online and realtime tracking with a deep association metric. In 2017 IEEE International Con- ference on Image Processing, pages 3645-3649. IEEE, 2017.
  25. [Yang et al., 2023] Fan Yang, Shigeyuki Odashima, Shoichi Masui, and Shan Jiang. Hard to track objects with ir- regular motions and similar appearances? make it easier by buffering the matching space. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 4799-4808, 2023.
  26. [Yang et al., 2024] Mingzhan Yang, Guangxin Han, Bin Yan, Wenhua Zhang, Jinqing Qi, Huchuan Lu, and Dong Wang. Hybrid-sort: Weak cues matter for online multi- object tracking. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 6504-6512, 2024.
  27. [Yu et al., 2023] En Yu, Tiancai Wang, Zhuoling Li, Yuang Zhang, Xiangyu Zhang, and Wenbing Tao. Motrv3: Release-fetch supervision for end-to-end multi-object tracking. arXiv preprint arXiv:2305.14298, 2023.
  28. [Zeng et al., 2022] Fangao Zeng, Bin Dong, Yuang Zhang, Tiancai Wang, Xiangyu Zhang, and Yichen Wei. Motr: End-to-end multiple-object tracking with transformer. In European Conference on Computer Vision, pages 659- 675. Springer, 2022.
  29. [Zeng et al., 2025] Rui Zeng, Yuanzhou Huang, and Song- wei Pei. Tgformer: Transformer with track query group for multi-object tracking. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 39, pages 9824- 9832, 2025.
  30. [Zhang et al., 2023] Yuang Zhang, Tiancai Wang, and Xi- angyu Zhang. Motrv2: Bootstrapping end-to-end multi- object tracking by pretrained object detectors. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22056-22065, 2023.
  31. [Zhang et al., 2024] Yang Zhang, Xiaoyan Cui, and Yu Bai. Research on an improved motr-gru multi-target tracking and behavior analysis method. In 2024 4th International Symposium on Artificial Intelligence and Intelligent Man- ufacturing, pages 623-626, 2024.
  32. [Zhang et al., 2025] Bozhou Zhang, Nan Song, Xin Jin, and Li Zhang. Bridging past and future: End-to-end au- tonomous driving with historical prediction and planning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6854-6863, 2025.
  33. [Zheng et al., 2024] Guangze Zheng, Shijie Lin, Haobo Zuo, Changhong Fu, and Jia Pan. Nettrack: Tracking highly dy- namic objects with a net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19145-19155, 2024.
  34. [Zhu et al., 2020] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards