TrajAR: Long-Term Trajectory Prediction at Urban Intersections via Multi-scale Interaction Perception

1 Introduction

Accurate trajectory prediction of multiple road usersincluding motorized and non-motorized vehicles and pedestrians-represents a critical task in cooperative vehicleinfrastructure systems (CVIS), and has extensive applications in intelligent transportation systems (ITS), such as intelligent surveillance and traffic strategy simulation [Wang et al., 2025a; Jiang et al., 2025; Rudenko et al., 2019; Goldhammer et al., 2018; Li et al., 2022; Shen et al., 2025]. This study focuses on multiple road users' trajectory prediction, analyzing the trajectory of road users derived from unmanned aerial vehicles (UAVs) recorded videos. These UAVs are deployed as roadside perception devices to record trajectories at intersections.

∗Corresponding author.

One challenge of multiple road users' trajectory prediction stems from the complexity of real-world traffic.

Fig. 1(a) presents two scenes at urban intersections, where each road user includes 4 timesteps of future motion. In Scene #1, the road users do not interact with each other, and their future motion can be simply predicted based on the historical motion trend. Scene #2 is much more complicated with interactions between multiple road users. Specifically, left-turning Veh. #A will interact with straight-moving Veh. #B in the next 2 timesteps and give way to Veh. #B. Based on the motion trend and positions of Veh. #C and Peds. #A at the 3rd timestep, they do not interfere with Veh. #A's left-turn motion, thus their future motion will not interfere with that of Veh. #A. At timestep 4, the motion of Veh. #C and Veh. #E indicates potential collision risk, requiring the turning of Veh. #E to give way and avoid interfering with the straight-moving Veh. #C. Meanwhile, Veh. #D's right-turn motion does not obstruct Peds #A's path even though they are close, thus their motion can be regarded as non-interactive in this scenario. Accurate multiple road users' trajectory prediction necessitates incorporating implicit interactions derived from both dynamic information (including signals, road users' positions, motion trends) and static information, including road users' types and traffic restrictions (e.g., rules where turning vehicles give way to straight-moving traffic, right-turning vehicles give way to left-turning ones in countries following right-hand traffic (RHT), and all vehicles give way to pedestrians). While existing methods demonstrate satisfactory performance in simple scenarios like that in Scene #1, they struggle to incorporate the implicit interactions like that in Scene #2. Thus, their effectiveness in real-world traffic environments remains limited. [Bhattacharyya et al., 2024;

(a) Intersections scenes

(b) Comparison of different output frameworks

Figure 1: (a) illustrates the interactions among road users in a simple Scene #1 and a complex Scene #2, with each scenario showing the positions of all road users over the next four timesteps. (b) presents three decoding strategies for trajectory prediction: autoregressive, non-autoregressive, and our proposed multi-scale trajectory prediction approach.

Jiao et al., 2023; Chen and Wang, 2024]

Another challenge lies in achieving accurate long-term trajectory prediction. Existing trajectory prediction methods typically adopt one of the two frameworks shown in Fig. 1(b), either autoregressive or non-autoregressive. [Wei et al., 2024; Westny et al., 2023] The autoregressive approach suffers from error accumulation, leading to significant deviation in the latter part of the predicted trajectory. On the other hand, the non-autoregressive method predicts all future steps in parallel, which tends to introduce bias into the whole predicted trajectory. Each prediction step lacks correction mechanisms with insights from the previous step, risking violation of kinematic constraints, such as abrupt acceleration changes or unrealistic turning radii.

To address these challenges, we propose a long-term Trajectory prediction multi-scale AutoRegressive framework (TrajAR), which follows an encoder-decoder structure. The TrajAR encoder is a dynamic perception encoder that models their motion trends, road user types, and signals, incorporating implicit constraints. The TrajAR decoder introduces a multiscale trajectory prediction structure inspired by VAR [Tian et al., 2024] in image generation. As illustrated in Fig. 1(b), the TrajAR decoder progressively generates trajectories from a coarser scale rn to a finer scale rn+1. The coarse-scale predictions form the backbone of long-term trajectories and receive more frequent supervision to minimize error accumulation, while finer-scale predictions maintain precision through cubic interpolation constraints based on previous scales. These designs together enable both dynamic perception of trajectory and long-term forecasting, allowing TrajAR to understand complex scenarios while delivering precise trajectory predictions effectively. Our contributions are summarized as follows:

• We reformulate autoregressive trajectory prediction from next-step to next-scale, enabling coarse-to-fine trajectory generation. This approach reduces error accumulation and improves long-term prediction accuracy.
• A motion MoE module and a GAT-based interaction layer are introduced to jointly model road user types, motion trends, traffic signals, and implicit interactions. This allows TrajAR to better handle complex scenarios at intersections.
• Extensive experiments on four real-world intersection datasets show that TrajAR outperforms state-of-the-art meth-

ods by up to 32.0% in FDE and 29.8% in APDE, demonstrating its robustness and generalization ability.

2 RelatedWork

Trajectory prediction for road users is vital in complex urban environments and can be broadly categorized into two scenarios: highway on-ramps/off-ramps and intersections.

Trajectory Prediction at HighwayOn/Off-Ramps In highway on-ramp and off-ramp scenarios, researchers primarily focus on predicting vehicle behaviors such as lane changing and acceleration or deceleration, while accounting for the influence of surrounding vehicles on target vehicle motion. Notable approaches include CS-LSTM [Deo and Trivedi, 2018] and PiP [Song et al., 2020], which introduce social pooling layers to capture inter-vehicle interactions. WSiP [Wang et al., 2023] advances this concept by modeling interactions as wave superposition through a wave-pooling mechanism, which C2F-TP [Wang et al., 2025b] further enhances using diffusionbased trajectory generation. BAT [Liao et al., 2024b] employs dynamic geometric graphs for driving behavior modeling, while HLTP [Liao et al., 2024a] incorporates vision-aware pooling with a knowledge distillation framework to refine interaction representations. These scenarios, however, typically involve only one type of vehicle, resulting in relatively homogeneous interaction patterns. Moreover, the datasets used in these studies are predominantly collected under free-flowing traffic conditions, lacking validation in congested or highly interactive environments that are targeted in this paper.

Trajectory Prediction at Intersections Trajectory prediction at intersections is highly challenging due to multiple road users (vehicles, bicycles, pedestrians) and explicit traffic regulations (e.g., traffic lights, right-of-way). These scenarios involve diverse and dynamic interaction patterns, where accurate prediction must consider not only road users' motion trends and kinematic constraints but also their implicit interactions with various surrounding road users. Early efforts [Abdelraouf et al., 2022; Kawasaki and Tasaki, 2018] primarily focuses on single road user prediction using LSTMbased or physics-based models, but fails to capture multiple road users' dynamics. Addressing this limitation, GRIP++ [Li et al., 2019] and LG-ODE [Huang et al., 2020] utilize GNNs to encode multi-agent scenes and apply RNN- or neural ODEbased decoders for prediction. MTP-GO [Westny et al., 2023] introduces multi-modal probabilistic outputs using mixture density networks and Kalman filtering, while FJMP [Rowe et al., 2023] combines attention mechanisms with sparse interaction graphs based on influencer-reactor relationships to improve future interaction modeling. Recognizing the impact of traffic signals, KI-GAN [Wei et al., 2024] incorporates vehicle motion, signal states, and spatial relations through multiple encoders and an attention pooling module, enabling more accurate prediction under traffic regulations. Furthermore, intersections frequently experience congestion and conflict points, further complicating accurate long-term forecasting. While many models perform well in controlled or highwaylike environments, their generalization to complex, multiple road user urban intersections remains limited.

3 Preliminary

Trajectory Features zν t of a road user ν at time t include road users' motion positions τνt = (xνt , yν t ) and the traffic signal state st ∈ RNs at time t. Road users include the target road user νT , and road users νI ∈ VI interacting with it (denoted as interactive road users). Each road user has a type attribute, such as vehicle, bicycle, pedestrian, etc., represented as a one-hot vector av, and Na is the number of unique types. The traffic signal state lt ∈ RNl is a one-hot vector, representing one of the states among all possible combinations of traffic signals at the intersection. The set VI of interaction road users is selected as the NR road users closest to the target road user.

Problem Definition The trajectory prediction problem is formulated as estimating the future motion positions ⟨τνT t0+1 , τνT t0+2 , . . . , τνT t0+tf ⟩ of the target road user given the history H, where tf is the prediction horizon and t0 is the current time. The history H = ⟨zν t0-th+1 , zν t0-th+2 , . . . , zν t0 ⟩, ν ∈ VI ∪ {νT } consist of the history trajectory features of target and interactive road users from time t0 - th + 1 to t0, where th is the history horizon.

4 Methodology

4.1 TrajAR Encoder

The TrajAR encoder comprises three key layers: (1) Motion trend layer that standardizes variable-length road users' motion positions into unified-length motion trends; (2) Motion MoE layer combining indicator and route experts to model road users' motion intentions; (3) Interaction-aware layer that uses dynamic interaction graphs to construct implicit interactions of the target road user, then incorporates these implicit interactions with explicit traffic regulations via Transformer-based situation awareness. Since the encoder processes historical trajectory features, we assume that the time range in this section is t0 - th < t ≤ t0 unless otherwise specified.

Motion Trend Layer The motion trend layer primarily captures the motion trends of road users from their historical trajectory features, providing a foundation for subsequent modeling of interactions between road users. We implement this layer as a single-layer Transformer encoder. Because road users enter and leave the frame at different times, their trajectory feature sequences can vary in length. As illustrated in Fig. 2(a), we temporally align the trajectory features of each road user and pad all sequences to the same length th. For each road user's trajectory feature sequence ⟨zν t ⟩, we process it through the motion trend layer. The output at step t, denoted as ˜ zν t , represents the road user's motion trend at time t.

MotionMoE The motion intention of a road user, defined as its planned future movement behavior, depends on multiple aspects of information, including its specific type attributes, motion trends, and signal states at a specific time. To extract motion intention in complex scenarios, we propose motion Mixture-of-Experts (MoE). This architecture enhances effectiveness by incorporating multi-aspect information through specialized experts, each handling distinct informational aspects. Compared with traditional MoE architectures like GShard [Lepikhin et al., 2020], Motion MoE employs a specialized structure: indicator experts for modeling motion trends, and route experts for modeling type attributes and signal states. The output of Motion MoE, denoted as zν t , is calculated as:

where pn,t = Sigmoid(˜ zν t cn) represents the trend-to-expert affinity, indicating the relevance to the n-th routing expert. Here, I a n and I sn are one-hot indicators that mark the positions of the current road user type and traffic signal state, respectively. We use 8 routed experts. Na and Nl denote the numbers of road user type indicator experts, signal state indicator experts, respectively. FFN(a) n (·), FFN(s) n (·) and FFN(r) n (·) represent the n-th indicator expert or routed expert, respectively; Kr specifies the number of activated routed experts; gn is the gating value for the n-th route expert; cn is the centroid vector of the n-th routed expert; and TopK(·, K) denotes the set comprising K highest scores among the affinity scores calculated for the motion trend ˜ zν t and all routed experts. The indicator experts handle discrete features such as road user types and traffic signal states, while route experts model continuous features, including motion trends.

Interaction-Aware Layer This layer models implicit interactions between the target road user and its interactive road users by integrating motion intention information. To capture these interactions, we employ GAT [Velickovic et al., 2017] layers to compute attention weights between the target road user and each interactive road user. This mechanism enables the network to selectively focus on more relevant interactive road users in complex traffic scenarios. Specifically, the attention weights are determined by:

where a˜α andW˜α are learnable parameters, and γ is the leaky ReLU activation function. Note that the edge weight βν,νT t is included as a feature in the computation of ˜αν,νT t . These edge weights are computed using a Gaussian kernel:

where dν,νT is the Euclidean distance between road users ν and νT . The parameter σe controls how much to weigh down edges to road users that are far away. This parameter is learned jointly with the rest of the model. The representation of implicit interactions hνT t is then computed using the attention weights as:

Finally, we process the sequence ⟨hνT t ⟩ with a single-layer Transformer encoder. At each time step, it takes hνT t as input and produces the corresponding output oνT t . The final output of the TrajAR encoder is obtained by concatenating the outputs over the history horizon, denoted as Ohis = [oνT t0-th+1 ∥oνT t0-th+2 ∥ · · · ∥oνT t0 ].

4.2 TrajARDecoder

Multi-scale Autoregressive Prediction To address the longterm trajectory prediction challenge discussed in Sec. 1, we take inspiration from visual autoregressive modeling and reconceptualize autoregressive trajectory prediction from "next-step prediction" to "next-scale prediction". In each autoregressive step, we predict a finer resolution scale of future motion positions, rather than the next step of motion position. We start by grouping the future motion positions ⟨τνT t0+1 , τνT t0+2 , . . . , τνT t0+tf ⟩ of the target road user into K multi-scale motion position resolutions (S1, S2, . . . , SK):

As shown in Fig. 2(b), each resolution Sk is progressively finer compared to its preceding resolution Sk-1, with the final scale SK aligning with the target prediction resolution. The autoregressive likelihood is defined as:

where each autoregressive step Tk = [S0∥ ˆ Sk-1] serves as the source sequence for ˆ Sk, and T1 = [Ohis∥S0] serves as the source sequence for ˆ S1 when k = 1. During the k-th autoregressive step, the whole sequence ˆ Sk is generated in parallel.

Cubic Interpolation As shown in Fig. 2, when k = 1, we use a fully-connected layer ϕ1 to map the autoregressive step T1 to its latent embeddings E1 = ⟨es⟩. When k > 1, we perform cubic interpolation to estimate a finer resolution scale ¯ Tk on top of the motion positions in Tk. More specifically, we apply cubic interpolation to insert a new motion position between every two consecutive positions in Tk, and lastly drop τνT t0 . The result is formulated as:

where ¯τνT t denotes an interpolated motion position, and ˆτνT t denotes a motion position in ˆ Sk-1. Then, we use a fullyconnected layer ϕk to map each motion position in ¯ Tk to its corresponding latent embedding, and obtain the sequence:

Compared to alternative interpolation methods such as linear interpolation, cubic interpolation fits a smooth third-degree polynomial between motion positions, ensuring the interpolated positions' continuity in position, velocity, and curvature.

TrajAR Transformer The final output layer of the TrajAR decoder is an ld-layer Transformer decoder. To ensure the information of finer resolution scales does not leak to coarser ones, we use a block-wise causal attention mask. This ensures that each ˆ Sk can only attend to the source sequences corresponding to resolution scales not finer than k, i.e., Tj where 0 < j ≤ k. To produce the prediction of motion positions at resolution scale k, we use the sequence of autoregressive steps ⟨T1, T2, ..., Tk⟩ as the memory sequence, and the step Tk as the target sequence of the Transformer decoder. The output sequence from the decoder is fed into a one-layer MLP (with BatchNorm and Sigmoid activation function) to calculate the prediction as:

Finally, the prediction of the finest resolution scale, denoted as ˆ SK, is the output prediction of the TrajAR decoder.

Loss Function During the training phase, trajectory predictions at all resolution scales are simultaneously compared with the ground truth using RMSE loss:

5 Experiments

We focus on urban intersections due to their high practical relevance and inherent complexity. These scenes involve diverse road users-including vehicles, bicycles, and pedestrians-interacting under traffic signals and right-of-way constraints. Intersections are also common locations for vehicleinfrastructure systems deployment and are associated with a high risk of traffic conflicts. While other settings, such as highways or campuses, are also of interest, they typically feature more uniform traffic compositions and simpler interaction patterns. Our goal is to address the most safety-critical and interaction-rich scenarios, leaving broader generalization to future work. To ensure the fairness of the experiments, we run each set of experiments 5 times and report the mean value.

Datasets In our experiments, we use four real-world datasets: SinD-Tianjin, SinD-Xian, InD-Bendplatz, and InDFrankenburg. The two SinD [Xu et al., 2022] datasets record signalized intersections in Tianjin and Xian, China, at 10 Hz. The two InD datasets 1 contain recorded trajectories of four types of road users from different intersections in Germany, with several hours recorded at 25 Hz. The input and target data are down-sampled by a factor of 3, resulting in an effective sampling interval of 0.12s. To evaluate the model's performance in predicting long-term trajectory for different types of road users, the prediction step is set to 32, matching the observation window length. We split it into training, validation, and test sets in a 6:2:2 ratio based on the recorded time.

1 http://www.inD-dataset.com

Baselines We compare our method with a set of baselines across three categories, including the most classical, the most representative, and the most recent state-of-the-art methods. First, we include classic physics-based models: CV (Constant Velocity) and CA (Constant Acceleration) assume simplified motion patterns, while Cubic generates a smooth trajectory via piecewise cubic polynomial interpolation. Second, we evaluate several learning-based methods specifically designed for vehicle trajectory prediction: C2F-TP [Wang et al., 2025b], WSiP [Wang et al., 2023], HLTP [Liao et al., 2024a], PiP [Song et al., 2020], STDAN [Chen et al., 2022], and BAT [Liao et al., 2024b]. Finally, we include multiple road users trajectory prediction baselines: MTP-GO [Westny et al., 2023], FJMP [Rowe et al., 2023], and KI-GAN [Wei et al., 2024].

Evaluation Metrics We use four types of basic metrics, the same as MTP-Go [Westny et al., 2023]: ADE (Average Displacement Error), FDE (Final Displacement Error), MR (Miss Rate), APDE (Average Path Displacement Error).

Implementation Settings For all datasets, we select the optimal parameter combinations based on validation dataset performance, including the Top value of MoE Kr and the number of TrajAR Transformer layers ld. We set the number of road user type experts Na and signal state experts Nl equal to the number of unique road user types and traffic signal states in each corresponding dataset. Specifically, we configure the parameter combinations {Kr, ld,Na,Nl} for SinDTianjin, SinD-Xian, InD-Bendplatz, and InD-Frankenburg as {2, 4, 7, 6}, {2, 6, 7, 4}, {4, 6, 4, -}, and {2, 8, 4, -}, respectively. Since the InD dataset contains no signal information, we omit the signal state experts when experimenting on this dataset to prevent interference. To ensure a fair comparison, no external contextual information is introduced during the generation phase. We set the number of closest road users of targent user NR = 15. The learning rate is set to 1e-4, and the model is pre-trained for 100 epochs on the training sets with an early-stopping patience of 5. All experiments are conducted on Intel Rapids-SP CPUs and NVIDIA A40 GPUs.

Trajectory Prediction Comparison Results in Tab. 1 and Tab. 2 show that TrajAR consistently outperforms all baselines on both datasets across all metrics, surpassing the best baseline by up to 32.0% in FDE and 29.8% in APDE. Compared with the second-best method, TrajAR achieves average improvements of 10.0%-29.0% over short- (step@8), mid- (step@16), and long-term (step@24/FDE) predictions. In contrast, traditional, physics-based, and prior deep learning methods suffer from error accumulation with increasing horizons. While graph networks and adversarial models outperform traditional methods, they still lag behind TrajAR with its dynamic perception encoding.These experimental results clearly demonstrate TrajAR's robustness and generalization capabilities across diverse road scenarios and long-term prediction tasks.

Ablation of Modules Our ablation study evaluates the contribution of each module by systematically removing individual components. Results shown in Tab. 3 that all modules are critical to TrajAR. Removing the Motion MoE significantly increases errors (ADE/FDE +118.1%/+105.8% on SinDTianjin), indicating its importance for capturing road user types and traffic light dynamics. Excluding the InteractionAware layer similarly degrades performance (ADE/FDE +115.6%/+103.1%), highlighting its role in modeling implicit multi-agent interactions. Replacing encoders with fully connected layers leads to substantial performance drops. Notably, removing the TrajAR decoder causes the largest ADE and FDE increase, validating the effectiveness of the next-scale autoregressive prediction strategy. Overall, the full model achieves the lowest errors, demonstrating the strong synergy of all components.

Probabilistic & Safety Performance We further evaluate probabilistic and safety performance by comparing TrajAR with KI-GAN on four datasets using minADE, Negative LogLikelihood NLL, and Collision Rate as shown in Tab. 4. Despite using a deterministic RMSE objective, TrajAR consistently outperforms KI-GAN across all metrics.

Statistical Analysis We run each method with 5 random seeds and report the mean ± standard deviation. Statistical significance is evaluated using a paired t-test comparing TrajAR with the strongest baseline KI-GAN on the primary metrics. This paired setting ensures that the observed performance differences are attributed to the modeling capability rather than random initialization or data variance. The results shown in Tab. 5 demonstrate that the improvements of TrajAR are consistent and statistically significant.

Per-type and Scenario-based Evaluation To address the diversity of road users and traffic scenarios, we extend our evaluation in two aspects. We first perform a per-category analysis on the SinD dataset, including vehicles, bicycles, and pedestrians. As shown in Tab. 6, TrajAR achieves consistently strong performance across all categories. We further evaluate performance under different traffic conditions by splitting the test set into Off-Peak and Peak periods. While all methods degrade in Peak traffic, TrajAR exhibits a substantially smaller performance drop than KI-GAN, demonstrating stronger robustness in dense and highly interactive scenarios.

Efficiency Analysis As shown in Fig. 3(a), TrajAR achieves an optimal accuracy-efficiency trade-off, delivering the highest prediction accuracy with under 100 ms latency. Its lightweight design ensures minimal computational cost, enabling stable, real-time industrial deployment on resource-constrained platforms with reduced overhead.

Expert Routing Analysis We analyze MoE expert effectiveness. Fig. 3(b) shows distinct route expert loads for different motion trends, confirming routing specialization. Fig. 3(c) demonstrates that ablating user types, signals, or shuffling expert routes degrades performance; shuffling causes the most severe drop, highlighting the route expert's necessity. Finally, Fig. 4 confirms optimal ADE is achieved using 8 experts with top-Kr routing.

Interpolation Analysis As shown in Tab. 7, cubic interpolation outperforms other methods by generating smooth, oscillation-free trajectories. Unlike linear interpolation (which degrades for fast agents) and Lagrange (prone to Runge effects), cubic interpolation offers the optimal stability-accuracy balance, enhancing long-horizon autoregressive prediction.

6 Conclusion

We propose TrajAR, a lightweight, multi-scale autoregressive framework that mitigates error accumulation for real-time multi-agent trajectory prediction in urban intersections. Future work will explore probabilistic multimodal prediction and broader urban generalization.

Ethical Statement

There are no ethical issues.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 62272033).

Contribution Statement

The authors Letian Gong and Yan Lin contributed equally to this work and share co-first authorship.