HLMTrans: A Sim-to-Real Transfer Framework for Spatial Crowdsourcing with Human-Guided Language Models

1 Introduction

Task assignment [Li et al., 2024; Li et al., 2025] is a critical problem aimed at maintaining the operation of Spatial Crowdsourcing (SC) platforms (e.g., Uber, Gigwalk, and Waze). Recently, Reinforcement Learning (RL) methods have demonstrated remarkable success in addressing task assignment challenges through trial and error in simulators [Li et al., 2023; Wang et al., 2023; Wu et al., 2024], which are valuable tools for training RL models (RLMs) by providing simplified and controlled environments compared to the complexities of real-world settings. However, due to discrepancies in system dynamics between training simulators and actual road conditions, a notable performance gap (i.e., sim-to-real gap [Da et al., 2023]) emerges when transferring simulator-trained RLMs to real-world environments.

∗Corresponding author.

The misalignment of travel time is a key factor contributing to the sim-to-real gap, as travel time calculations in simulated environments usually rely on simplified distance-time relationships or fixed traffic parameters, failing to capture the system dynamics of real-world road conditions [Da et al., 2024]. As shown in Fig. 1, we assume that the RLM's optimal assignment policy in the simulator is {(w2, τ1), (w3, τ2)}, yielding a total travel time of 3 (=1+2). However, in the real world, traffic congestion (i.e., red roads) leads to an increase in total travel time to 9 (=3+6), even though the travel distance remains unchanged. In this case, the optimal assignment policy becomes {(w1, τ1), (w2, τ2)}, with a total travel time of 5 (=2+3).

Existing studies on travel time estimation can be categorized into two types: data-driven and learning-based approaches. Data-driven methods typically divide a route into discrete segments and predict travel time for each using rulebased [Wang et al., 2014], statistical [Wang et al., 2023], or spatio-temporal pattern analysis techniques [Tang et al., 2018]. These methods overlook the dynamics and uncertainty of traffic systems, such as intersection and traffic light delays, reducing segment-level accuracy and causing cumulative errors and biases. Moreover, the impact of personalized preferences on travel time is often neglected when routes are treated as simple sequences of road segments, such as workers may prioritize time-efficient routes during peak hours but opt for distance-efficient routes during off-peak hours. In contrast, learning-based approaches can directly predict travel time for entire routes by leveraging deep learning models like CNNs [Wang et al., 2018], RNNs [Ye et al., 2022], and GNNs [Liu et al., 2024] to implicitly capture the interactive relationships between intersections and road segments. However, their accurate predictions usually require large amounts of real-world data covering the entire state distribution.

Therefore, mitigating the sim-to-real gap by aligning travel time still faces the following challenges: 1) the real-world data with system dynamics is both costly and sparse, making it challenging to accurately estimate travel times in realworld settings with limited data; 2) the workers' preferences for travel routes vary across different contexts, and these preference significantly impact travel time estimation, how to effectively capture these preferences is another challenge.

To tackle the above challenges, we propose HLMTrans, a Sim-to-Real Transfer framework with Human-Guided Language Models, aimed at bridging the performance gap between simulator-trained RLMs and real-world deployment caused by travel time misalignment. In this framework, we first propose a Large Language Models (LLMs)-based method for operationalizing Grounded Action Transformation (GAT) [Hanna and Stone, 2017], which integrates prompt generation and dynamics modeling modules to capture changes in system dynamics during sim-to-real transfer. By leveraging LLMs with prompts and zero-shot chainof-thought reasoning, we can accurately estimate travel time with limited data and enhance RLMs' understanding of realworld dynamics. Furthermore, we design a human preference learning mechanism that fine-tunes LLMs using human feedback, enabling them to align with human preferences and improve the relevance and quality of their generations. Our main contributions can be summarized as follows:

• We investigate the RLMs-based Task Assignment (RTA) problem in spatial crowdsourcing, focusing on bridging the performance gap between simulator-trained RLMs and real-world settings due to travel time misalignment. To this end, we propose HLMTrans, the first sim-to-real transfer framework that leverages human-guided LLMs to estimate travel times.
• We present a Monte Carlo Graph Search-based method to identify minimal congestion subgraphs affecting worker-task travel times, serving as contextual prompts to accelerate LLMs' inference. Additionally, we introduce a human preference learning mechanism to finetune LLMs with human feedback, enhancing its alignment with human preferences and generation quality.
• We extensively evaluate HLMTrans against three widely used benchmarks on two real-world datasets. The results demonstrate that HLMTrans outperforms state-ofthe-art methods, validating the effectiveness of HLMTrans.

2 RelatedWork

Sim-to-real Transfer. The sim-to-real transfermethods can generally be divided into three main categories: 1) domain randomization [Wu et al., 2023], aims to train policies capable of adapting to diverse environmental variations. This approach leverages simulated data and is particularly beneficial when handling uncertain or evolving target domains; 2) domain adaptation [Bousmalis et al., 2018], aims to achieve domain distribution shifts by aligning features between the source and target domains. Many domain adaptation methods emphasize bridging perception gaps in robotics [Fang et al., 2018; James et al., 2019]. However, in travel time estimation, the primary domain gap lies in dynamics rather than perception, since it typically represents observations in vectorized forms, such as road-level worker number and average speed; 3) grounding methods, enhance simulator accuracy by correcting biases relative to the real world. Recently, Grounded Action Transformation (GAT) has shown significant promise in sim-to-real transfer of traffic signal control [Da et al., 2024; Da et al., 2023]. Different from system identification techniques [Cutler et al., 2014; Cully et al., 2015] that estimate precise physical parameters, GAT [Hanna and Stone, 2017; Desai et al., 2020] adjusts the simulator's dynamics to align with real-world actions without requiring a parameterized simulator for modification. Building on GAT, our HLMTrans utilizes human-guided LLMs to estimate travel time and inputs it as domain knowledge into the forward model of GAT, enhancing the action transformation of RLMs.

Large Language Models with Human Guidance. Large Language Models (LLMs) have demonstrated exceptional linguistic capabilities; however, their alignment with human intentions remains challenging due to pre-training on large and noisy datasets. Human guidance plays a pivotal role in refining LLMs to generate coherent, ethical, and humanaligned outputs [Vafa et al., 2024; Jagadish et al., 2024]. The traditional approach for fine-tuning LLMs based on human preferences involves learning a reward signal using the Bradley-Terry model [Bradley and Terry, 1952], followed by applying RL to optimize against the learned reward signal [Griffith et al., 2013; Christiano et al., 2017]. Highquality human feedback datasets, including VisAlign [Lee et al., 2023] and PKU-SafeRLHF [Ji et al., 2023], have further enhanced LLMs' ability to interpret and respond to diverse instructions. However, inconsistencies among annotators and misalignments with human intentions can compromise the quality of feedback, posing challenges to effective model fine-tuning. Recent advancements, such as Direct Preference Optimization (DPO)[Lee et al., 2023], provide a model-free alternative that eliminates the need for explicit reward learning. Inspired by DPO, we propose a human preference learning mechanism that enables fine-tuning of LLMs using human feedback without an explicit reward model.

3 Preliminaries

3.1 Concepts of RTA and RL Solutions

On the SC platform, a road network, is denoted as a weighted graph G =< V,E,U >, where vi ∈ V is an intersection, eij ∈ E is a road that connects vi and vj, and uij ∈ U is the weight of eij, determined by factors such as road length and road type. Users can dynamically publish spatial tasks at any time, each of which can be denoted as τ =< lτ , tτ , dτ , ρτ >, where lτ ∈ V is the task's location, tτ is the publish time, dτ is the task's deadline, and ρτ is the payoff for serving the task. Once tasks are published, the platform assigns mobile workers to physically travel to the task locations and complete them. Here, a worker can be denoted as a triplet w =< lw, cw, sw >, where lw ∈ V is the worker's location, cw is the worker's status (i.e., idle or busy), and sw is the worker's service range. The RTA problem can be formulated as: Given a set of mobile workers W and a stream of spatial tasks Γ, the problem aims to apply RLMs-based approaches to find an assignment plan M = {(w, τ)|w ∈ W, τ ∈ Γ} that maximizes the platform's total revenue EM:

where (w, τ) is a valid worker-task pair that satisfies the spatiotemporal constraints (e.g., reach the specified location before the deadline), and R(w, τ) = ρτ -κ · Twτ is the revenue function, in which Twτ is the travel time of w finishing τ , and κ is the travel cost per unit time.

The RTA problem is typically modeled as a Markov Decision Process (MDP), defined as M =< S,A,R, P, γ >, where S represents the state space, A represents the action space, rt ∈ R is the reward obtained by executing action at ∈ A in state st ∈ S at time slice t, P is the probability of transitioning from state st to state st+1 after executing at, and γ ∈ [0, 1] represents the discount factor describing the importance of future rewards. An RL approach solves this problem by learning an optimal policy π that maximizes the the long-term expectation of discounted accumulation reward E(s,a)∼π[∑Tt=0 γ T-t r(s, a)], where r(s, a) is the immediate reward obtained by agent after executing action a in state s.

3.2 Grounded Action Transformation

The Grounded Action Transformation (GAT) is a framework proposed to address the challenges of transferring policies learned in simulated environments to real-world applications [Hanna and Stone, 2017]. This framework focuses on bridging the discrepancies between simulated dynamics and real-world dynamics. Within the GAT framework, the simulated environment Esim is modeled as an MDP, where the transition dynamics, parameterized as Pϕ(·|s, a), is imperfect but can be adjustable. By leveraging real-world trajectories, GAT iteratively refines the simulated dynamics to better reflect real-world behavior. Given a real-world trajectory dataset Dreal = {o1, o2, . . . , oI}, where each trajectory oi =< si0 , ai0 , si1 , ai1 , . . . , siT-1 , aiT-1 , siT > represents a sequence of states and actions collected by executing a policy π within the real-world environment Ereal. The goal of GAT is to optimize ϕ∗ such that the differences between the transition dynamics of the simulated environment and the realworld environment is minimized:

where P ∗ (sit+1 |sit , ait ) is the transition dynamics of Ereal, Pϕ(sit+1 |sit , ait ) is the transition dynamics of Esim, and d(·) quantifies the discrepancy between P ∗ and Pϕ.

Given the current state st and the action at predicted by the policy π, the grounded action ˆat, executed in Esim, adjusts the resulting next state st+1 in Esim to approximate the predicted next state ˆ st+1 in Ereal. This process ensures that the simulated dynamics Pϕ(st+1|st, ˆat) align closely with the real-world dynamics P ∗ (ˆ st+1|st, at). Hence, the policy π, trained in Esim using Pϕ, achieves dynamics more similar to P ∗ , reducing the performance gap when transferred to Ereal.

4 Methodology

4.1 Overview

The overview of HLMTrans, as depicted in Fig. 2, consists of three key components: (a) RLMs Decision for Task Assignment, which employs the RLMs to dynamically allocate tasks to workers based on real-time observed states, with the objective of maximizing the platform's total revenue, (b) LLMs-based Sim-to-Real Transfer, which utilizes the LLMs to bridge the sim-to-real gap by adapting simulated dynamics to align with real-world dynamics, ensuring more reliable policies, and (c) Preference Learning from Human Feedback, which enables the LLMs to iteratively learn human preferences through interaction, using this knowledge to fine-tune their generates and enhance the alignment between worker travel times and real-world settings, further improving the RLMs' performance.

4.2 RLMs Decision for Task Assignment

In this paper, we formulate the RTA problem as an MDP, where the SC platform is treated as an agent that learns an optimize policy through interactions with the environment. Specifically, the agent observes real-time states, and makes decisions for worker-task matching in a trial-and-error manner. Next, we detail the modeling of state space S, action space A, and reward function R, as these are the critical components in reinforcement learning.

State Space S. At any time slice t, the state st ∈ S can be represented as a six-tuple: st =< DΓ , LΓ , LW, lτ , lw, t >, where DΓ is the deadline distribution of unassigned tasks Γ , LΓ is the location distribution of unassigned tasks Γ, LW is the location distribution of idle workers W, lτ is the location of task τ ∈ Γ , and lw is the location of worker w ∈ W.

Action Space A. During time interval ∆t, the SC platform accumulates a batch of unassigned tasks Γ and a set of idle workers W. For each task τ ∈ Γ, the agent observes the current state st and selects an action at based on the learned policy π. For each worker w ∈ W, the action at is defined as: at = {0, 1}, where 1 indicates assigning the worker w to τ , and 0 indicates not assigning.

Reward Function R. After the agent executes action at in state st, it obtains a reward rt, which is designed to reflect the difference between task payoff and the travel cost incurred by assigned worker. The reward can be calculated as: r = R(w, τ ) = ρτ - κ · Twτ .

Following the previous works [Li et al., 2023; Wu et al., 2024], we use Deep Q-Network (DQN) algorithm to optimize the RL policy of RTA problem.

4.3 Sim-to-Real Transfer with LLMs

Congestion Influence Subgraph. Identifying the Congestion Influence Subgraph (CIS), which represents the minimal traffic subgraph influencing the travel time from a worker's location to a task's location, is a challenging issue due to the complexity and dynamics of real-world road network. To address this issue, we propose a Monte Carlo Graph Searchbased Congestion Influence Subgraph Identify (MCGS-CI) algorithm for CIS identification. Compared with traditional Monte Carlo Tree Search methods [Shi et al., 2021], which rely on a tree structure for sequential decision-making, and face limitations in such scenarios due to redundant node expansions and insufficient information sharing between nodes, our approach explores a graph structure rather than a tree, enabling effective information sharing across nodes and reducing the number of required simulations [Xie et al., 2024]. This design improves computational efficiency while maintaining high accuracy in identifying the CIS.

Our MCGS-CI algorithm starts with an initial graph G =< V, E, U >, where V represents all traffic intersections and E denotes road segments with weights U. The algorithm dynamically builds and updates the graph structure during the search process, using shared information from previously explored nodes to refine subsequent simulations. Specifically, it balances exploration and exploitation by leveraging the Upper Confidence Bound (UCB) mechanism to guide the selection of the most promising paths within the graph. The key steps of MCGS-CI algorithm are described as follow:

Initialization: The graph G is initialized with the worker's location lw as the starting node and the task's location lτ as the goal node. The weight uij ∈ U of edge eij ∈ E in G is initialized to:

where ℓij is the length of road segment eij, and νij denotes the historical average travel speed on eij. The term qij indicates the historical average traffic volume on eij , while qmax is the maximum capacity of eij . η ∈ [0, 1] is a hyper-parameter that balances the travel time and congestion level.

1) Selection: Starting from lw, the algorithm recursively selects the neighboring nodes with weight not less than the threshold σ (i.e., uij ≥ σ) based on the UCB formula Fucb, and the score of child node v can be calculated as:

where ¯ Sv is the average score of child node v, n is the visit count of the v's parent node, nv is the visit count of child node v, and α is a constant to balance exploration and exploitation.

2) Expansion: When the algorithm encounters an unexpanded node, it incorporates the neighboring nodes into the graph G.

3) Simulation: During the simulation phase, the algorithm starts from the current node and performs simulation actions to evaluate the feasibility of adding expanded nodes to the graph G. It first checks whether the expanded node is valid by determining if there exists a path through this node that can reach the goal node lτ . If such a path exists, the algorithm quantifies the node's contribution to congestion by assigning a weight, and its corresponding edge and node are incorporated into the graph G.

4) Backpropagation: After the simulation phase, the results are propagated back through the explored graph, updating the weights and visit counts of all nodes and edges along the search path. Specifically, the weight of each edge on the path is incremented by 1 to reflect its contribution to the current simulation, and the visit count of each node is also increased by 1. This process refines future decision-making by reinforcing the importance of nodes that contribute to successful simulations, enabling the algorithm to prioritize promising paths in subsequent explorations.

Terminal Condition: The search process continues until a predefined number of iterations or lτ is explored. The algorithm then outputs the resulting subgraph Gcwτ ⊆ G, consisting of the nodes and edges with the higher cumulative impact on travel time, as the identified CIS.

Prompt-based LLMs. Large Language Models (LLMs) have demonstrated superior zero-shot and few-shot reasoning abilities by adapting to unseen tasks through in-context learning [Alayrac et al., 2022]. Building on these capabilities, we propose an LLMs-based approach to estimate travel time from worker's location to task's location in a real-world road conditions. The LLMs (e.g., LLaMA) utilize the CIS, denoted as Gcwτ = ⟨V c ,Ec ,U c ⟩, and structured contextual knowledge to achieve accurate travel time estimations. The LLMs' prompt template is defined as:

where ⟨Question⟩ is the task description that LLMs require reasoning, such as estimating the travel time from worker's location to task's location, ⟨[Context]⟩ is structured knowledge about worker-task pairs, traffic conditions, and domain info (e.g., weather, time of day, day of week), and ⟨Response⟩ specifies the generate format of LLMs. For each worker-task pair (w, τ) ∈ M, the context includes detailed information as follows:

The LLMs utilize the prompt structured in Eq. (5) to infer the real-world travel time Twτ :

where Dt encodes contextual knowledge such as weather conditions, traffic density, and time of day. Then, we fuse Twτ and Dt with the state feature:

where Concate denotes the concatenation operation, and zt represents the temporary feature embedding.

Next, we use a forward model fϕ+ to predict the grounded state ˆ st+1 in Ereal as follow:

where fϕ+ is implemented using a neural network and ϕ+ can be optimized by minimizing the Mean Squared Error (MSE) loss:

where sit , ait , sit+1 are the ground trajectories sampled from real-world environment Ereal.

Ultimately, we can predict the grounded action ˆat that can enable the transition from state st to state ˆ st+1 in Esim by an inverse model hϕ-:

Compared to the forward model fϕ+, which incorporates detailed dynamics knowledge for precise state transitions, hϕ- (ˆ st+1, st) can operate with lower computational complexity by relying solely on the observed states within the simulators. Thus, we can implement the inverse model using a neural network, and optimize it by minimizing the Categorical Cross-Entropy (CE) loss:

where sit , ait , sit+1 are the predicted trajectories sampled from simulated environment Esim.

4.4 Preference Learning fromHuman Feedback

To enable LLMs to estimate travel times in alignment with human preferences, i.e., dynamically adjusting priorities such as time and distance, we propose a Human Preference Learning Mechanism (HPLM). This mechanism introduces a preference vector p =< p1, p2, . . . , pn >, where each p ∈ [0, 1] represents the importance of a specific factor (e.g., time, distance, and road type) derived from human feedback. Assuming a dominant preference is specified, such as pk (e.g., prioritizing distance), pk can be set to 1, while other components of p are set to 0. Accordingly, the input prompt is refined to embed p, ensuring that the LLMs' generations align with human preferences:

Given a preference dataset Dp = {(xi, Twiτi , ˜ Twiτi )}Mi=1 , where xi is the LLMs' input prompt, Twiτi denotes the dispreferred output (abbreviated as T ), and ˜ Twiτi represents the preferred output (abbreviated as ˜ T ), the optimal LLM policy π∗ under the Bradley-Terry model [Rafailov et al., 2023] satisfies the following preference probability:

where δ denotes the logistic function, and ω is a scaling parameter that represents the divergence from the basic LLM policy πl, which does not incorporate human preference adaptation.

To fine-tune the LLMs' preference policy πp, we define a negative log-likelihood loss function based on the preference probability p∗ :

where r∗ l (·) is a reward model that scores LLMs' outputs to align with human preferences. This loss function guides the LLMs to assign higher probabilities to preferred responses while minimizing those for dispreferred ones. By iteratively updating the policy πp through HPLM, the LLMs can adapt to nuanced preference shifts influenced by contextual factors such as day-of-week, time-of-day, or weather conditions, thereby generating more accurate and context-aware travel time estimates.

5 Experiments

5.1 Experimental Settings

In this section, we conduct extensive evaluations of HLMTrans on two real-world datasets, comparing it with three state-of-the-art methods to demonstrate its effectiveness.

Datasets. The Chengdu (CD) and Xian (XA) datasets are selected to evaluate our proposed approaches. Each dataset comprises two main components: road network data and order data. The road network data is extracted from OpenStreetMap 1 , representing the intricate road networks of Chengdu, with 36,630 nodes and 50,786 edges, and Xian, with 7,215 nodes and 9,643 edges. The order data, which forms the basis for task generation, is sourced from taxi orders collected by Didi Chuxing 2 . The Chengdu dataset contains 546,671 orders recorded from November 1 to 8, 2016, while the Xian dataset includes 102,674 orders recorded from October 1 to 8, 2016. Each order is a six-tuple, including order index, origin, destination, departure time, arrival time, and travel trajectory. In the experiments, we use the order's travel trajectory to calculate its real-world travel time, and the road network distance to calculate its simulated travel time. The experimental parameters are summarized in Table 1, where bold values indicate default parameter values.

Compared Methods. We compare the performance of HLMTrans with three approaches: dTrans [Wu et al., 2024] (a direct transfer method which deploys the RLM trained in Esim to make decisions in Ereal without adaptation), MulTTTE [Liao et al., 2024] (a multi-faceted route representation learning model that captures spatial, attribute, and contextual information through sequential learning and transformer encoding, enabling more accurate travel time estimation), PromptGAT [Da et al., 2024] (a prompt-based grounded action transformation model which leverages the inference abilities of LLMs to profile system dynamics and guide policy actions for mitigating the sim-to-real performance gap).

Metrics. We compare the performance of RL models trained in a simulation environment when transferred to real-world settings using commonly used task assignment metrics [Xia et al., 2019; Li et al., 2023]: average total revenue (ATR), average task completion ratio (ACR), and average elapsed time (AET). Additionally, we evaluate the travel time estimation performance of large language models using commonly used spatiotemporal prediction metrics [Wu et al., 2024; Liao et al., 2024]: MAE and RMSE.

Environment. All machine learning methods are implemented with PyTorch 2.2 and Python 3.12, and trained with Intel i9-13900K@3.0GHz CPU, GTX3090 GPU and 32GB RAM. The platform ran on Ubuntu 18.04 LTS.

1 https://www.openstreetmap.org

2 https://www.didiglobal.com

5.2 Analysis of Experimental Results

Effect of Language Models. The experimental results, as shown in Table 2, compare the travel time prediction errors of four large language models and the learning-based method MulT-TTE. The prompt for the large language models is set to:"Please estimate the travel time in minute from nodes worker location to task location based on the following traffic perceptive information: on day-of-week at time-of-day, weather is weather, and the road network status (data format:[start node, end node, road type]) is CIS. Please only answer by replacing {value} in the format below: [travel time: {value}].". It can be observed that the error metrics (MAE and RMSE) of the large language models are significantly lower than those of MulT-TTE, attributed to their robust zero-shot chain-of-thought reasoning capabilities. Among the models, the LLaMA series outperforms ChatGLM and Qwen in travel time estimation. Furthermore, larger parameter models in the LLaMA series (e.g., LLaMA3-8B) demonstrate superior performance compared to smaller parameter models (e.g., LLaMA2-7B).

Effect of |W|. To evaluate the effect of worker number |W| on the sim-to-real gap, we varied |W| from 50 to 800 on both the CD and XA datasets. The experimental results, as shown in Fig. 3, indicate that as |W| increases, all four methods exhibit a rapid initial improvement in total revenue and task completion ratio, followed by a gradual plateau. This trend arises because the increase in worker availability allows more tasks to be completed, but once the worker number surpasses a certain threshold, the limited number of tasks constrains further revenue growth. HLMTrans achieves the best performance on both datasets, with total revenue improvements of approximately 21.9% and 23.7% over dTrans on the CD and XA datasets, respectively. However, HLMTrans incurs slightly higher time costs compared to the other methods, primarily due to its ability to allocate workers more effectively for rapid task completion, which necessitates a higher number of model decisions.

Effect of |Γ|. Fig. 4 presents the experimental results with task number |Γ| varying from 5K to 30K. As |Γ| increases, total revenue and time costs show an upward trend, while the completion ratio declines. This is because an increase in task number leads to more tasks being completed, thereby increasing both revenue and time costs. However, due to the limited number of workers, a larger proportion of tasks expire without being assigned to available workers. Among the methods, HLMTrans consistently achieves the optimal performance, followed by PromptGAT, MulT-TTE, and dTrans. Effect of dτ . The experimental results in Fig. 5 illustrate the effect of varying task deadline dτ from 1 to 10 on the CD and XA datasets. As the deadline increases, ATR, ACR, and AET all exhibit an upward trend. This is because an extended deadline allows more tasks to be completed, while simultaneously increasing the model's number of decisions. HLMTrans achieves significant improvements in total revenue compared to dTrans, with maximum gains of up to 25.4% on the CD dataset and 39.7% on the XA dataset.

Figure 5: Effect of the deadline of tasks dτ .

6 Conclusion

In this paper, we propose HLMTrans, a human-guided LLMsbased sim-to-real transfer framework, designed to mitigate the performance gap when transferring RLMs trained in simulation environments to real-world task assignment scenarios. By leveraging the zero-shot reasoning capabilities of pre-trained LLMs and fine-tuning with human preference feedback, HLMTrans can improve the predictive accuracy of GAT's forward model, and effectively align with the system dynamics of real-world environments. Extensive experiments demonstrate that RLMs trained with the HLMTrans framework in simulation environments, benefiting from more accurate predictions of travel times, achieve superior performance in real-world settings compared to SOTA methods.

Acknowledgments

This work is supported by the following grants: NSFC Grants 62372416, 61972362, 62036010, 62325602, 62302460 and 62402453; HNSF Grant 242300421215; CPSF Grant 2022TQ0297.