Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling

1 Introduction

The Dynamic Flexible Job Shop Scheduling Problem (DFJSP) is a core online decision problem in modern manufacturing, where each ready operation must be assigned to a feasible heterogeneous machine under routing flexibility and precedence constraints [Xu et al., 2025]. When machine failures, urgent job arrivals, or processing-time variations change the shop-floor state, delayed dispatching decisions can block available machines and propagate idle time through subsequent operations.

∗Corresponding author.

Data-driven dispatching policy learning is achievable with Deep Reinforcement Learning (DRL), but interpreting its decisions poses significant challenges and these decisions may be vulnerable to variations in training details [Zhang et al., 2020]. Priority Dispatching Rules (PDRs) are easy to execute and inspect, yet their rigid structures often fail to adapt when the bottleneck shifts [Holthaus and Rajendran, 2000]. By generating executable rules with LLMs, Code as Policy (CaP) improves the transparency of scheduling decisions [Liang et al., 2023]. One remaining obstacle is response latency: making LLM calls during the control loop operation is too slow to satisfy the real-time demands of scheduling systems.

The fundamental conflict lies in the temporal granularity between reasoning and execution. Advanced reasoning models such as GPT-4 require several seconds to process context and generate code [Hurst et al., 2024]. In contrast, dynamic scheduling decisions in a high-speed production line must be made within milliseconds to avoid blocking machine operations [Singh et al., 2026; Ouelhadj and Petrovic, 2009]. Simple integration strategies that pause the production line to await external reasoning result in unacceptable throughput losses. This leads to a frequency mismatch where the slow cognitive cycle of the LLM cannot synchronize with the fast physical cycle of the manufacturing floor [Karami et al., 2025]. Consequently, current approaches are forced to compromise by either relying on pre-generated static rules that lack adaptability or employing smaller and less capable models that sacrifice reasoning quality for speed [Shah et al., 2026].

Furthermore, existing hybrid frameworks typically separate learning from execution in a manner that precludes continuous online adaptation. Most methodologies rely on an offline training phase, where scheduling rules are derived from historical data [Priore et al., 2014]. Once deployed, these rules remain static and cannot respond to unseen disturbances such as sudden machine breakdowns or drastic shifts in order priority. Although some adaptive systems attempt to periodically retrain policies, they often require synchronous interactions that disrupt the ongoing scheduling process. A key limitation in current research is the lack of mechanisms that allow for the safe and asynchronous evolution of control logic during system operations. An ideal scheduling system should enable the control logic to improve continuously without impeding the real-time responsiveness of physical machines.

While inspired by the asynchronous dual-stream concept [Chen et al., 2026], directly applying existing embodied agents to industrial scheduling is hindered by the requirement for interpretability and strict safety guarantees. Rather than treating LLM-based code generation, simulation validation, and iterative refinement as new ingredients, we focus on how these mechanisms can be safely integrated into a running DFJSP control loop. We introduce RACE-Sched 1 , a framework where the Reactive Stream runs low-latency symbolic heuristics, while the Deliberative Stream leverages an LLM to analyze summary statistics from a sliding window of recent decisions, generate candidate Python heuristics, and test them in a sandbox that replays representative instances under the same constraints. Only candidates that meet predefined acceptance criteria are promoted. The Reactive Stream does not lie on the critical path: it never waits for deliberation, and updates are executed through an atomic pointer swap of the active rule set.

We also maintain a rule repository that indexes validated heuristics together with lightweight instance-level metadata, such as numbers of jobs and machines. The Deliberative Stream retrieves the most similar rules to warm-start candidate generation and sandbox evaluation, which helps shorten prompts and enhance robustness across problem scales.

Our contributions are as follows.

• An asynchronous symbolic evolution framework adapted for industrial constraints, which decouples realtime dispatching from LLM-driven heuristic synthesis via a safe "code-as-policy" mechanism, ensuring the control loop remains responsive.
• A sandbox-based validation and safe deployment mechanism, where candidate heuristics are evaluated offline through constrained replay, and promoted only if they meet predefined acceptance criteria, followed by an atomic pointer swap of the active rule set.
• A rule repository that indexes validated heuristics with lightweight instance-level metadata, enabling warmstart retrieval and improving transferability across different benchmarks and problem scales.
• An extensive evaluation on GEN-Bench, MK-Bench, and JMS-Bench, including a machine-failure stress test, showing that our approach achieves higher solution quality and faster adaptation than competitive DRL baselines and direct LLM control.

1 To facilitate reproducibility, our code is available at https:// github.com/cls1277/RACE-Sched

2 RelatedWork

Heuristics and Evolutionary Scheduling. Priority Dispatching Rules remain the industrial standard for dynamic scheduling due to their minimal computational cost of constant time complexity and ease of interpretation [Holthaus and Rajendran, 2000]. However, fixed rules such as Shortest Processing Time and Most Work Remaining are inherently myopic. They rely on local buffer status and often fail to maintain performance when system bottlenecks shift due to stochastic disturbances. To automate the design of dispatching policies, Genetic Programming (GP) has been widely applied to evolve symbolic priority functions that combine production attributes into complex rules [Mei et al., 2016]. While advances such as surrogate-assisted GP [Mei et al., 2016] and multitask evolution [Zhang et al., 2023] have improved convergence speed, the evolutionary process remains computationally expensive and offline. This limitation prevents GP from achieving the rapid online adaptation required to handle sudden machine failures or urgent order insertions in real-time environments.

DRL for Dynamic Scheduling. To capture complex system dynamics, recent research has shifted toward DRL by modeling scheduling as a Markov Decision Process [Xu et al., 2025]. State representation has evolved from simple feature vectors to Graph Neural Networks that encode the nonEuclidean topology of job shops [Zhang et al., 2025]. The Dual Attention Network [Wang et al., 2024b] established a benchmark by jointly attending to operation precedence and machine contention. Specialized architectures have also been developed for specific disruptions such as IDDQN [Wu et al., 2025] for robustness against machine breakdowns and hierarchical frameworks such as HMPSAC [Ding et al., 2025] for multi-objective trade-offs. Despite these gains, DRL faces critical deployment barriers. The resulting black-box neural policies lack the interpretability required for safety-critical manufacturing [Li et al., 2026] and often exhibit poor generalization when transferring between different problem scales.

LLM Reasoning and Code-as-Policy. LLMs offer a promising avenue to address the interpretability and reasoning gaps in DRL. While techniques such as Chain-ofThought [Wei et al., 2022] and Tree of Thoughts [Yao et al., 2023] enhance logical planning, the high latency of inference creates a temporal mismatch with the millisecondlevel response requirements of industrial control. To address this discrepancy, the CaP paradigm proposes generating executable programs rather than direct text actions [Liang et al., 2023]. This approach generates fast and interpretable code as validated by open-ended agents such as Voyager [Wang et al., 2024a]. In the scheduling domain, ReflecSched [Cao and Yuan, 2025] recently utilized LLMs to generate hierarchical reflections for guidance. Prior LLM reflection and CaP scheduling methods mainly focus on synthesizing or refining executable policies, while hybrid offline-online heuristic evolution and safe update schemes primarily address policy improvement and deployment risk as separate stages. RACESched instead focuses on the control-integration problem: it confines slow LLM deliberation, sandbox validation, and candidate policy improvement to the background, while the online scheduler executes only the currently validated symbolic rule and receives accepted updates through atomic hotswap.

3 ProblemFormulation and Preliminaries

We consider the DFJSP, which entails the assignment of n jobs J = {J1, . . . , Jn} to a set of m heterogeneous machines M = {M1, . . . ,Mm}. Each job Ji ∈ J comprises a strictly ordered sequence of ni operations Oi = {oi,1, oi,2, . . . , oi,ni }. Routing flexibility permits each operation oi,j to be executed on any machine within an eligible subset Mi,j ⊆ M, where pi,j,k denotes the deterministic processing time on machine Mk ∈Mi,j [Cao et al., 2023].

The system state at time τ comprises the progress of active jobs, machine availability, and the backlog of pending operations. Temporal evolution is driven by asynchronous stochastic disturbances that induce discrete state transitions, including non-deterministic job arrivals and machine failurerecovery cycles. Under a preempt-resume scheduling policy, interrupted operations remain suspended and are resumed from the point of interruption upon the restoration of machine capacity. The objective is to minimize the expected makespan Cmax, where Ci denotes the completion time of job Ji and Cmax = maxJi∈J Ci, subject to real-time decision constraints in a dynamic manufacturing environment.

We formulate this task as a sequential decision-making process where a policy π maps system states to operationmachine assignments at each decision epoch t, while satisfying precedence and availability constraints. In this stochastic setting, performance optimization requires strategic reasoning to address complex operational tradeoffs, such as the proactive allocation of machine capacity to alleviate anticipated bottlenecks.

4 Methodology

A detailed workflow diagram of RACE-Sched is provided in Figure 1. The Reactive Stream runs a symbolic priority rule inside the control loop. A rule repository is maintained to store validated heuristics together with lightweight instancelevel metadata, facilitating retrieval-based warm starts. The Deliberative Stream improves the active rule asynchronously using compact summaries of recent states and example actions, together with constrained code generation and sandbox evaluation procedures.

4.1 Asynchronous Dual StreamArchitecture

We model DFJSP as a Semi-Markov Decision Process with state st and action at at each decision epoch t [Chang et al., 2022]. The Reactive Stream maps st to a dispatching action using a symbolic rule, while the Deliberative Stream performs rule updates using LLM-based reasoning, which operates on a slower timescale.

Reactive Stream for Execution

The Reactive Stream runs inside the scheduling loop, with the objective of confining the decision latency within the millisecond range [Ferreira et al., 2022]. It executes a symbolic priority function fϕ implemented as a compiled Python rule derived from the current code representation. At decision epoch t, it selects

The interpreter is constrained to use only observable features in st and does not permit side effects, thereby the rule remains deterministic and operates with millisecond-scale execution time.

Deliberative Reasoning Stream for Policy Evolution

The Deliberative Stream updates the symbolic rule off the critical path. It takes a compact summary of recent states and example actions together with the current rule ϕold and retrieved knowledge K and proposes a new rule [Yang et al., 2024; Gao et al., 2025]

where ξ denotes the summarized observation window. The Reactive Stream keeps running ϕold while the proposal is generated and tested in the sandbox.

Trigger Update Protocol and Atomic Transition

Deliberation is triggered either every K decision steps or when a monitored metric drops relative to its sliding-window best of length L. Let mt denote the metric and m∗t = maxi∈[t-L+1,t] mi. Define the periodic trigger indicator as

and the performance-based trigger indicator as

When Iperiodic(t) ∨ Iperf(t) holds, the Deliberative Stream proposes ϕnew and atomically swaps it in only if sandbox screening accepts it.

4.2 The Agentic Loop: A Safety-Constrained Evolutionary Cycle

The Strategic Planner produces a compact directive based on the current system profile. The Syntactic Enforcer converts this directive into an executable priority dispatching rule, subject to syntactic safety and complexity constraints. The Simulation Validator evaluates candidate rules in a sandbox, allowing an update only if it meets predefined feasibility and improvement criteria. Concurrently, the Semantic Reflector records concise diagnostics to justify the acceptance or rejection decision for human inspection.

Strategic Planner for Bottleneck-Aware Reasoning

The Strategic Planner generates a compact profile by summarizing the current state and formulates a local strategy, such as shifting priority toward the current bottleneck or favoring short remaining processing time on a congested machine. We formalize this mechanism as

where Mobj is the objective and Splan is a short directive consumed by the Syntactic Enforcer.

Syntactic Enforcer for Executable Logic Generation

The Syntactic Enforcer converts the strategic plan Splan into executable logic in the form of PDRs. It produces symbolic Python functions whose source code can be inspected by domain experts.

Rule generation relies on constrained prompting, which encourages the adoption of a standardized set of shop floor attributes and per operation features. The Coder outputs a priority function fϕ that scores each feasible operation and induces a dispatching decision.

The Syntactic Enforcer enforces basic safety and executability by compiling candidate code in a restricted execution context with a limited set of built-in functions and restricted import permissions. If the generated code exceeds a fixed length threshold, it is truncated before compilation, and non-compilable candidates are discarded. The module further extracts lightweight complexity statistics for bookkeeping and subsequent downstream selection.

Simulation Validator for Sandbox Screening

The Simulation Validator screens a candidate policy ϕcand against the incumbent ϕold in a sandbox before deployment [Liu et al., 2023]. On an evaluation pool Dval, it runs rollouts for both policies under the same decision constraints and disturbance process, yielding per episode objective values {yold i }ni=1 and {ycand i }ni=1 . Define the empirical estimate

For a minimization objective, the relative improvement is

with

Let µold, µcand and s2old , s2cand be the sample means and variances of the two rollout sets. The validator computes

and

A candidate is accepted only if sandbox execution is feasible and

where ϵrel, dmin, and Tα are fixed thresholds. The validator returns Ecand containing ̂ Mobj(ϕold), ̂ Mobj(ϕcand), rt, def f t , the number of episodes used, and Iacc.

Semantic Reflector for Feedback and Validation

The Semantic Reflector completes the optimization loop by converting sandbox evaluation data into actionable feedback for the next iteration. Functioning as an interpreter, it synthesizes observed tradeoffs and provides a concise diagnostic assessment that rationalizes the acceptance or rejection of each candidate.

This diagnostic process maps sandbox evaluation statistics and high-level policy descriptions into structured feedback. In cases where a candidate policy improves makespan but degrades maximum tardiness, the module interprets the statistical summaries under the chosen objective, produces concise comments that highlight the tradeoff, and determines whether further exploration is warranted.

Diagnostic feedback is logged in the trajectory records, where it serves to rationalize the acceptance or rejection of individual candidates. Its decision signal controls whether the search runs another iteration. The final rule selection, meanwhile, depends solely on sandbox evaluation results and a complexity term.

4.3 Rule Repository for Lifelong Learning

To mitigate knowledge volatility inherent in conventional reinforcement learning and enhance adaptation efficiency across heterogeneous environments, we introduce the Semantic Knowledge Repository. This module persists and retrieves optimized symbolic heuristics, serving as reusable knowledge for RACE-Sched. Through the decoupling of knowledge retention from parameter updates, it preserves high-performing heuristics under nonstationary dynamics. During runtime, the retrieval of validated priors enables faster adaptation than reoptimizing from scratch.

Formal Structure of the Knowledge Repository

The Semantic Knowledge Repository is organized as an indexed library of interpretable symbolic heuristics. Each entry constitutes a tuple Ti = ⟨mi, ϕi, vi, Ci⟩ that links a synthesized policy to its operational context. The meta feature vector mi encodes global problem topology including job and machine cardinality to facilitate similarity based retrieval within the vector space. The component ϕi represents the executable policy source code while the scalar vi quantifies the normalized performance improvement over the baseline. To promote generalizability we incorporate a structural complexity score Ci derived from code statistics such as operator count and branching factor. This metric functions as a regularization penalty during retrieval and prioritizes parsimonious logic that avoids overfitting to specific training instances.

Retrieval and Cross Scenario Transfer

Upon encountering a new problem instance mnew, the system initiates a warm start by retrieving the nearest policy neighbors from the repository. The retrieval score S(Ti|mnew) balances historical efficacy, feature alignment, and complexity:

where vi is the policy efficacy, D(mnew,mi) is the feature distance between the new instance mnew and repository entry mi, and Ci is the policy complexity.

The distance D(mnew,mi) is calculated as:

where Nj and Nm represent job volume and machine capacity, respectively.

This regularization prevents maladaptive transfer by penalizing shifts in job volume and machine capacity.

5 Experiments

5.1 Experimental Setup

Benchmark Datasets and Evaluation Metrics

We validate performance across three benchmark suites encompassing standardized testbeds to high-fidelity simulation settings. GEN-Bench assesses policy generalization capabilities, while MK-Bench extends classical Brandimarte instances to the dynamic domain [Brandimarte, 1993]. JMSBench introduces industrial complexity through the simulation of semiconductor cluster tools with reentrant flows [Cao and Yuan, 2025]. Regarding evaluation metrics, we measure solution quality using the Relative Percent Deviation (RPD) from the Best Known Solution and the average rank across all test instances. We further perform an analysis of cumulative token consumption and wall-clock time to assess the tradeoff between computational efficiency and real-time responsiveness.

Baseline Methods and LLM Backends

We evaluate RACE-Sched against a spectrum of symbolic, neural, and generative baselines. Classical approaches are exemplified by a GP dispatcher [Mei et al., 2016]. Neural methods encompass the attention-based DAN [Wang et al., 2024b] alongside three Reinforcement Learning agents: the breakdown robust IDDQN [Wu et al., 2025], the option-based PPO-OC [Yuan et al., 2025], and the hierarchical HMPSAC [Ding et al., 2025]. Finally, we compare against two LLM baselines: a zero-shot LLM-Direct agent and ReflecSched, which employs reflection-driven simulations [Cao and Yuan, 2025].

The proprietary GPT models include GPT-4o [Hurst et al., 2024] and GPT-5 Nano [Singh et al., 2025], while the open-weight models cover DeepSeek-V3 [DeepSeek-AI, 2024], DeepSeek-V3.2 [DeepSeek-AI, 2025], and the Qwen3 series with 8B, 14B, 32B parameters and Qwen3-Coder30B [Yang et al., 2025]. Inference is managed by using the vLLM [Kwon et al., 2023] high-throughput serving engine, hosted on a compute node configured with an Intel Xeon Platinum 8468V CPU and an NVIDIA H100 80GB GPU.

5.2 Performance and Efficiency

Scheduling Performance Analysis

As shown in Table 1, in normal scale scenarios, where combinatorial complexity is most pronounced, RACE-Sched exhibits superior convergence performance toward global optimality. Specifically, the Qwen3-8b variant achieves an average RPD of 5.33%, outperforming the leading DRL baseline IDDQN with RPD 9.82% by 45.7%. This performance difference highlights the capacity of symbolic evolution to address long-horizon strategic dependencies, which are often obscured in the latent representations of neural approximations.

In Normal scale instances, RACE-Sched variants consistently occupy the upper tiers of the performance hierarchy. Specifically, the Qwen3-8b configuration attains a mean rank of 6.75, maintaining a significant lead over the top performing DRL baselines, IDDQN with rank 12.1 and DAN with rank 12.55. In Small scale instances, despite the compressed optimization margins characteristic of smaller search spaces, RACE-Sched retains a distinct performance over LLM-Direct and ReflecSched. This cross scale stability underscores that the synthesized symbolic code captures fundamental scheduling principles, enabling generalization that remains invariant to shifts in job volume and machine density.

Token Consumption Assessment

Table 2 presents a comparative evaluation of cumulative token expenditure. The Semantic Knowledge Repository significantly reduces reliance on token-intensive generative exploration. Across all model backends ranging from Qwen3 to GPT-5, RACE-Sched consistently requires the lowest input and output volume. Specifically, under the Qwen3-8b configuration, our framework lowers input token consumption by approximately 60% relative to LLM-Direct baseline. This efficiency gain arises from two structural advantages. First, the retrieval mechanism utilizes archived priors to prune the search space and eliminate redundant inferential steps. Second, the transition from per step action generation to compact policy synthesis drastically minimizes output volume. By confining LLM activation to performance critical triggers, the architecture effectively balances the complexity associated with deliberative reasoning and computational sustainability.

Real-Time Latency Evaluation

Table 3 highlights the significant discrepancy between heuristic execution and deliberative reasoning. The Reactive Stream consistently exhibits sub-millisecond latency across all benchmark suites. Notably, the DeepSeek V3.2 and GPT5 Nano configurations achieve a mean response time of less than 0.1 ms. This deterministic performance ensures that the tactical control loop complies with the strict real-time constraints imposed by industrial manufacturing environments.

In contrast, the Deliberative Stream functions on a timescale that is orders of magnitude longer with latencies ranging from 77 to 114 seconds per policy update. Synchronous integration of such high-latency reasoning would result in catastrophic blockages of the production line. RACE-Sched mitigates this risk by isolating the Deliberative Stream from the critical execution path. New rules are deployed only after validation via an atomic update. This architectural design maintains dispatching latency at the microsecond level, while enabling continuous long-horizon optimization.

5.3 Ablation Studies

We compare four structural variants.

• Full: The complete architecture integrating both repository retrieval and iterative deliberation for continuous

rule optimization.

• w/o Loop: Disables the reasoning loop to generate a single candidate rule per step without retrospective feedback.
• w/o Repo: Removes knowledge retrieval to force a cold start initialization without historical priors.
• w/o Both: Excludes both components to serve as a baseline lacking memory and self correction mechanisms.

We retrieve archived rules based on normalized Manhattan distance within a meta-feature space defined by job and machine counts. To ensure reproducibility during benchmarking, a serialized execution protocol is implemented, while the deployed system maintains asynchronous decoupling.

Contribution of Epistemic Transfer via Rule Repository

Table 4 highlights the critical role of the Semantic Knowledge Repository in mitigating cold start inefficiencies. The comparison between the Full architecture and the variant excluding the repository reveals a substantial performance gap. On the GEN-Bench Small dataset, the retrieval of historical priors reduces the Relative Percent Deviation from 26.19% to 10.41%. This finding indicates that initializing deliberative reasoning with heuristics derived from similar problem instances effectively prunes the search space and minimizes reliance on stochastic exploration. Although the magnitude of improvement varies across specific datasets, the repository provides significant stability on a global scale. The Consolidated metrics confirm this structural advantage: the Full model achieves a global average deviation of 9.39%, in contrast to 14.07% for the unaugmented baseline.

Impact of Recursive Agentic Refinement

A comparison between the Full model and the variant without the loop reveals that refinement generally improves solution quality for complex tasks. For instance, on the GENBench Normal dataset, enabling the loop reduces the RPD from 8.17% to 6.16%. This gain stems from the capability of the system to analyze sandbox feedback and rectify logical inconsistencies in subsequent iterations. Although the singlestep approach performs competitively on simpler tasks, the Full architecture secures a superior consolidated average rank of 2.32 compared to 2.40 for the variant lacking the loop. This confirms that iterative validation provides greater stability across diverse problem distributions.

5.4 Resilience Under Machine Failures

We evaluate operational robustness under nonstationary conditions using a stress test within the JMS Bench framework. Figure 2 depicts the system's response to an abrupt capacity loss at a critical workstation, which occurs at time Tfail = 12.0.

The baseline approaches demonstrate inadequate adaptability to this structural shift. The Single Cold variant results in the worst makespan of 35.0 due to its reliance on a static heuristic that cannot adjust to the new bottleneck. Correspondingly, the DRL baseline HMPSAC exhibits high variance and a prolonged recovery period, which suggests insufficient generalization to out-of-distribution states.

In contrast, RACE-Sched demonstrates superior resilience. Upon detection of the throughput degradation, the Deliberative Stream asynchronously synthesizes and deploys a new priority rule tailored to the altered system dynamics. This rapid adaptive response restores throughput to pre-failure levels and results in the minimum final makespan of 29.6. These results confirm that the asynchronous decoupling effectively supports continuous policy refinement without blocking the real-time control loop.

6 Conclusion

We introduce RACE-Sched to reconcile the reasoning capability of LLMs with the latency constraints of industrial control systems. By decoupling fast heuristic execution from asynchronous policy synthesis, RACE-Sched maintains millisecond-level responsiveness while continuously evolving symbolic control logic. The Semantic Knowledge Repository reduces redundant deliberation through retrieval-based reuse, and sandbox validation prevents unverified rules from entering the online control loop. Extensive experiments across GEN-Bench, MK-Bench, and JMS-Bench show that RACE-Sched outperforms state-of-the-art DRL and generative scheduling baselines while preserving low-latency online execution.

RACE-Sched also has several deployment limitations. Its rule evolution quality depends on the capability of the background LLM, since weaker models may generate invalid or sub-optimal candidate rules that are rejected by sandbox validation. The construction of representative sandbox scenarios and trigger conditions requires engineering effort, because the validation environment must reflect the main disturbance patterns of the target shop floor. Although the asynchronous design prevents deliberation from blocking the control loop, reaction to brand-new disturbance types can still be delayed when candidate synthesis and validation require multiple deliberation cycles.

Ethical Statement

There are no ethical issues.

Acknowledgments

This work was partially supported by Shenzhen Loop Area Institute under Project No. FP202602. The authors gratefully acknowledge the support from the project team members involved in this work.