SCOPE: Safety-Constrained Online Preview Enforcement for Efficient Encirclement in Multi-UAV Pursuit-Evasion

1 Introduction

In safety-critical pursuit-evasion scenarios, safety constraints often require that unmanned aerial vehicle (UAV) swarms intercept a rogue UAV in a non-destructive manner. To achieve non-destructive interception, a common solution is cooperative encirclement, especially in cluttered low-altitude airspace [Yu et al., 2024; Valianti et al., 2024; Ma et al., 2020]. Compared with direct capture, encirclement requires multiple UAVs to close in simultaneously and maintain a coherent formation under dense interactions among UAVs, the evader, and moving obstacles [Dong et al., 2020]. To address this problem, research on reinforcement learning for safetycritical has attracted considerable attention.

∗Corresponding authors.

Multi-agent reinforcement learning (MARL) has achieved encouraging success in safety-critical autonomy, including robotic control [Duan et al., 2024], autonomous driving [Lee and Kwon, 2024], and airspace security [Du et al., 2021]. However, safety constraints are frequently incorporated only as a negative reward, which may not explicitly prevent instantaneous violations during execution [Qu et al., 2024; Zhang et al., 2023b; Zhang et al., 2024]. In practice, soft penalty allows the policy to trade safety for return, so it may incur instantaneous violations for higher efficiency, which is not permitted in safety-critical encirclement. As illustrated in Figure 1, multi-UAV pursuit-evasion requires efficient encirclement under safety constraints. Since the UAV swarm must satisfy strict multi-constraint safety, even instantaneous violations can cause collisions or mission failure [Lv et al., 2024]. In such settings, traditional MARL can lead to safetyefficiency trade-off: an unsafe task policy that violates safety constraints, or a conservative policy that prevents successful encirclement [Valianti et al., 2024].

Recently, more research has focused on safe MARL to handle safety constraints. Nevertheless, for multi-UAV pursuitevasion, it is still hard for existing safe MARL methods to achieve safe and efficient encirclement [Chen et al., 2025; Li et al., 2023]. In general, safe MARL approaches can be divided into objective-level and execution-level methods. Objective-level methods, such as primal-dual formulations, penalty-based schemes [Liu et al., 2020; Zhang et al., 2022], and risk-sensitive objectives [Chow et al., 2015], build constraints into the return or the optimization objective [Achiam et al., 2017], but they often provide only soft guarantees and may still allow brief safety violations during execution. Execution-level methods adjust actions before execution to enforce safety online [Alshiekh et al., 2018; Zhang et al., 2023a; Zhou et al., 2025], but they usually do not account for whether safety remains feasible over predicted trajectories, and they are often designed mainly for obstacle avoidance while ignoring inter-agent safety constraints [Zhao et al., 2025; Wabersich and Zeilinger, 2023].

To achieve safe and efficient encirclement, three challenges need to be addressed. (1) Safety constraints are often violated when there's no preview. With moving obstacles, aggressive closing rapidly shrinks safety margins, making nearfuture violations hard to predict and easy to miss until they occur. (2) Safety constraints often induces conservative policies that undermine encirclement efficiency. Conservative policies can fragment the formation and delay or prevent encirclement, making it difficult to enforce strict safety constraints without sacrificing efficiency. (3) Ensuring forward feasible under dense interaction is challenging. Safety constraints depend on other moving entities and evolve over time, making forward feasible safety difficult to maintain and constraint satisfaction hard to guarantee in future states.

To address these challenges, we propose SafetyConstrained Online Preview Enforcement (SCOPE) for safe multi-UAV pursuit-evasion. SCOPE combines an online Safety-Preview Dynamics Model (SPDM) to produce shorthorizon safety previews and predict future trajectory under dense interactions. Then Preview-Fused Actor-Critic uses these previews to learn efficient encirclement policies without excessive conservatism. Hierarchical Safety Enforcement applies safety look-ahead and strict margin tightening to correct joint actions and maintain forward feasible safety. In addition, the safety enforcement provides a safety loss to networks update, encouraging safe and effective coordination during training. Together, these components address the safety-efficiency trade-off by improving encirclement performance while maintaining safety constraints.

The main contributions are summarized as follows:

• We design an online SPDM with preview-fused actorcritic that uses short-horizon safety previews to better mitigate the safety-efficiency trade-off in encirclement.
• We propose hierarchical safety enforcement, which enforces safety constraints via online action correction and safety loss during policy learning, while safety lookahead maintains forward feasible safety.
• We build a challenging multi-UAV pursuit-evasion task and validate SCOPE on it, showing advantages over state-of-the-art baselines via comparative and ablation studies. Results show that SCOPE completes tasks efficiently while maintaining forward feasible safety.

2 RelatedWorks

2.1 Model-BasedMethods for Encirclement

Encirclement and pursuit-evasion have long been studied from the perspective of model-based control and planning.

Cyclic-pursuit coordination is a typical choice for forming enclosing patterns [Kim et al., 2010]. Under limited sensing, range-only and coordinate-free guidance has been explored [Dong et al., 2020]. Model predictive control is often used when safety constraints are needed in dynamic encirclement [Hafez et al., 2014]. These methods can work well under their assumptions, but they are difficult to extend to dense agent interactions and moving obstacles.

2.2 Safe MARL for Encirclement

Safe MARL introduces safety costs into MARL and optimizes return under a violation budget [Hazra et al., 2025]. Dense constraints and non-stationary learning, however, make safe coordination difficult. Existing methods are often discussed at two levels: objective-level and execution-level. For objective-level methods, costs are folded into the optimization objective through primal-dual, penalty-based, or trust-region style updates [Liu et al., 2020; Zhang et al., 2022; Achiam et al., 2017]. MAP3O is a representative penaltybased method [Zhang et al., 2022]. Lyapunov-based methods enforce long-horizon safety via local constraints [Chow et al., 2018], whereas PID-Lagrangian stabilizes multiplier updates [Stooke et al., 2020]. Objective-level methods can be sensitive to hyperparameters, and their constraint satisfaction is usually soft or asymptotic [Xiao and Feroskhan, 2024; Li and Azizan, 2024]. In dense interactions, this can result in conservative policies or occasional instantaneous violations.

Execution-level methods, often referred to as shielding, modify actions at runtime to prevent violations [Alshiekh et al., 2018; Brorholt et al., 2024]. Control barrier function based filters support online checking and correction [Cai et al., 2021], and recovery-based methods invoke recovery behaviors near unsafe regions [Zhao et al., 2025]. Frequent interventions may alter coordination strategies and degrade task efficiency in dense interactions [Sun et al., 2023].

3 Preliminaries

We model the multi-UAV pursuit-evasion problem as a Constrained Markov Decision Process (CMDP), defined by the tuple ⟨S, O, A, R, C, P, γ⟩, where S denotes the state space, O denotes the observation space, A denotes the joint action space, R : S × A → R is reward function, C : S × A → R is the cost function used to quantify safety constraints, P : S × A × S → [0, 1] is transition probability function,γ ∈ (0, 1) is the discount factor for calculating the return. At time step t, the environment is in state s(t) ∈ S and each agent i receives a observation o(t) i drawn from an observation kernel Ω : S → ∆(O). The objective of CMDP is to find a joint policy π : O → ∆(A), that maximizes the expected discounted return:

where ΠC = {π ∈ Π : JC(π) ≤ d} is allowable policy set, τ denotes a joint trajectory, ˜a(t) denotes joint actions. JC(π) is the cumulative constraints similar to reward and d is the constraint threshold. Thus, the task reduces to learning a joint policy π that maximizes J (π) while ensuring JC(π) ≤ d.

4 Methodology

Pipeline of SCOPE is shown in Figure 2. SCOPE combines short-horizon safety preview with online constraint enforcement for multi-UAV pursuit-evasion. Safety-Preview Dynamics Model (SPDM) predicts future trajectory for PreviewFused Actor-Critic. Hierarchical Safety Enforcement corrects actions and injects safety loss to guide learning, enabling forward feasible safety in dense environments.

4.1 Safety-PreviewDynamics Model

The safety margin between agents and obstacles changes rapidly as the formation closes in during pursuit-evasion. To address this, we construct a SPDM to predict short-horizon future observations and provide previews for policy learning under safety constraints.

For stability and low online overhead, we employ a parameter-shared SPDM assign each agent a dynamic normalizer to handle distribution shift from evolving policies and heterogeneous observations. The surrogate dynamics model H, parameterized by ϕ(t) , takes as input a length-H history of normalized observations Z (t) i = [ z(t-H+1) i , . . . , z(t) i ] and generates a K-step rollout for all N agents:

where ˆ T (t) i = [ˆl(t+1) i , . . . ,ˆl(t+K) i ] denotes the trajectory within horizon K for agent i. For initial timesteps with insufficient history, we pad missing observations with a mask so as to preserve input length and temporal alignment.

Unlike conventional dynamics models trained offline, the SPDM is updated alongside the RL process using experience sampled from the replay buffer D. At each update step, we sample a mini-batch of B transitions and define a mapping f : {1, . . . ,B} → N that associates batch indices with global timesteps. For the b-th sample in the mini-batch, we denote its corresponding global timestep by tb = f(b). The per-agent dynamic normalizers are updated incrementally with each batch, in parallel with the SPDM parameter updates. For the SPDM, each sampled trajectory T (tb) i = [ l(tb-H+1) i , . . . , l(tb) i , . . . , l(tb+K) i ] is transformed into a normalized input Z (tb) i and ground-truth targets Y (tb) i . Feeding the history Z (tb) i into the dynamics model yields the trajectory rollout ˆ T (tb) i , which is kept accurate for predicting imminent safety violations by online training. We train SPDM by minimizing the discrepancy between rolled-out and groundtruth trajectories, with the rollout loss defined as:

The learned SPDM is then used online to generate the Kstep preview ˆ T (t) i , which is fused with the current observation to form the inputs to the actor-critic described in Sec. 4.2.

4.2 Preview-Fused Actor-Critic

Early training can be particularly challenging due to highdimensional inputs and tightly coupled multi-agent learning dynamics. General actor-critic often struggles here: the actor may overfit noisy or irrelevant channels, and the Critic may fail to capture inter-agent dependencies while policies co-adapt. We introduce two lightweight enhancements as illustrated in Figure 3: a gating module in the actor for datadependent feature selection, and a multi-head attention module in the Critic to model interactions between agents and downweight unreliable near-future previews.

Gating-Enhanced Actor

To improve robustness under high-dimensional inputs, we introduce a gating module related to observation that performs feature selection. For each agent i, the actor µi, parameterized by θ(t) i , takes the local observation o(t) i and produces a control decision u(t) i before the hierarchical safety enforcement. We first transform the local observation o(t) i into a feature vector x(t) i . Then, the gating module generates a set of weight from o(t) i and uses it to scale the features:

where ˆx(t) i is scaled feature, W (t) , b(t) are trainable parameters of the gating unit, σ(·) is the sigmoid activation function and ⊙ is the Hadamard product. The gate vector has the same dimensionality as the feature. This design improves robustness when SPDM rollouts are noisy or partially missing.

Multi-Head Attention Critic

We introduce a multi-head attention critic to robustly aggregate multi-agent information. The critic Qi, parameterized by ψ(t) i , is responsible for estimating the expected return of agent i given the joint state-action pair [s(t) , a(t) ]. Finishing partitioning, a shared state-action network encodes each agent's feature into an embedding. For attention head h:

where qi,h is a learnable query from o(t) i and Kh, V h are the key and value derived from s(t) . The weighted feature embeddings are aggregated into global representation ch = ∑Ni=1 αi,hV h and fused with the original features.

4.3 Hierarchical Safety Enforcement

To further provide runtime safety protection and prevent instantaneous violations under coupled constraints, especially in high-risk regions. Therefore, we introduce hierarchical safety enforcement, an online mechanism based on control barrier functions. Given the actor's decision ˜u(t) , this module computes minimally modified joint actions ˜a(t) to satisfy coupled safety constraints. Meanwhile it provides safety loss through a CBF-based penalty integrated into the actor loss.

Basic CBF Enforcement

At each timestep t, the actor network outputs joint decision ˜u(t) , and the basic CBF performs a preliminary enforcement step. We cast action correction as a joint quadratic program that satisfies the safety constraints while minimally deviating from the policy intent. Specifically, the basic CBF enforcement finds the minimally modified action closest to ˜u(t) subject to the CBF constraints. The objective of the basic CBF is to minimize the magnitude of action corrections while enforcing the joint safety constraints ξ(·):

where ˆa denotes the corrected actions, λ1, λ2 > 0 are damping factors that regulate the responsiveness of the barrier function. The safety constraints are detailed in Appendix C.

Safety Look-Ahead

While the basic CBF enforcement guarantees instantaneous safety, it does not ensure forward feasible safety. This may lead to deadlock or infeasibility in highly dynamic pursuitevasion encirclement. To address this issue, we add a onestep safety look-ahead feasibility check and invoke a strict CBF enforcement when necessary.

Specifically, after obtaining ˆa(t) , we use the SPDM to produce the one-step preview ˆs(t+1) from the observation history, which is then used to maintain forward-feasible safety:

Feasibility at ˆs(t+1) holds if there exists a conservative action a(t+1) ∗ satisfying the safety constraints.

Strict CBF Enforcement

When the conservative action a(t) ∗ does not exist, the basic CBF enforcement may fail to maintain continuous feasibility. Therefore, we introduce a strict CBF enforcement that tightens the safety margin with strictness parameter δ > 0 to recover feasibility. The optimization objective remains the same as Equation. (6), while we adopt a strict CBF constraint:

We integrate a smooth CBF-based penalty into the actor loss, jointly optimizing safety and task performance. The loss due to CBF constraint is derived as: LCi (ℓ(tb) i ) = log(1 + exp(-ℓ(tb) i )). The combined loss is stated as follows:

Algorithm 1 Overall pipeline of SCOPE

Input: Initial parameters ˜θ(0) , ˜ψ(0) ,ϕ(0) , Initial state s(0) Output: Trained parameters ˜θ(T) , ˜ψ(T) ,ϕ(T)

• 1: for t = 1 to T or mission complete do
• 2: # Hierarchical safety enforcement
• 3: For each agent i, select u(t) i = µi(o(t) i |θ(t) i ) +N (t)
• 4: Correct preliminary actions ˜u(t) = (u(t) 1 , . . . , u(t) N ) to ˜a(t) = (a(t) 1 , . . . , a(t) N ) using Equation. (5)
• 5: Execute actions ˜a(t) and receive reward r(t) , new state s(t+1) and raw observation l(t+1)
• 6: # SPDM rollout and update
• 7: Roll out { ˆ T (t+1) i }Ni=1 using Equation. (2) and get fused observation ˜o(t+1) = (o(t+1) 1 , . . . , o(t+1) N )
• 8: Sample a randomminibatch of B trajectories fromD
• 9: For each batch b, split {T (tb) i }Ni=1 into normalized input {Z(tb) i }Ni=1 and ground truth {Y (tb) i }Ni=1
• 10: Update parameters ϕ(t) using Equation. (3)
• 11: # Policy update
• 12: for agent i to N do

• 15: Update actor θ(t) i minimizing Equation. (9)
• 16: end for
• 17: Update target network parameters for each agent i: ˆω(t+1) i ← (1- τ) ˆω(t) i + τ ω(t) i ,ωi ∈ {θi,ψi}.
• 18: end for

where ηi > 0 is the penalty factor, ℓ(tb) i = ¨ ξi(s(tb) , ˜a(tb) ) + λ1 ˙ ξi(s(tb) )+λ2ξi(s(tb) ) denotes the CBF constraint of agent i. The overall pipeline of SCOPE as Algorithm 1.

The output of the algorithm is multi-agent policies and surrogate dynamic model. Here ωi is the target network parameters of agent i.

5 Experiment

In this section, we present the experimental evaluation of SCOPE on multi-UAV pursuit-evasion environments and compare its performance with safe MARL baselines. For benchmarking, we consider the following methods.

• Lagrangian method: MADDPG [Lowe et al., 2017] with a Lagrangian multiplier and MACPO [Gu et al., 2022].
• Penalty-based method: MAP3O [Zhang et al., 2022].
• Shield-based method MADDPG with a control. barrier functions [Cai et al., 2021]
• Recovery RL method: MSMAR [Zhao et al., 2025].

All the above algorithms use either on-policy or off-policy replay buffers, following their respective original implementations. Additionally, we include a comparison with the MADDPG algorithm [Lowe et al., 2017] as a baseline. Baseline implementations partially rely on the OmniSafe repository [Ji et al., 2024] for standard safe MARL components. We evaluate all methods in our multi-UAV pursuit-evasion environments, where multiple interceptor UAVs must collaboratively encircle a rogue intruder while satisfying safety constraints with respect to other agents, obstacles, and workspace boundaries. The environments include both static and moving obstacles that test the agents' ability to maximize cumulative rewards while adhering to the safety constraints formalized in Equation. (1). We additionally evaluate SCOPE in MuJoCo [Todorov et al., 2012] to verify its effectiveness under physical dynamics, with details reported in Appendix B.

5.1 Multi-UAV Pursuit-Evasion Setup

Our multi-UAV pursuit-evasion environment simulates a team of N = 4 interceptor UAVs attempting to safely encircle a rogue UAV in a cluttered airspace. The environment is modeled as a continuous workspace with boundaries and several obstacles, where each interceptor is equipped with a short-range jamming device.The objective is to encircle the fleeing rogue UAV for on-board jamming while maintaining minimum separations among UAVs and from obstacles. For these experiments, we use a fixed safety radius rsafe and jamming range rjam, task duration T = 400, and evaluate each trained policy on NE = 200 test episodes. Detailed environment descriptions are provided in the supplementary material.

5.2 EvaluationMetrics

We evaluate all methods from both task performance and safety perspectives. Task performance is measured by:

1. Success rate SR: the fraction of episodes in which the interceptors successfully encircle the rogue UAV within the horizon T ;
2. Average time AT: the average number of steps to success, where unsuccessful episodes are assigned T .

Safety performance is measured by:

1. Average agent cost AC: the average number of safetyconstraint violations per agent during an episode:

where NE is the number of evaluation episodes, N is the number of interceptors, and Ce denotes the total safety cost accumulated in episode e. Specifically, Ce consists of the magnitude of constraint violations against obstacles and other agents, and an additional cost for shield activation when a shielding or safety enforcement intervenes.

5.3 Performance Comparison

We evaluate SCOPE against the safe MARL baselines listed above under identical interaction budgets and environment configurations. All methods share the same reward design, termination conditions, observation space and action spaces. We report the mean performance over multiple random seeds, and evaluate each trained policy on test episodes using the metrics in Sec. 5.2.

Figure 4 shows that SCOPE achieves higher episodic returns while maintaining low safety costs more consistently throughout training in both static- and moving-obstacle settings. Vanilla MADDPG exhibits large safety-cost spikes and unstable learning, and many runs time out at T = 400, indicating failures to complete encirclement under unconstrained exploration in cluttered scenes. MAP3O reduces safety costs relative to MADDPG, but converges to lower returns with longer episode lengths (Steps), while MACPO shows low SR and high AC, suggesting conservative policies that hinder efficient pursuit-evasion. MSMAR similarly suppresses safety costs, yet also converges to lower returns and longer Steps, indicating conservative recovery behaviors that slow down encirclement. MADDPG-Lagrange shows degraded performance in the moving-obstacle setting with increased costs and more failed episodes, highlighting the difficulty of balancing coupled constraints under non-stationary dynamics. MADDPG-CBF improves returns compared with penaltybased baselines, but still incurs substantially higher safety costs than SCOPE, consistent with frequent online interventions in dense interactions.

Table 1 further confirms that SCOPE achieves the strongest overall performance in both scenarios. With moving obstacles, SCOPE reaches the highest success rate of 61.40% and the lowest safety cost of 16.07, while keeping the completion time competitive at 279.38 steps. With static obstacles, SCOPE attains 83.10% success, the fastest completion time of 218.77 steps, and the lowest safety cost of 12.67. Other methods typically sacrifice either success and speed for safety, or safety for task completion, whereas SCOPE remains strong on both.

5.4 Ablation Studies

To examine the contribution of each component in SCOPE, we perform ablation studies by removing one module at a time while keeping the training protocol and hyperparameters fixed. Figure 5 plots the learning curves of episodic returns and safety costs. We conduct ablations in the movingobstacle setting, as it is the most challenging scenario with non-stationary interactions and thus better differentiates the effects of each module.

Overall, SCOPE achieves the best safety-performance trade-off, converging to the highest return while maintaining consistently low cost. Removing the SPDM (SCOPE-SPDM ) slows convergence and reduces the final return, indicating that short-horizon previews provide anticipatory information for coordinated pursuit-evasion. Removing the hierarchical safety enforcement (SCOPE-HSE ) incurs persistently higher costs and leads to a markedly lower return plateau, suggesting that execution-time safety correction is critical and that frequent constraint violations can hinder policy learning. Removing the preview-fused actorcritic network enhancements (SCOPE-GA) results in only a minor drop in return with comparable costs, implying that the gating/attention modules mainly improve coordination and credit assignment. Finally, removing both preview and safety enforcement (SCOPE-All ) yields the worst performance, characterized by pronounced early cost spikes and a low-return regime throughout training.

5.5 Component Study on SPDM

We study the impact of the predictive backbone used in SPDM. To ensure a fair comparison, we keep the environment, RL algorithm, reward design, and hierarchical safety enforcement unchanged, and replace the SPDM with six models: DLinear, PatchTST, iTransformer, TimesNet, LSTM, and TimeMixer.

As shown in Figure 6 (right), TimesNet and iTransformer yield the highest and most stable returns, while DLinear achieves competitive performance with fast early improvement, suggesting that different backbones trade off early learning speed and asymptotic performance. PatchTST and TimeMixer converge to lower returns, and LSTM performs worst, indicating that the predictor choice can affect cooperative pursuit-evasion and training robustness. Notably, Figure 6 (left) shows that lower prediction loss does not necessarily imply better control performance: although LSTM attains a low SPDM loss, it leads to poor returns, suggesting that minimizing the regression loss does not necessarily lead to optimal control performance.

5.6 Parameter Sensitivity Analysis

We analyze SCOPE's sensitivity to the strictness parameter δ by varying δ while keeping all other hyperparameters fixed. Since δ directly sets the tightening level of the enforced CBF constraint, it affects the size of the feasible action set and thus influences both task execution speed and constraint satisfaction. We evaluate each setting using the metrics in Sec. 5.2.

As shown in Figure 7, SCOPE is stable for δ ≤ 0.09 with similar convergence returns, while larger margins degrade learning. δ = 0.15 leads to a clear return drop and δ = 0.21 collapses to a low-return regime, suggesting that overly strict tightening restricts the feasible action set and hinders coordinated pursuit. Meanwhile, increasing δ monotonically increases the time-related metric Rt, indicating slower encirclement, whereas the safety cost Rc is minimized at a moderate margin and rises again for larger δ, reflecting a clear safety and efficiency trade-off. We therefore use δ = 0.06 in the remaining experiments.

6 Conclusion

In this paper, we proposed Safety-Constrained Online Preview Enforcement (SCOPE), a MARL-based algorithm for multi-UAV encirclement in pursuit-evasion. SCOPE integrates a Safety-Preview Dynamics Model (SPDM) for shorthorizon anticipation with hierarchical safety enforcement and look-ahead feasibility checking to maintain forward feasible safety. Extensive experiments in challenging scenarios, together with additional evaluations in MuJoCo, demonstrate that SCOPE consistently improves task performance while substantially reducing safety cost compared to state-of-theart safe MARL baselines.

Future work will focus on scaling SCOPE to larger swarms and more diverse team sizes, where coordination and safety constraints become increasingly complex. We also plan to incorporate richer physical constraints and more realistic onboard limitations, and to study adaptive safety margins that adjust online to the environment dynamics and risk level.

Ethical Statement

This work targets civilian public-safety applications, such as interception of unauthorized drones in restricted airspace. The proposed method is not intended for military, offensive, or harmful use, and should be deployed only under legal regulations and proper human oversight.

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (Grant No.62472220, No.62571240, No.U22B2062, No.62502204), the Natural Science Fund of Jiangsu Province (Grant No.BK20251381), the Open Project Program of State Key Laboratory of Novel Software Technology, Nanjing University (KFKT2025B14), the open research fund of National Mobile Communications Research Laboratory, Southeast University(Grant 2026D06), the Fundamental Research Funds for the Central Universities (NS2025041), Collaborative Innovation Center of Novel Software Technology and Industrialization. This work was partially supported by High Performance Computing Platform of Nanjing University of Aeronautics and Astronautics.