Strategies for Cooperative UAVs Using Model Predictive Control

1 NMPC Based Approach for Cooperative Target Defence

The multi-agent pursuit-evasion problem is an application that appears in several domains. An interesting subset of the multi-agent pursuit-evasion problem is the three agent case, involving target, attacker and defender [Garcia et al., 2015]. If the target and the defender cooperate, then they can maneuver such that defender intercepts the attacker while target maximizes the chances of escape.

There are many applications like aerial surveillance, communication relaying, where the target may need to stationary. In such situations, a defender tries to defend the target from any attacking agents. The limitations with the earlier methods considering the three-body interception scenario are that (i) the attacker states are assumed to be known accurately and (ii) the target has constant speed.

In this study [Manoharan et al., 2019b], we propose a control strategy that combines a Nonlinear Model Predictive Control (NMPC) scheme with state estimation using an Extended Kalman Filter (EKF) while addressing (i) and (ii). The formulation also considers the case in which the target can be stationary unless strictly necessary.

1.1 NMPC Formulation

We propose a strategy wherein the target-defender team will use NMPC to compute the control commands so that the target evades the attacker, while the defender intercepts the attacker before it could capture the target. The formulation allows the target to move or stop based on the engagement geometry. The objective of the target is to stay still so as to capture images of the area of interest, and to move only when necessary. This condition can be achieved by defining a threshold, e which is the safe distance of the target from the attacker. The target should always try to keep this safe distance, e from the attacker. The NPMC problem is defined as:

where X is the state vector, U is the control vector, ux, uy are the velocity components of the target in x and y directions respectively, R and r represents the distance between the attacker and the target and the distance between the defender and the attacker respectively (see [Manoharan et al., 2019b] for more details).

1.2 Results

The performance of the proposed MPC formulation was evaluated through numerical simulations and hardware experiments performed using ground rovers. A comparison with previously studied CLOS formulation [Ratnoo and Shima, 2011] was also carried out to show the efficacy of the proposed scheme. The experimental results are shown in fig. 1. A video of preliminary results can be seen at: https://youtu.be/IuvKP1vifh0.

2 Nonlinear Model Predictive Control to Aid Cooperative Localization

We propose a nonlinear model predictive control (NMPC) scheme to tackle the problem of localization and path planning of a group of unmanned aerial vehicles (UAVs) in global positioning system (GPS) denied environments [Manoharan et al., 2019a]. It is assumed that the UAVs can cooperate by sharing information among themselves. It is also assumed that the area under consideration contains some landmarks with known locations. The NMPC computes the optimal control inputs for the vehicles such that the vehicles cooperate to transit from a source location to a destination while choosing a path that will cover enough landmarks for localization. An Extended Kalman Filter (EKF) is used to estimate the vehicle positions using only relative bearing measurements.

2.1 Proposed NMPC Formulation

is the cost associated with minimizing the distance between the vehicle and the destination, Xi is the state vector of ith vehicle, nv is the number of vehicles and C2i is the cost associated with maintaining the connection with the landmarks which is defined as:

where η is a constant representing the minimum number of required connections and λi is defined as the maximum eigenvalue of the Laplacian matrix. The weight W in equation (3) is equal to W, if 3σp ≥ σc, else W = 0. σp is the standard deviation in the estimated position of the vehicles and σc is the critical value of standard deviation. This adaptive weight formulation enables determination of the optimal path for the vehicle while satisfying the estimation error within bounds (see [Manoharan et al., 2019a] for more details).

2.2 Results

Monte-carlo simulations were performed using three UAVs with different number and configurations of landmarks. One such scenario is shown in fig. 2 for 3 UAVs and 20 landmarks.

3 FutureWork Involving AI Techniques

Determining optimal multi-agent control theoretic strategies for large number of agents is difficult due to the sensing and computational limitations. However, AI techniques have shown promise with reduced computation time at the run time while providing solution close to optimal.

Reinforcement Learning (RL) is a promising candidate that has been applied for multi-agent systems [Panait and Luke, 2005] and shepherding problem [Go et al., 2016]. SARSA and Q-learning are two well-known examples for learning algorithms in reinforcement learning that are also promising approaches. The main difference between the classical dynamic programming methods and the reinforcement learning algorithms is that the latter do not assume knowledge of an exact mathematical model of the Markov Decision Process (MDP) and thus can be used for large MDPs where exact methods become infeasible.

Therefore the multi-vehicle cooperative localization problem can be extended to use AI techniques for finding the best path with less computational effort and time. Also, for the extension of the single target-attacker-defender problem to a pursuit-evasion problem with multiple number of agents, AI techniques are the best suited option.