PRASS: Probabilistic Risk-averse Robust Learning with Stochastic Search

1 Introduction

Deep neural networks (DNNs) have seen widespread adoption across a multitude of domains [Zhang et al., 2020b; Mu et al., 2021]. Their success, however, has been accompanied by growing concerns about robustness, especially in safety-critical environments [Huang et al., 2020; Huang et al., 2017]. This is due to the vulnerability of neural networks to adversarial perturbations, where subtle modifications to benign inputs can significantly mislead the model's predictions without significantly affecting human perception. Considering the wide range of adversarial attacks, a variety of defense methods have arisen, with adversarial training [Madry et al., 2018] notably taking the spotlight. However, existing research primarily focuses on improving adversarial robustness against the worst-case risks posed by explicit adversaries. Though highly suitable in some instances, these approaches are not universally applicable.

∗Corresponding Author

Firstly, there can be concerns related to robustness against naturally occurring corruptions or random input perturbations, as opposed to an explicit adversary. For example, in autonomous systems, input data may be derived from various sources, such as onboard sensors, GPS, or interaction or communication with a complex environment. These may contain noise, be subject to signal degradation, or other variations due to environmental factors such as weather or electromagnetic interference. We desire our neural networks in such situations to accurately process and analyse input data, making safe and robust decisions in the presence of such random variations. In this context, a classifier must account for these variations, but some degree of perturbation or risk will typically be deemed acceptable. The crux is not about achieving complete immunity from adversarial examples, as that might often be unrealistic or even excessive. Rather, there is a balance to be had: enabling robustness with acceptable risk thresholds, ensuring optimal performance even in the face of uncertainty.

Secondly, in practice, we are usually concerned with the overall network robustness, i.e., robustness across the range of potential inputs, rather than its input-specific robustness. This has inspired the exploration of robust learning with adversarial risk as a metric of the model's worst-case performance on adversarial perturbations. Based on such an ideal robustness definition, a classifier is deemed satisfactory only if it can withstand the full gamut of potential perturbations. However, such stringent safety requirements are rarely achievable and applicable in real-world settings. As [ISO, 2014] posits: "safety risks and dangers are inescapable; residual risks endure even after risk reduction measures have been executed". Furthermore, prior work [Schmidt et al., 2018; Yin et al., 2019] has demonstrated that these robust learning approaches exhibit poor generalisation from training to testing phases, substantiated by both theoretical analyses and empirical tests on real networks, significantly diminishing their broad-scale applicability.

To address these limitations, we propose a risk-averse robust learning framework for DNNs, which originates from relaxing worst-case adversarial risks by introducing Entropic Value-at-Risk (EVaR) risk and taking perturbation distributions into account. The risk of a classifier, termed as the probabilistic robustness, is calculated with a very tight upper bound derived from the Chernoff inequality over an input perturbation distribution. Further expanding on this concept, the risk metric evolves into a distributional robustness definition by employing adaptive stochastic search to iteratively update the perturbation distribution. It's crucial to understand that our framework does not seek to replace adversarial risk or serve as a means to learn adversarially robust networks; instead, it pioneers a risk-aversion training scheme that capably mitigates the potential risk of perturbations overall, rendering it more appropriate and pragmatic in some scenarios.

In summary, our contributions in this work encompass: (C1) Ours is the first work expanding on risk-sensitive robust learning framework for DNNs from a distribution perspective. Inspired by risk-aware optimisation and stochastic optimisation, we present Probabilistic Risk-Averse Robust Learning with Stochastic Search (PRASS). Here, the robustness risk is calculated with the tightest upper bound from the Chernoff inequality. (C2) We provide theoretical and empirical results showing that the proposed risk metric has superior generalisation performance to its corresponding adversarial risks, particularly in high-dimensions, with bounds on the generalisation error respectively scaling as O (log (d)) and √d in the size of the network, respectively. (C3) We propose a tractable algorithm for risk-sensitive robust learning generalised to the point-wise distribution with stochastic search. We conduct experiments on the MNIST, CIFAR10 and CIFAR-100 datasets, and the results validate the effectiveness of our proposed methods on building risk-averse models. These models are robust to the majority of perturbations, and achieve superior performance compared to the alternative state-of-the-art robust learning methods.

2 RelatedWork

Robust learning is an an emerging research topic with various attack and defense methods being proposed. We now discuss at a high level some of the major approaches.

Empirical Adversarial Training. A widespread empirical defence approach, empirical adversarial training [Wang et al., 2021; Jin et al., 2022; Zhang et al., 2020a; Gowal et al., 2019; Balunovic and Vechev, 2020] endeavours to approximate the solution to a minimax problem, thereby identifying an optimal hypothesis f from a hypothesis class, F. The inner maximisation problem is frequently approximated by empirical adversarial attacks, which include gradient descentbased methods such as FGSM [Wong et al., 2020] and PGD [Madry et al., 2018]. The outer minimisation problem, on the other hand, optimises model parameters using gradient descent-based optimisers, similar to conventional training frameworks. Adversarially trained neural networks, despite demonstrating empirical resilience to adversarial attacks, remain susceptible to advanced attack methodologies due to the lack of verifiable theoretical guarantees [Tjeng et al., 2019].

Certified Adversarial Training. Certified adversarial training focuses on enhancing models' provable robust accuracy, validated by robustness verifiers [Balunovic and Vechev, 2020; Croce and Hein, 2020; Croce et al., 2019; Gowal et al., 2019; Zhang et al., 2020a; Fan and Li, 2021]. This technique minimises an upper bound of the loss across all perturbations, as opposed to training with adversarial examples. Certified bounds can be derived from the dual optimisation problem, or via linear relaxation [Balunovic and Vechev, 2020; Croce and Hein, 2020; Mirman et al., 2018; Croce et al., 2019; Zhang et al., 2020a], and interval bound propagation (IBP) [Zhang et al., 2020a; Lyu et al., 2021; Zhang et al., 2021; Shi et al., 2021]. An emerging branch of robust defence focuses on probability-based defence methods, which we now briefly discuss.

Probability-based Defense. A popular implementation of this technique is randomised-smoothing approaches, as methods using this are capable of providing probabilistic robustness guarantees. Several works [Cohen et al., 2019; L´ecuyer et al., 2019; Zhai et al., 2020; Salman et al., 2019; Awasthi et al., 2020] explore robust training approaches for robust randomised models. For instance, one can augment the training dataset with noise [Cohen et al., 2019; L´ecuyer et al., 2019] - an augmentation approach that has proven effective in practice and is currently one of the most widely-used training approaches under probabilistic robustness settings. Very recently, Robey et al. 2022 establish a probabilistically robust learning paradigm capable of balancing accuracy and robustness by enforcing robustness for the majority of perturbations, rather than all. Li et al. 2023 propose tilted empirical risk minimisation (TERM) to use exponential tilting to flexibly tune the tradeoff between averageloss and worst-loss.

We introduce a risk-averse training framework aimed at improving the probabilistic robustness under an uncertain perturbing distribution. Our method differs from current works that only focus on tail samples, EVaR considers all samples across the entire distribution, leading to a more comprehensive risk assessment (see Sec 4.1); instead of considering a specific predefined perturbation distribution, e.g. the uniform or normal distribution, we adopt an adaptive stochastic search to identify worst-case perturbation distributions, providing a practical and feasible closed-form solution.

3 Preliminaries

Supervised Learning. In classification tasks with K categories, we consider data sourced from an unknown distribution D, encompassing feature-label pairs (x, y), where x ∈ X = Rn are n-dimensional instances, and y ∈ Y = {1, . . . ,K} are the associated labels. The goal of supervised learning is to identify an optimal hypothesis f ∗ : X → Y from a class of hypotheses F, typically comprising models parameterised by θ ∈ Θ. This hypothesis can be obtained by minimising the expected risk associated with f , as defined:

where, L : Y × Y → R+ is a loss function. Ordinarily, as the distribution D is unknown, calculating the objective in Eq. 1 is infeasible. However, given a training set of N distinct i.i.d. samples {(x1, y1), . . . , (xN, yN)} drawn from D, the objective can be approximated using empirical risk, the average loss over samples:

Adversarial Training. Adversarial training aims to generate hypotheses f ∈ F that remain resilient to input perturbations. Specifically, if a hypothesis f correctly classifies an input x, it should also correctly classify a slightly adjusted input ˜x = x + δ. Here hypotheses are developed using the essential supremum of the objective function across the perturbation space, reflecting the worst-case scenario:

where ∆ ⊂ Rn denotes a set of "imperceptible" perturbations, e.g., ∆ = {δ ∈ Rn : ∥δ∥∞ ≤ ϵ}. Eq. 1 can be considered a special case of Eq. 3 when ϵ = 0. This minimax optimisation is typically approached in two stages: first, the inner maximisation is approximated to generate adversarial examples, followed by optimising the neural networks' parameters based on these adversarial examples. For instance, PGD approximately solves the inner maximisation involving multiple gradient steps, i.e., δt+1 i = Π (δti + α · sign (∇xL (fθ (xi + δti ) , yi ))), where δti is the adversarial perturbation at the t-th step, Π(·) is the projection function, and α is a small step size. In essence, δti always converges to a local optima influenced by the initialisation δ0i .

Distributional Adversarial Training. Adversarial distributional training [Dong et al., 2020] has been proposed to defend against perturbations of the captured distribution surrounding samples. Here, adversarial perturbations around each input sample xi follow the corresponding distribution p(δ) within ∆. This method can expressed as a distributionbased minimax optimisation problem:

where P = {p : supp(p) ⊆ ∆} denotes a set of distributions with support within the ∆ neighbourhood of natural examples.

Probabilistically Robust Learning. The goal of probabilistically Robust Learning is to provide robustness against the majority of perturbations, with a tolerance for a small proportion of perturbations in regions of negligible volume in the perturbation space ∆. Consequently, the objective here is to upper bound the loss function for a proportion 1 - γ of the mass of the perturbation space ∆, not for all δ ∈ ∆. Thus the following provides a definition of the associated operator introduced in [Robey et al., 2022]. This function operates on the loss random variable L(f(x + δ), y) and yields the value r for which the loss does not exceed r for a proportion 1 - γ of the perturbation space.

Definition 1. Let L(f(x + δ), y) denote a random variable representing the loss of model f for input x perturbed by δ and true label y. Let ∆ be the perturbation space with a defined measure. Then, the γ-ess sup operator (or γ-essential supremum) is then formulated as an optimisation problem, i.e., γ-ess sup = minr∈R Pδ∼∆[L(f (x+δ), y) ≤ r] > 1-γ, where γ is a small positive value that represents a tolerable level of probabilistic robustness.

Here, this definition frames γ-ess sup as a tool that quantifies the robustness of model f with respect to the proportion γ when accounting for losses caused by perturbations. The objective function for probabilistically robust learning, which focuses on the misclassification probability across a perturbation distribution, can be expressed as:

Adaptive Stochastic Search. The adaptive stochastic search method, proposed by Zhou and Hu 2014, is designed to solve a general maximisation problem: z∗ ∈ arg max z∈Z G(z), where Z ⊆ Rn is a nonempty compact set in Rn , and G : Z → R is a deterministic real-valued function. Traditional optimisation methods often falter when the objective function G(·) exhibits non-convex, discontinuous, and non-differentiable traits. This method overcomes such hurdles by stochastically approximating the function. A solution z is sampled from a probability distribution p(z; η) from the exponential family, parameterised by η. The reformulated

problem becomes:

This reformulation, thanks to its probabilistic nature, exhibits properties conducive to optimisation. The algorithmic implementation can be further simplified by introducing a continuous, non-decreasing shape function, S(·) : R → R+, ensuring the optimal solution remains unaffected. Thus, the final problem becomes: η∗ = argη max ∫ S(G(z))p(z; η)dz = argη max Eη[S(G(z))].

To tackle this optimisation problem, candidate solutions, z, are drawn from p(z; η) within the solution space Z. Afterwards, a gradient ascent method can be applied to Eq. 6 to update the parameter η. Depending on the chosen probability distribution for sampling z, a closed-form solution for the gradient of the above objective function with respect to η may be available. (The derivation is included in the Appendix A.1.)

4 Methodology

This section proposes a novel and generalisable framework: Probabilistic Risk-averse Robust Learning with Stochastic Search (PRASS). Its primary goal is to characterise the perturbation distribution and train a risk-averse model via mitigating the Entropic Value-at-Risk (EVaR) risk over the captured worst-case perturbation distribution. This particular risk exhibits convexity, facilitating a straightforward and efficient resolution for risk-averse robust learning. Additionally, EVaR risk capitalises on the entire samples across sampling distribution, as opposed to solely relying on tail samples.

4.1 Probabilistic Risk-averse Robust Learning

Optimising the objective function for probabilistically robust learning as per Eq. 5 is very challenging. This is due to the γ-essential supremum operator associated with random perturbations being non-convex, non-smooth, and highly stochastic. Note that the γ-essential supremum is alternatively termed Value-at-Risk (VaR) [Robey et al., 2022], especially within the risk-aware control domain [Majumdar and Pavone, 2017; Ahmadi et al., 2022].

Definition 2. Given a random variable r and a function g[·] yielding a scalar output (i.e., the real scalar value correlated with the loss function), and considering a risk level γ ∈ [0, 1], the Value-at-Risk, denoted as V aRγ(g(r)), is the infimum over ζ ∈ R where g(r) situates beneath ζ with a probability not less than 1 - γ. Formally, this can be expressed as V aRγ(g(r)) = infζ∈R ζ : P [g(r) ≤ ζ] ≥ 1 - γ.

Unfortunately, the tractability of VaR is hampered by several constraints, such as the non-coherent risk measure and non-convex function. Thus, VaR is frequently replaced with Conditional Value-at-Risk (CVaR), which calculates the expected value of those losses that lie in a tail region where the threshold V aRγ is exceeded.

Definition 3. The Conditional-Values-at-Risk with a risk level γ ∈ (0, 1], denoted as CV aRγ(g(r)), is the expected value of f within the tail distribution that equals or surpasses the threshold V aRγ, i.e., CV aRγ(g(r)) = E[g(r) | g(r) ≥ V aRγ(g(r))] = infζ∈R {ζ + E[g(r)-ζ]+ 1-γ }, where [·] = max{0, ·}.

The inf operator inside the CV aRγ(g(r)) is convex w.r.t. ζ; this property ensures that the optimisation problem to estimate CVaR can be efficiently solved, since [·] is increasing and convex [Rockafellar et al., 2000]. Despite its computational advantage and valuable insights into the extreme events (in the tail of loss distribution), it does so at the expense of overlooking the remainder of the loss distribution. Thus we employ Entropic Value-at-Risk (EVaR) that is the tightest upper bound derived from the Chernoff inequality for the VaR.

Definition 4. The Entropic Values-at-Risk with risk level γ ∈ (0, 1] denoted as EV aRγ(g(r)) is defined as the infimum over ζ > 0 of the Chernoff bound for g(r) w.r.t. random

While EVaR serves as the tightest upper bound derived from the Chernoff inequality for the γ-essential supremum, it also incorporates the entirety of the loss distribution, not just the tail. The definition of EVaR uses the exponential moment of the entire loss function weighted by a scalar ζ. This implies that every sample from the loss distribution has a role in shaping the EVaR value, thereby ensuring a more comprehensive view of the risks embedded in the distribution.

Proposition 1. The EVaR is an upper bound for the γ-ess sup and CVaR with the same risk γ, that is, γ-ess supg(r) ≤ CV aR1-γ(g(r)) ≤ EV aR1-γ(g(r)), with equality when γ = 0 or γ = 1. Moreover, E[g(r)] ≤ EV aR1-γ(g(r)) ≤ ess sup(g(r)), where EV aR0(g(r)) = E[g(r)] and limγ→0 EV aR1-γ(g(r)) = ess sup(g(r)).

The relationship between the three risk measures is illustrated in Fig. 1. As Proposition 1 shows, EVaR risk is a more risk-averse risk metric than both γ-ess sup and CVaR, and can enhance the overall robustness of the models with a balance between capturing average and worst-case scenarios. While γ-ess sup and CVaR hone in exclusively on the tail distribution-essentially the extreme scenarios-EVaR integrates the full breadth of the loss distribution, facilitating a more rounded understanding of risks.

Thus, our proposed risk-averse robust learning paradigm is to optimise the upper bound (i.e., EVaR) of the objective in Eq. 5 to fortify models against probabilistic uncertainties. Mathematically, the new paradigm replaces the inner optimisation in Eq. 4 with EVaR, the objective function is:

where γ is user-predefined risk-averse parameter.

Similar to adversarial training, the proposed learning paradigm constitutes a composite optimisation problem, i.e., an inner minimisation over ζ to compute EVaR and an outer minimisation to update the hypothesis parameters. However, different to the inner maximisation in adversarial training, the inner minimisation in this case is convex w.r.t. ζ regardless of other variables, including the parameters of the hypothesis. Consequently, this property allows the gradient of the inner objective to be obtained in a closed form.

4.2 Theoretical Generalisation Analysis

The inferior generalisation properties of adversarial training in high-dimensional spaces fundamentally constrain its applicability: independent of the solvability of the optimisation procedure during training, it lacks guarantees (or even a likelihood) that the risk-sensitive classifier will exhibit robustness at test-time. We now demonstrate that our risk metric is not subject to this limitation. As shown by Shalev-Shwartz and Ben-David 2014, the generalisation error of a learning algorithm can be probabilistically upper-bounded using statistical learning theory, employing concepts of complexity on the admissible set of hypotheses and loss function. Here we introduce the empirical Rademacher complexity below for a hypothesis class.

Definition 5. The empirical Rademacher complexity for a hypothesis class F : X → Y and sample set S = {x1, . . . , xN} is

where σ1, . . . , σN are independent Rademacher random variables, with the value -1 or +1, each with probability 1/2. Intuitively, it measures the complexity of the class by determining how many different ways functions f ∈ F can classify the sample S.

Subsequently, given a neural network hypothesis class F and loss function class LF ≜ {(x, y) → L(x, y, f) : f ∈ F}, we can bound the generalisation error of a classifier using the following theorem [Mohri et al., 2018]:

Lemma 1. Suppose ∀x, y, f : L(x, y, f) ∈ [0, c]. For m ∈ {1, . . . ,M}. Additionally, assume further that the samples S = {(x1, y1), . . . , (xN, yN)} are i.i.d. from a distribution D. Then for any δ ∈ (0, 1), with probability no less than 1-δ the following holds for all f ∈ F:

This bound is probabilistic, data-dependent and uniform over all functions in the hypothesis class, f ∈ F, meaning it holds for all such functions, including those trained on the dataset S. Informally, the empirical risk on the training dataset will be "close" to the true risk (i.e., the difference bounded by the term on the right hand side) with high probability (in the formal sense).

To take advantage of this bound, we need to be able to compute RadS(LF). The empirical Rademacher complexity (see Eq. 8) of the hypothesis class RadS(F) can be upper bounded [Bartlett et al., 2017; Yin et al., 2019] by an expression O(log (dmax)) that depends on the logarithm of dmax (i.e., the maximum number of nodes in a single layer). Thus we simply need to relate RadS(LF) to RadS(F).

Lemma 2 (Talagrand's contraction principal). Let g be an L-Lipschitz continuous function, and F is a function class. Then, Rad((g ◦ F)) ≤ L · Rad(F).

Considering the natural risk, if L(f(x), y) is Γ-Lipschitz in the first argument, we can use Lemma 2. The lemma gives that RadS (LF) ≤ ΓRadS (F). Thus, substituting this inequality into Eq. 9, we have:

such that our generalisation error bound scales as O (log (dmax)) (as RadS(F) is O (log (dmax))).

We now introduce an analogous result for our probabilistic risk in Theorem 1, and the detailed proof can be founded in Appendix B. In this case, the empirical risk we will use is the Monte Carlo estimate since this is what we actually compute.

Theorem 1. Suppose ∀x, y, f : L(x, y, f) ∈ [0, c]. For m ∈ {1, . . . ,M}, define S′m = {(x′1,m, y1 ) , . . . , (x′N,m, yN )}, such that it contains the m-th perturbed point x′ = x+δ from each of the N original inputs. Then for any δ ∈ (0, 1), with probability at least 1 - δ the following holds for all f ∈ F:

where

We can see, the generalisation error is upper bounded by an expression that varies as O(log (dmax)). In contrast, for the adversarial risk, defined as supδ∈∆L(f(x + δ), y), its empirical Rademacher complexity is lower bounded by an expression containing explicit dependence on √din, where din is the dimension of the input layer of network [Yin et al., 2019]. While this lower bound does not allow us to directly bound the generalisation error using Eq. 9 or compare it with the upper bound of the complexity of the EVaR objective, it does suggest that in high dimensions the adversarial generalisation error can be much greater than the natural and the proposed EVaR risk gaps. This indicates it will typically be difficult to train networks that are adversarially robust at test time for high-dimensional datasets. Our experiments will show that with networks trained with the proposed objective it may be easier to get narrower train-test gaps for high-dimensional datasets compared with an adversarial objective.

4.3 Generalising to Distributional Robustness via Stochastic Search

To achieve more risk sensitivity, we compute the EVaR risk based on the worst-case scenarios for perturbation distributions. To this end, we enable it to generalise to distribution robustness with adaption. These perturbations are not static; they are constantly adjusted before any model updates occur in the backward pass. The uncertainty variable (η in Eq. 6) can be adapted using a rule such as gradient descent w.r.t. the inner-loop objective function for a fixed number of steps.

Assuming that p(x; δ) is the continuous probability density around the input data x, and its support is contained in ∆, then we rewrite the risk-averse robust learning framework (Eq. 7) in the form of distributional robustness by minimising the Entropic-VaR of the loss function L(·) subject to perturbation δ following uncertain perturbation distributions as:

Equivalently,

Intuitively, we propose using an adaptive stochastic search to address the inherent stochastic nature of perturbations. The EVaR containing perturbation δ in the inner loop is computed with respect to uncertainty distributions, and G(z) can be of any functional form, i.e., loss function of a neural network. We then define a sampling distribution for these perturbations using the exponential family of densities defined below.

Definition 6. A family {p(z; η) : η ∈ Θ} is an exponential family of densities if it satisfies:

where T(z) = [T1(z), T2(z), . . . , Td(z)]⊤ is the vector of sufficient statistics, ϕ(η) = ln {∫ exp (η⊤T (z)) dz} is a normalisation factor that ensures p(z; η) to be a pdf. Θ = {η : |ϕ(η)| < ∞} is the natural parameter space with a nonempty interior and η = [η1, η2, . . . , ηd]⊤ is the vector of natural parameters.

The inner loop in Eq. 11 is performed by maximising EV aR1-γ (L(f (x+δ), y); δ) with p(δ) = p(δ, η) ∈ P with respect to the natural parameters:

With adaptive stochastic search, a shape function is always introduced S : R → R+ which allows for different weighing schemes of the cost levels, leading to different optimisation behaviours [Ollivier et al., 2017]. Then, the problem is transformed into:

where S(x) is non-decreasing in x and bounded from above and below for bounded x (see Appendix C.1). Finally, we can obtain a scale-free gradient in a closed form (see Appendix A.2):

With the analytical expression of the gradient we are ready to use a gradient-based approach to update the parameter η on an epoch basis during the training process. In practice, we approximate the inner expectation w.r.t. updated perturbation distribution in Eq. 15 with M Monte Carlo (MC) samples, and perform K steps of gradient ascent on δ to solve the inner problem. After obtaining the optimal parameters of the perturbation distribution, we use the adversarial distribution to update model parameters θ through risk-averse robust learning. While any probability function of the exponential family will work, in this study we sample perturbations from a Gaussian distribution. The full algorithm is summarised in Algorithm 1 in Appendix C.

5 Experiments

Experiment Setup. We conduct an extensive evaluation of the risk-averse robust learning method on three datasets: MNIST, CIFAR-10 and CIFAR-100. For MNIST, we adopt a ReLU network architecture with two convolutional layers, while for CIFAR-10 and CIFAR-100, we utilise an 18layer residual network architecture. Moreover, the uncertainty set under consideration is a perturbation set, defined as ∆ = {δ ∈ Rd : ∥δ∥∞ ≤ ϵ}, situated within a Gaussian distribution set p(δ) ∈ P. We set ϵ = 0.3 for MNIST and ϵ = 8/255 for CIFAR-10 and CIFAR-100. Full details are provided in Appendix C. All the experiments are executed on a system with a 32-Core AMD EPYC 7452 CPU and an NVIDIA A100 40GB GPU.

EvaluationMetrics. To evaluate the performance of the algorithms, we record the natural accuracy on the test set and the empirical robust accuracies of each algorithm. The latter is quantified on perturbed samples for each dataset, assessed via two distinct ways: (i) Empirical Augmented Accuracy: For each data point, we randomly draw 200 perturbations around inputs, subsequently recording the average accuracy over these perturbed samples, denoted as "Aug. Acc.". (ii) Empirical Probabilistic Accuracy: To assess the models' probabilistic accuracy, we compute the empirical probabilistic accuracy, signifying the proportion of samples that are empirically probabilistically robust with a tolerance level γ. The mathematical expression is defined as Prob. Acc.(γ) = 1[Pδ∼∆[f (x+δ) ̸= y] < γ] [Robey et al., 2022] and, notably, the same number of perturbations are utilised to calculate the empirical probabilistic accuracy, denoted as "Prob. Acc.(γ)".

5.1 Empirical Generalisation Error

As discussed, training neural networks to achieve high testtime adversarial accuracy on high-dimensional datasets is challenging. However, as implied by our analysis in Section 4.2, the generalisation gap for our EVaR based method should be closer to that obtained when using empirical risk. To empirically investigate this, we use four different methods to train networks: standard training (i.e., ERM as defined in Eq. 2), adversarial training techniques (i.e., FGSM and PGD with 7 gradient steps) and the proposed training approaches (i.e., PRASS) with tolerance levels γ = 0.01 and γ = 0.1. In accordance with these methods, we evaluate the performance of the networks using natural accuracy, the previously mentioned probabilistic robust accuracy, and adversarial accuracy by using a 20-step PGD adversary.

The results are presented in Table 1. These findings corroborate our theoretical analysis in Section 4.2: 1) For the low-dimensional MNIST dataset, all training methodologies exhibit exceptional generalisation performance, with traintest generalisation gaps being quite small, around 3% for all evaluation metrics, i.e., clean accuracy, adversarial accuracy, and probabilistic accuracy. 2) For the high-dimensional CIFAR-10 dataset, ERM and PRASS show fairly small generalisation gaps, about 8%. Conversely, adversarial training approaches, particularly on the PGD-trained network, show a markedly larger gap, nearing 30%. 3) For CIFAR-100, which is even more challenging due to more classes, the generalisation gap increases across methods. Still, PRASS outperforms the PGD-trained network by a significant margin (around 20%). This underscores a clear limitation of adversarial training compared to our proposed risk-averse training.

5.2 Effectiveness of PRASS

Baselines & Evaluation Metrics. To validate the efficacy of PRASS, we consider baselines including ERM, TRADES [Zhang et al., 2019], DALE [Robey et al., 2021], TERM [Li et al., 2023] and PRL [Robey et al., 2022]. Besides the aforementioned metric, we also adopt a certified probabilistically robust accuracy with a tolerance level γ and confidence level δ = 10-10 , computed with PRoA [Zhang et al., 2022], denoted as "Cert. Acc.". Note that we consider probabilistically robust accuracy, i.e., Cert. Acc.(γ) and Prob. Acc.(γ), as the key metrics for assessing the level of probabilistic robustness that models trained with various algorithms can achieve. The highest and second highest performances are highlighted in Bold and Underlined. The most critical metric is emphasised in blue.

As shown in Tables 2, 3 & 4, PRASS shows significant improvements in risk reduction for random perturbations. For both empirical and certifiable probabilistic metrics, our method consistently showcases the best test-set performance across all risk levels. This improvement is most conspicuous on CIFAR-10 and CIFAR-100, the higher-dimensional datasets, and, to some degree, it aligns with our generalisation result. Another noteworthy observation is that our PRASS method does not suffer the same limitation as adversarial training does; in fact, PRASS's natural accuracy surpasses that of ERM by a margin of around 1% and 50%, respectively on these data. As expected, PRASS does not perform well on adversarial accuracy, given that its focus lies on risk aversion as opposed to explicit adversaries. Finally, we observe that all adversarial training approaches do not contribute positively to improving probabilistic robustness, despite achieving consistently commendable test-set adversarial performance. This indicates that adversarial risk is neither a relevant nor optimal objective for addressing risk reversion.

6 Conclusion

In this paper, motivated by scenarios where models' threats are not adversarially generated but arise probabilistically, we introduced a new framework called risk-aversion robust learning. Instead of focusing on the worst-case robustness, in this framework, robustness is enforced with high probability over perturbations by updating the perturbation distribution and minimising the upper bound of the γ-essential supremum across updated distributions. Our exploration of the practical and theoretical aspects of this framework led to a new algorithm that achieves superior generalisation performance compared to adversarial training and effectively enforces probabilistic robustness in practice.