DDDM: A Brain-Inspired Framework for Robust Classification

1 Introduction

Deep learning has shown outstanding performance in many areas, including image, audio , and text classification. Nevertheless, deep learning lacks robustness; that is, deep artificial neural networks are susceptible to manipulations of inputs. Small perturbations on inputs that are imperceptible to humans may result in significant differences in outputs of deep neural networks [Goodfellow et al., 2015]. Such susceptibility is not only of theoretical interest but also represents a severe practical problem, which causes security issues in real-world applications, including face recognition and autonomous driving.

∗Co-first authors

†Corresponding authors

Recently, a huge amount of effort has been dedicated to addressing this issue. Adversarial training [Madry et al., 2019] is considered to be the most promising. It adds adversarial samples to the training set and automatically lets the network model adapt to them through learning. Despite these efforts, current defenses against adversarial attacks are still far from satisfactory. Most defenses only target one or a few types of attacks and are often restricted to particular tasks, such as image classification. Consequently, it is unclear whether they can be applied to other kinds of attacks or tasks.

In contrast, humans and animals are much less vulnerable. Even though time-limited human subjects can be misled by deliberately crafted images, they successfully restore accuracy by spending more time facing these uncertain and noisy inputs [Elsayed et al., 2018]. The more uncertain the input is, the longer the brain takes to make a decision. This ability of trading decision speed for accuracy has been well studied in humans and animals by researchers in neuroscience and psychology [Ratcliff and Rouder, 1998; Roitman and Shadlen, 2002]. In particular, the celebrated Drift-Diffusion Model (DDM) and its variants stand as one of the most successful models that account for the decisionmaking process in humans and animals. The DDM describes a sequential sampling process in which signals with both noises from the external sources and the internal neuronal noises are accumulated as evidence for decisions. The decision is made once the accumulated evidence reaches a predetermined threshold. A high threshold leads to accurate but slow decisions, while a low threshold results in fast but inaccurate decisions. Neuronal activities in decision-makingrelated brain areas exhibit response patterns described by the DDM [Roitman and Shadlen, 2002].

The ability to trade speed for accuracy characterizes one of the most striking differences between biological and artificial neural networks. Enhancing artificial neural networks with a dynamic inference process similar to those in human and animal brains would potentially improve their robustness. Guided by this intuition, we propose the Dropout-based Drift-Diffusion Model (DDDM) as a general framework for enhancing the robustness of arbitrary neural networks. Depicted in Figure 1, in contrast to adversarial training, which adds noises to inputs during the training phase, DDDM adds noises to neural networks via random dropouts during the test phase, resulting in multiple stochastic copies of the original one. Then, DDDM employs a DDM-like evidence accumulation mechanism to make the decision based on these outputs.

1 The full version of our paper: https://github.com/zhouyilab/ DDDM/blob/master/DDDM-IJCAI22.pdf; Implementation: https: //github.com/XiYuan68/DDDM.

We conduct experiments on three types of datasets, including MNIST [Lecun et al., 1998] and CIFAR10 [Krizhevsky, 2009] for image classification, the SpeechCommands [Warden, 2018] for audio classification, and the IMDB dataset [Maas et al., 2011] for text classification. We compare the networks' performance with and without the DDDM against a variety of adversarial attacks, including both the white-box and the black-box ones. The experimental results show that the DDDM improves the robustness of the network, providing defense against all adversarial attacks tested on all three datasets.

2 RelatedWork

Adversarial training (AT) [Goodfellow et al., 2015; Madry et al., 2019] is one of the most popular adversarial defense methods. It augments the clean data with adversarial ones during the trianing process. There are works that try to improve robustness by introducing additional noise in their models. For example, [Guo et al., 2018] applied transformation on images before feeding the network, including image quilting, total variance minimization, JPEG compression, and bit-depth reduction. Similar to our work, [Dhillon et al., 2018] randomly dropped neurons with a weighted distribution. Note that all the defense methods mentioned above focus on either filtering out adversarial perturbations or creating obfuscated gradients [Athalye et al., 2018], leaving room for more sophisticated attacks. On the contrary, our framework injects randomness into the network model during the testing phase, and the resulting stochastic predictions are agnostic to the type of adversarial attacks. Furthermore, a temporal evidence accumulation mechanism is employed to offset the accuracy reduction due to the added randomness and retain a good performance with robustness.

In neuroscience, recent advances in the study of perceptual decision-making have started to reveal the underlying neural mechanism [Gold and Shadlen, 2007]. The random-dot motion (RDM) direction discrimination task is one of the most popular tasks for studying decision-making with noisy evidence [Roitman and Shadlen, 2002]. In this task, subjects are asked to judge whether a patch of noisy dots moves toward left or right. The subjects' decision accuracies and response times can be well described by the DDM [Ratcliff and McKoon, 2008]. Furthermore, it was demonstrated that neurons in the posterior parietal cortex increased their firing when evidence toward their favored choice was being accumulated [Roitman and Shadlen, 2002]. The response increase correlated with the quality of evidence, which may be quantified as the log-likelihood ratio [Gold and Shadlen, 2007; Yang and Shadlen, 2007], and reflected the evidence accumulation process. Additional studies extended DDM into probabilistic inferences based on images and identified a prefrontal-parietal circuitry that carries out evidence accumulation process in probabilistic inferences [Yang and Shadlen, 2007]. Response patterns that reflect evidence accumulation were also found in many other brain regions, including multiple regions in the prefrontal cortex and the basal ganglia, suggesting evidence accumulation is a universal decision-making mechanism employed by the brain [Gold and Shadlen, 2007]. DDM and the related sequential sampling models in neuroscience describe the brain's solutions for optimizing decisionmaking with noisy inputs. However, the DDM has not been used together with deep neural networks in the machine learning field and has not been considered as a defense mechanism against adversarial attacks.

3 Model Description

3.1 Test-Phase Dropout

Dropout was initially proposed by [Hinton et al., 2012] to address the problem of overfitting. In this paper, we use it to introduce randomness in the system. It simulates the internal neuronal noise in synaptic transmissions [Rusakov et al., 2020]. With dropouts applied temporally, the original network model effectively extends to an ensemble of stochastic copies, whose outputs serves as the evidence. Let ha,b be an arbitrary deep neural network with one or several dropout layers. The dropout layers share the same dropout rate. Subscript a denotes the dropout rate used for model training and validation, and b denotes the dropout rate at the test phase. We refer to ha,b as the dropout classifier in the following context to emphasize the involvement of the test-phase dropout mechanism.

3.2 Drift-DiffusionModel

In a typical two-choice scenario, the canonical DriftDiffusion Model (DDM) (see Figure 2) describes decisionmaking as a process in which evidence that is noisy and fluctuates over time (momentary evidence) is accumulated toward a preset threshold A. Mathematically, DDM corresponds to the following stochastic differential equation

where X is known as the decision variable representing the cumulated evidence, and µ is the drift or the mean of the noisy momentary evidence. The Gaussian process ξ(t), with mean 0 and variance σ2 , captures the uncertainty in the evidence. Coefficient τ is the characteristic time scale. The decision variable X evolves until reaching one of the two thresholds ±A at which the corresponding decision is made. These thresholds correspond to the choice options, and the time needed for the accumulated evidence to reach a particular threshold reflects the response time (RT).

The canonical DDM can be extended to decisions with more than two alternatives. Equation (1) can be derived from a set of stochastic differential equations [Roxin, 2019]. Following this route, it is straightforward to increase the number of choices by extending the number of differential equations. Another route to extend the DDM to the multi-choice domain models the decision-making as a Bayesian multiple sequential probability ratio test (MSPRT). It has been demonstrated the two routes are equivalent [Roxin, 2019].

For our purpose to integrate the DDM with the test-phase dropout classifier, we use the Bayesian MSPRT route. Suppose there are n alternatives, and zi(t) denotes the instantaneous evidence for choice i. zi(t) is drawn from a Gaussian distribution with mean Mi and variance σ2 . Let O(T) := {zi(t) : 1 ≤ i ≤ n, 1 ≤ t ≤ T} be the collection of observations up to a time T , and Hi denote the hypothesis that choice i has the largest mean. Within the Bayesian MSPRT framework, one seeks to make a decision based on the posterior Pr(Hi|O(T )). With defining

where yi(t) := ∫ t 0 dt′ zi(t′ ) and Xk is the decision variable for the kth class, measuring how likely the original input belongs to class k given the accumulated evidence. It can be shown that Xk satisfies a stochastic differential equation of the same type as Equation (1). We refer the interested readers to [Roxin, 2019] for a complete deduction.

3.3 Dropout-based Drift-DiffusionModel

There are two components in the design of the Dropout-based Drift-Diffusion model (DDDM). The first component turns the static one-shot inference process into a dynamic, noisy inference process. The noisy predictions from the stochastic copies of neural networks simulate the series of temporal signals in the biological brain. Presently, we implement the dynamic inference by introducing the test-phase dropout, which will randomly disturb the loss landscape and potentially make adversarial attacks harder to succeed. This results in a dropout classifier whose outputs are subject to further processing.

For the second component, we take the noisy outputs from the test-phase dropout classifier as evidence and apply DDM with the Bayesian MSPRT implementation. More precisely, for a dropout classifier ha,b, let C be the number of categories and x be an input. We arrange L predictions to form a trial

where pij is the i-th predicted confidence of x belonging to label j. Let Ti denote the i-th row vector in matrix T, we compute the posterior at time step t

Let RT ≤ L be the smallest time step at which there exists a label winner satisfying P(winner,RT) ≥ A. We say that x belongs to the label winner with a response time RT . If the threshold is never surpassed during the whole trial, we will force a decision by assigning the label winner to be the one with the largest decision variable at time step L, i.e., P(winner,L) = max(P(1,L), P(2,L), . . . , P(C,L)). In practice, we apply multiple trials and an appropriate decision threshold A is chosen to offset the noise introduced by dropouts and ensure overall accuracy. To compute the posterior P(i, j), we apply Bayes theorem and assume equal priors for all hypotheses Hi. The problem is then reduced to computing the likelihoods Pr({T1, . . . , Tt}|Hi), which are approximately evaluated over the clean training data. The detailed description of our implementation can be found in the supplementary material.

In the following context, we will denote the dropout classifier equipped with DDM as Ha,b and refer to it as DDDM for simplicity.

4 Experiments and Results

The primary goal of the present experiments is to evaluate the robustness of our DDDMmodel. Our experiments cover three different modalities of data, i.e., image, audio, and text. Under all types of inputs, the DDDM is demonstrated to be effective in defending adversarial attacks.

For all the tasks, we run experiments over different combinations of dropout rates (a, b), both taking values from the set {0, 0.2, 0.4, 0.6, 0.8}. Therefore, twenty-five classifiers are tested in each case. In implementing DDM, we draw 100 predictions from each classifier on each generated adversarial example. Then, 10 trials of the length L = 25 are randomly sampled from those predictions. These trials are fed into the evidence accumulation mechanism with a decision threshold A = 0.99 for the final prediction.

4.1 Images Classification

MNIST Dataset

In the experiments, we adopt the same network architecture as in [Madry et al., 2019] with an additional dropout layer that follows each layer except the input/output ones.

To comprehensively evaluate the effectiveness of DDDM in defending adversarial attacks, we considered four whitebox attacks and four black-box attacks as listed below: the Fast Gradient Sign Method (FGSM) attack [Goodfellow et al., 2015], the Projected Gradient Descent (PGD) attack [Madry et al., 2019], the L2 Carlini and Wagner (L2 C&W) attack [Carlini and Wagner, 2017], the L2 DeepFool attack [Moosavi-Dezfooli et al., 2016], the Salt and Pepper attack, the L∞ uniform noise attack [Rauber and Bethge, 2020], the Spatial attack [Engstrom et al., 2019] and the Square attack [Andriushchenko et al., 2020]. The details of attack settings can be found in the supplementary material.

In Table 1, we compare the optimal DDDMs (H∗ a,b ) with the optimal dropout classifiers (h∗a,b ) on individual attacks and the clean data. The undefended model (h0,0) is included as the baseline. To demonstrate the effectiveness of DDDM under the combination of all attacks, we also include the results for Ha∗ ,b∗ and ha∗ ,b∗, i.e., the DDDM and the dropout classifier with a single optimal dropout rate pair (a∗ , b∗ ) across all the attacks. (a∗ , b∗ ) is determined with the criterion that the corresponding Ha∗ ,b∗ should try its best to avoid sacrificing the clean accuracy and, at the same time, provide as much defense as possible across different attacks. The results indicate that the test-phase dropouts effectively neutralize the adversarial perturbations against all attacks in our experiments, and the DDM retains the accuracy through evidence accumulation. Furthermore, Ha∗ ,b∗ outperforms ha∗ ,b∗ as well as h0,0 on all the cases.

Trade-off between Response Time and Accuracy. Animals and humans spend more effort in making difficult decisions, and the DDDM exhibits the same behavior. While the randomness in the test-phase dropout classifier neutralizes the adversarial perturbations, the DDM helps retain a decent accuracy by spending more time in inference. To demonstrate this trade-off, we plot in Figure 3 the accuracies and response times with respect to different adversarial perturbation sizes, comparing DDDM to the dropout classifier as well as the undefended baseline model. The response time is measured by the number of forward passes. The accuracy of the baseline model drops rapidly as the perturbation size increase. On the other hand, both the DDDM and the test-phase dropout classifier retain relatively high accuracy even when the perturbation is large. For small perturbations, the test-phase dropout classifier's accuracy curve lies well below the baseline model's curve. In contrast, the performance of DDDM dominates on almost the whole perturbation range except for near 0. Finally, the response time increases monotonically with perturbation level, demonstrating a trade for accuracy with time.

CIFAR10 Dataset

We further carry out experiments on the CIFAR10 [Krizhevsky, 2009] dataset to challenge our approach under a more realistic scenario. To implement the test-phase dropout classifier, we use the VGG16 architecture [Simonyan and Zisserman, 2015] without batch normalization. A dropout layer is added to each of the last six convolutional layers. We will discuss more the method of adding dropout layers in the supplementary material.

The results are summarized in Table 2, which includes three effective attacks in our test: PGD, L2DF, and Spatial. Note that these attacks cover both white-box and blackbox attack categories. Again, DDDM effectively defends all the attacks with only a minor sacrifice on the clean accuracy. Compared to the MNIST case, while Ha∗ ,b∗ keeps outperforming ha∗ ,b∗, their performance gaps become much smaller. This may be attributed to the fact that under a large model such as VGG16, six dropout layers do not introduce sufficient randomness in the dropout classifier. Hence, DDM does not work at its full power.

4.2 Audio Classification

To evaluate the cross-domain performance of DDDM, we consider the audio classification task on SpeechCommands [Warden, 2018] dataset, which contains 35 keywords and 105829 audio clips of those keywords. In the audio domain, present works focus on the classification and recognition of audio clips, which serve as the base of various applications such as voice input and voice assistants. The existing audio adversarial attacks [Carlini and Wagner, 2018; Qin et al., 2019] severely endangers the safety of such automatic speech recognition (ASR) systems.

In our test, we adopt a simplified version of the DeepSpeech2 model [Amodei et al., 2015] against the attack proposed by [Qin et al., 2019], which adds human-imperceptible perturbations to the audio waveform to mislead the model predictions. Our DeepSpeech2 model includes a Melspectrogram conversion layer, followed by one 1D convolutional layer, two LSTMs, and two fully-connected layers. Dropout is only applied once after the convolutional layer. After training, we attacked the model with the adversarial perturbation bound of 0.05.

Results in Table 3 show that DDDM providea defense against the imperceptible attack with only a minor decrease in clean accuracy. Comparing the dropout classifier ha∗ ,b∗ with the DDDM Ha∗ ,b∗, the latter show better performance under clean and adversarial case situations. We notice that the gap between ha∗ ,b∗ and Ha∗ ,b∗ is small. This is similar to the case for CIFAR10 and could be attributed to the insufficient number of dropout layers.

4.3 Text Classification

At last, we evaluate our approach on the text classification task. Natural languages consist of discrete characters and words. The process of mapping tokens to embeddings prevents directly applying attack methods from the image domain on textual classifiers. A text-specific attack methods have been developed [Li et al., 2019].

The DDDM is evaluated on the IMDB dataset [Maas et al., 2011] against the TextBugger attack [Li et al., 2019]. We use an LSTM classifier resembling the one in the TextBugger work, with adding an additional dropout layer to the preactivation outputs of each gate in the network. Words from the IMDB dataset are first converted into pre-trained GloVe embedding vectors of dimension 300, then fed to the LSTM classifier.

We summarize the experimental results in Table 4. DDDM Ha∗ ,b∗ successfully protected the classifier with a considerable improvement in performance compared to the dropout classifier ha∗ ,b∗. The robust accuracy of Ha∗ ,b∗ under the TextBugger attack approaches its clean accuracy. Moreover, we note that the clean accuracy of Ha∗ ,b∗ is higher than the one of the undefended model h0,0. This could be explained by the fact that we use the clean training data to evaluate the likelihood. We emphasize that this does not harm the effectiveness of our approach since it does not require information about the possible attacks beforehand.

5 Discussion

The current work is merely proof of the principle that combining an evidence accumulation mechanism with deep neural networks enhances robustness and provides defense against adversarial attacks. The key to the framework's success lies in the interplay between two components of the framework. The first is the mechanism to introduce noise into the system. Partly because the random noise itself degrades the performance; therefore, achieving a balance between performance and robustness poses a challenge. The second component, the evidence accumulation in the DDM, provides a mechanism to solve this problem and allows more flexibility in terms of the level of additional noise added into the system. In addition to artificially introduced noise, noise exists in many real-world situations, such as the automatic driving systems. In these cases, noise naturally presented in the input may be combined with artificially introduced noise for defense against attacks.

The attractiveness of the framework also lies in its simplicity. DDDM does not require the networks to be trained differently for any particular attacks. Instead, it is agnostic to the type of attack and its particular scheme of noise. Thus, it provides protection not only for the attacks tested here but also for other attacks. The framework also does not depend on particular types of networks. It can also be used in combination with more sophisticated networks and larger datasets.

Other than what has been mentioned above, several other directions to extend the current framework can be explored in the future. In addition to the dropouts, other ways of introducing noise into the system may help to battle perturbations. A mechanism of online adjustment of decision thresholds may help improve the speed-accuracy tradeoff in a dynamic environment. Finally, the evidence accumulation mechanism is not implemented with a neural network in the current study. However, previous studies have shown that simple recurrent networks may achieve the functionality [Wang, 2002; Wong and Wang, 2006], providing an end-to-end neural network solution for the DDDM.

6 Conclustion

Inspired by the neural mechanism of decision making, we develop a novel framework (see Figure 1) and demonstrate its effectiveness in improving the robustness of deep learning networks against adversarial attacks in different domains with a brain-like mechanism (see Tables 1 to 4). In contrast to adversarial training, the DDDM shows an advantage of sustaining the same level of clean accuracy after being attacked. By extending the one-shot classification into a temporal decision-making process based on evidence accumulation, our framework trades inference time for accuracy to achieve robustness.

To the best of our knowledge, this is the first framework to combine deep neural networks and a brain-inspired decisionmaking mechanism for defense against adversarial attacks and the first framework to provide cross-domain protection without adversarial training. We hope that our work brings the advances of neuroscience research to the attention of the artificial intelligence community and encourages researchers to take a closer look at the most powerful yet robust classifier -- our brain.

Acknowledgments

This work was supported by grants from the National Science and Technology Innovation 2030 Project of China to Yi Zhou (2021ZD0202600) and to Tianming Yang (2021ZD0203701). This work was also supported by grants from the Chinese Academy of Sciences (XDB32070100), and the Shanghai Municipal Science and Technology (2018SHZDZX05) to Tianming Yang.