Spatially Constrained Adversarial Attack Detection and Localization in the Representation Space of Optical Flow Networks

1 Introduction

Motion estimation is a key problem in computer vision. Motion in video is often represented in the form of dense optical flow fields, which specify the motion of each pixel from one frame to the next. It has diverse application areas, such as tracking [Shi and Tomasi, 1994], action recognition [Diba et al., 2017], 3D reconstruction [Agarwal et al.,

∗This work was done when Hannah Kim was an intern at IBM Research. She now works at Amazon.

†This work was done when Girmaw was at IBM Research. He now works at Microsoft. Email: gtadesse@microsoft.com

2009], video interpolation [Kim et al., 2022b], and video compression [Le Gall, 1991].

With the emergence of deep learning techniques and availability of large scale datasets, optical flow estimation performance has been significantly improved [Dosovitskiy et al., 2015; Ilg et al., 2017; Sun et al., 2018; Teed and Deng, 2020] across existing benchmarks [Butler et al., 2012; Menze and Geiger, 2015]. However, recent work [Ranjan et al., 2019; Schrodi et al., 2021; Schrodi et al., 2022] demonstrated that these networks are also vulnerable to adversarial attacks just as any other deep learning approaches [Papernot et al., 2017; Akhtar and Mian, 2018; Akinwande et al., 2020; Kim et al., 2022a], thereby raising security issues, e.g., in self-driving cars. These attacks on flow estimators come in the form of patches that are pasted onto the input image frames at the same pixel coordinates. This is motivated by cases where something gets stuck on the video camera (see Figure 1). These patch-based attacks cause occlusions on areas where the attack is placed, and motion boundaries along the boundaries of the attack, which is where current flow estimators suffer [Kim et al., 2021; Yu et al., 2022].

Schrodi et al. [2019] is the first work to tackle optical flow estimators with adversarial patches that are optimized so that the direction of the original flow estimations are shifted towards the opposite direction. They continue this work to show that the location these adversarial patches are applied on the input images is important [Schrodi et al., 2021; Schrodi et al., 2022]. Figure 1 shows the effect of these patch attacks on two flow estimators, FlowNetC [Dosovitskiy et al., 2015] (top) and RAFT [Teed and Deng, 2020] (bottom), the current state of the art on flow estimation. While RAFT is more robust to these attacks than FlowNetC, these attacks still disrupt the flow estimations nearby these attacks, e.g. estimations on moving white car and on arm and face. Schrodi et al. [2021; 2022] further explain that the vulnerability of flow estimators comes from the attacks' self-similar patterns. However, there is no existing work, to the best of the authors' knowledge, that was proposed to detect, localize, and characterize these potential attacks.

We present a Spatially constrained adversarial Attack Detection and Localization (SADL) framework that aims to detect and localize patch-based attacks given any pre-trained flow estimators in an unsupervised fashion without prior knowledge about these attacks. Specifically, we propose analyzing the inner layers' activations with a local-neighborhood constraint across the spatial dimension.

The main contributions of this paper are:

• First to detect and localize patch-based adversarial attacks for optical flow networks based on spatially constrained subset scanning on the inner layer activations, without any training of a dedicated model nor prior knowledge of the patch-based attacks.
• Analysis of the effect of the attacks across various inner layers of the flow networks.
• Validate our method across several state-of-the-art optical flow networks and datasets.

2 RelatedWork

In this Section, we reviewed the related work on flow networks, adversarial attacks on flow networks, and subset scanning techniques from anomalous pattern detection literature.

Flow networks Recent methods [Sun et al., 2018; Xu et al., 2017; Dosovitskiy et al., 2015; Teed and Deng, 2020] on optical flow estimation employed deep learning to achieve promising results with a faster inference speed. Starting with Chen and Koltun [2016], these deep learning approaches used cost volumes [Rhemann et al., 2013], which comprised of feature correspondence scores between frames to compute optical flow. PWCNet [Sun et al., 2018] built the cost volumes at multiple scales, and FlowNet [Dosovitskiy et al., 2015] included the full cost volume at a single scale. Ilg et al. [2017] proposed FlowNet2 by stacking four FlowNetbased models. Teed and Deng [2020] achieved state-of-theart performance with RAFT using a four-dimensional cost volumes for all pairs of pixels on the input images.

Adversarial attacks in optical flowneural networks Ranjan et al. [2019] is the first work to apply adversarial attacks on optical flow models, by pasting an p × p adversarial patch onto the input image frames I1, I2 at the same location. This adversarial patch can be obtained using any pre-trained flow network by minimizing the cosine similarity between the estimated flow maps before ˆf = (u, v) and after the attack ˜f = (˜u, ˜ v), i.e., ˆf · ˜f/||ˆf||||˜f||. This work was later extended by Schrodi et al. [2021] to show that patch-based attacks trigger matching ambiguities in the correlation output, which are successively spread in the decoder into a wider neighborhood. In their subsequent work, Schrodi et al. [2022] demonstrated the importance of the spatial location that these adversarial patches applied on the input image frames. Large flow regions, e.g., fast-moving objects, are susceptible to a severe deterioration of flow estimations. Yamanaka et al. [2021] used patch-based adversarial attacks to simultaneously attack two CNNs developed for optical flow and monocular depth estimation. Wortman [2021] showed that these patch attacks for optical flow estimation can be hidden by adjusting their alpha values while still affecting the flow estimations. Global noise-based attacks were recently reported [Schrodi et al., 2022; Schmalfuss et al., 2022]. Schrodi et al. showed that optical flow networks are robust to white-box global noisebased attacks, and Schmalfuss et al. [2022] proposed perturbation constrained flow attack. Unlike the existing works, we focus on detecting the patch-based attacks in an unsupervised fashion without requiring any prior knowledge of the attacks. Our work further localizes these patch attacks in the input images, finding the origin of the attack, which can be useful in future mitigation efforts. To the best of the author's knowledge, we are the first work to detect or localize adversarial attacks on optical flow estimators.

Subset Scanning for Anomalous Pattern Detection in Neural Networks Subset scanning has been used to detect anomalous samples in various computer vision and audio tasks, including creativity sample characterization [Cintas et al., 2022], audio adversarial attacks [Akinwande et al., 2020] and skin condition classification [Kim et al., 2022a]. To detect anomalous samples, subset scanning searches for the "most anomalous" subset of node activations from the inner layers of the given test sample on any pre-trained networks, and uses these subset of nodes to obtain the anomalousness score of the test sample.

While existing works employed subset scanning to detect adversarial attacks, we are the first to use it to detect patchbased attacks as opposed to global noise-based attacks, on sequential images across temporal dimension, and on networks for regression instead of classification task. Since patch attacks in flow estimation occur in a local region, we propose to apply spatial constraints to the scanning step to improve the detection and localization of the attacks.

3 Proposed Approach: Spatially Constrained Subset Scanning

Given node activations of an inner-layer, L, of a pre-trained optical flow network Θ, we aim to detect a subset of test samples with adversarial attacks and localize these attacks. To this end, we propose SADL: Spatially constrained Adversarial attack Detection and Localization framework (see Fig. 2) to be applied on any pre-trained flow estimators, in an unsupervised fashion without any prior knowledge of the attacks.

3.1 Overview

Figure 2 illustrates an overview of our proposed method. We extract the node-level activations across a given inner-layer of the flow network for a pair of input images. We compare this observed activation against an empirical distribution of expected activations extracted from clean (un-attacked) images at that same node. This node-level comparison provides a p-value for each node's observed activation. If the test image pair has the same representation as un-attacked images in this flow network layer, then these observed p-values across nodes should appear uniformly distributed. However, if an adversarial attack systematically increases the activations, then the observed p-values would appear non-uniform.

Our scoring function, to quantify the deviation of the observation from the expectation, is a non-parametric goodnessof-fit statistic which measures the Kullback-Leibler divergence between the observed distribution of p-values and the expected, uniform distribution [Berk and Jones, 1979]. A high-scoring image pair suggests it has a different inner-layer representation than the clean, un-attacked images. Critically, this scoring function satisfies the Linear-time subset scanning (LTSS) property [Neill, 2012; McFowland et al., 2013] which allows for exact, efficient maximization over subsets of nodes. However, un-constrained maximization across all subsets of nodes may lead to high scores even when a test pair of images is not attacked (false positive due to matching to noise). Therefore, SADL implements channel constraints and proximity constraints which result in higher detection power and accuracy for localized patch attacks. The end result of SADL is a high-scoring subset of node activations in a local, spatial neighborhood and subset of channels. The score is used for detection power calculations, while the subset of spatial nodes is used for detection accuracy. Details on the constrained optimization process are provided below.

3.2 p-value Computation

Let DH0 be a dataset of M background images that have not been attacked. Feeding the background image pairs into Θ, we can extract M background activations AH0 j from each node Oj ∈ L. Given a test sample I ̸∈ DH0 , we extract the activation aj at node Oj in L. We can compute the p-value of the activation aj of this test sample by computing the proportion of activations in AH0 j that are greater than aj:

where [·] is the indicator function.

3.3 Spatial Alignment

The inner layer under consideration, L, has spatial dimensions w (width), h (height) and channels c. In order to properly incorporate spatial information across channels, we make the alignment assumption that if spatial location w′ ×h′ is to be evaluated, then activations (p-values) from all channels under consideration also located at w′ and h′ must be scored as well. This intuitive assumption is visualized at the top of Figure 2. Although spatial alignment is always enforced, SADL does not require all channels to be included in the returned subset. This extension is covered in the Spatial-Channel Optimization section below.

3.4 Non-parametric Scan Statistics

Given a tensor of p-values, P , computed with Eq. 1 and with the same w×h×c dimensions as the inner layer L, our goal is to find the subset of nodes, aligned at spatial locations, such that the corresponding p-values of these node activations is the most non-uniform. We quantify divergence away from the uniform distribution using a non-parametric scan statistic (NPSS), such as Berk-Jones [Berk and Jones, 1979], with a general form:F(S) =max

where S is the subset p-values (nodes) under consideration, N(S) is the number of p-values in S, Nα(S) is the number of significant p-values less than a threshold α (OptimizeNPSS). NPSS is preferred as we make minimal assumptions on how the activations at a given node are distributed.

Without spatial alignment enforced, NPSS can be trivially maximized by creating a subset consisting of all-and-only pvalues that are less than α. However, this naive maximization would likely violate the spatial alignment requirement by not necessarily including all p-values from the same spatial location. Enforcing spatial alignment makes this maximization

input : P ∈ [0, 1]w×h×c output: Fmax, rmax, cmax 1 Fmax ←-1 ; 2 for hk in H = LinearSpace(0, h, k) do 3 for wk in W = LinearSpace(0, w, k) do 4 Pk ∈ [0, 1]kk×c ← FlattenSpatial(P[hk : hk +k,wk : wk + k, :]); 5 rk ← RandomlySelectRows(Pk) ; 6 ck ← RandomlySelectRows(P T k ) ; 7 F ←-1 ; 8 while F is increasing do 9 F , rk ← OptimizeNPSS(Pk[:, ck]) ; 10 F, ck ←OptimizeNPSS(Pk[rk, :]T ) ; 11 if Fmax < F then 12 Fmax, rmax, cmax ← F, rk, ck ; 13 return Fmax, rmax, cmax

more difficult because some not-significant p-values may be included in the highest-scoring spatially aligned subset, and some significant p-values may be excluded.

To this end, we employ the Linear-time Subset Scanning (LTSS) property of the Berk-Jones scan statistic [Neill, 2012; McFowland et al., 2013]. Returning to the dimensions of the tensor, P, each of the wh spatial locations contains up to c p-values. There are 2wh - 1 possible subsets of spatial locations to consider with each location contributing c pvalues to the scoring function. The LTSS property reduces this search space from exponential, O(2wh ) to linear O(wh) while still guaranteeing exactness [Neill, 2012]. To do this, each spatial location is sorted by a priority function, Nα c . A spatial location with a priority of 1 means every one of the p-values at that location across the c channels is significant at level α. A priority value of 0.5 means only half of the pvalues at that spatial location are less than α. Once the wh locations are sorted by this priority, the LTSS property states that the highest-scoring subset is guaranteed to be one of the linearly-many subsets created by incrementally including the c p-values of the next-highest priority location. Critically, the LTSS property turns the Berk-Jones from a metric that measures divergence into an objective scoring function that can be efficiently maximized over exponentially-many subsets. SADL, in its base form, searches over the exponentially many subsets of spatial locations, Ψ, to find spatially-aligned subset of p-values, S∗ , with the highest score, F ∗ :

where F(S) is a NPSS that satisfies the LTSS property such as the Berk-Jones scan statistic.

3.5 Spatial-Channel Optimization (SCO)

The base version of SADL optimizes a scoring function over the 2wh - 1 possible subsets of spatial locations in a tensor with w×h×c dimensions and is forced to include all c channels in the returned subset. See the top part of Figure 2 for a visual example. This forced inclusion of all channels naively implies that all channels in the inner layer being scanned are impacted by a patch attack on an optical flow neural network. The more complicated reality is that only some subset of the c channels may be affected by the attack. By introducing Spatial-Channel Optimization features to SADL we are returning a subset of spatial locations rmax crossed with a subset of channels cmax. Without SCO in place, SADL is forced to return a subset of spatial locations spanning all channels.

Including SCO in the search process can be done through two additional insights. The first is recognizing that the same process that maximizes a scoring function over subsets of spatial locations can also be used to maximize over a subset of channels. In other words, the LTSS property can be applied across a different mode of the tensor, P. The second is using an iterative ascent procedure that optimizes over spatial locations first, then over channels, and then back to spatial locations. Each step of the ascent is conditioned on the highestscoring subset found so far, and this ascent is guaranteed to converge to a local maxima such that any change to either the subset of spatial locations or a subset of channels would decrease the score [McFowland et al., 2013]. Two random restarts are used to approach the global maximum.

3.6 Proximity Constraint (PC)

While SCO allows us to find an anomalous subset of node activations across spatial locations, these detected anomalous locations may be far apart from each other, spanning the entire frame of the given test image pairs. We want to further constrain the subset scanning so that the detected anomalous locations strictly come from a local spatial neighborhood. Thus, we enforce a proximity constraint where our optimization is only applied to the k × k spatial region (yellow region in Figure 2) within the given h×w×c p-values where k << h and k << w (see Algorithm 1). This will ensure that the detected subset of anomalous locations only comes from this k×k spatial neighborhood. We do this for all k×k locations in the given h × w locations with a stride of s = k/2, and return the subset of p-values from a k × k neighborhood that yields the highest NPSS score Fmax.

3.7 Patch Attack Localization

Given the subset of detected p-values from a k × k neighborhood with Fmax, we can easily localize the attack by finding which locations these detected p-values occur in the inner layer feature map. These detected locations in the feature maps can be up-sampled, if needed, to match the resolution of the input images, or vice versa. For instance, if a pixel at location (10, 10) is detected as anomalous in a feature map of size 20×20×c under consideration, then SADL predicts 3×3 attack to be in the input images of size 60 × 60 × 3 centered at location (30, 30). The localized attacks can be utilized for attack mitigation, which we leave for future work.

4 Experimental Setup

FlowNetworks We validate our approach using four stateof-the-art flow estimators, namely, FlowNetC [Dosovitskiy et al., 2015], FlowNet2 [Ilg et al., 2017], PWCNet [Sun et al., 2018], and RAFT [Teed and Deng, 2020]. When selecting inner layers to apply our proposed method, we choose the first layer in each of the network's components. Specifically, we select the first layer of the encoder and decoder module, and their correlation layer. FlowNet2 contains four FlowNet-based models and a fusion module, and we apply our proposed subset scanning on the first layer in each of the components. For PWCNet, we select the first layer from the encoder, the cost volume layer and the optical flow estimator. Finally, for RAFT, we select the first layer in the feature encoder and context encoder, and correlation and flow layer in the iterative update block. For layers that process multiple features, (e.g. Siamese encoder of RAFT), we concatenate the features across the channel dimension and apply subset scanning on the concatenated features.

Datasets Following the literature [Schrodi et al., 2022; Teed and Deng, 2020], we use KITTI 2015 [Menze and Geiger, 2015], raw KITTI [Geiger et al., 2013], MPISintel [Butler et al., 2012], and raw Sintel [Liu et al., 2019] datasets. KITTI consists of road scene images with sparse optical flow labels (2015) and without labels (raw). MPISintel contains 23 sequences from computer-animation short "Sintel" with flow labels. Its raw frames without labels have also been used in previous un- or semi-supervised flow estimators [Liu et al., 2019; Yuan et al., 2022; Yuan et al., 2023].

Generating patch-based attacks Following Ranjan et al. [2019] and Schrodi et al. [2022], we construct patch-based adversarial attacks on the four flow networks. Table 1 shows the change in end-point error with and without the adversarial patch attack. We use patches of four different sizes (p×p) in respect to the input image resolution, 4.8% (p = 153), 2.1% (p = 102), 0.5% (p = 51), 0.1% (p = 25). These patches are trained using KITTI raw and Sintel raw datasets for each network, and evaluated on the labeled test set of KITTI 2015 and on MPI-Sintel, respectively. As expected, we see worse performances as we increase the patch attack sizes. In terms of flow networks, these patch attacks harm the performance of FlowNetC the most and RAFT the least (see Figure 1).

4.1 Performance Evaluation

We use the labeled data of Sintel and KITTI datasets to evaluate the attack detection and localization method. For KITTI 2015 dataset, we use the test split of its data as the training split has been used in training the existing flow networks. The test split of KITTI 2015 consists of 200 image pairs, and we use 100 of these image pairs as background image pairs (I H0 1 and I H0 2 ) to obtain our expected distribution of activation, the other 100 sample pairs as our test images (I t1 and I t2 ), where 50 are non-attacked clean image pairs and the other 50 are anomalous image pairs with the patch attacks. Similarly, for Sintel dataset, we randomly select 200 image pairs for evaluation, 100 image pairs for background image pairs (I H0 1 and I H0 2 ), 50 sample pairs for anomalous test samples, and the last 50 sample pairs for clean test samples (I t1 and I t2 ).

To evaluate the detection power of adversarial attacks, we follow the subset scanning in neural networks literature [Cintas et al., 2020; Kim et al., 2022a] and use the area under the receiver operating characteristic curve (AUC). For localization of the attacks, we report average precision (AP) and average recall (AR), where we compute precision and recall of the correctly detected patch attack locations for each sample and average over the entire test set. Here, the correctly detected locations occur when the detected anomalous locations on the given inner layer feature overlap with the true attacked location in the input space, down-sampled to match the resolution of the feature.

5 Results

In this section, we show our results on the detection and localization of adversarial patch attacks on optical flow estimation. As SADL is the first work to detect and localize these patch attacks, we use a naive scanning method without any spatial constraints as the baseline method.

Patch Attack Detection We show in Table 2 the performance of patch attack detection in terms of AUC comparing the proposed SADL (ours) with the baseline ([Cintas et al., 2020]). Overall, we see higher detection performance with our proposed method (bolded). Figure 3 visualizes the distribution of Fmax of the clean (blue) and attacked (orange) test set corresponding to the values in Table 2 for a patch attack of size p = 153 on KITTI 2015. The two distributions are more separable using SADL than the baseline. While our method shows clear improvements across multiple networks and datasets, both methods suffer on MPI-Sintel for FlowNetC and on KITTI 2015 for PWCNet. We show in Section 6 that the hyper-parameter k can be optimized to improve their performance.

Patch Attack Localization Table 3 lists the performance of patch attack localization in terms of average precision and average recall (see Section 4). As expected, we see higher localization performances for larger patches. While our method sometimes fails to localize patch attacks of size p = 51, these small patch attacks comprise 0.5% of the input images and do not have much effect on the flow estimation performance as shown in Table 1. Similar to attack detection results above, SADL struggles to localize attacks on KITTI 2015 for PWCNet and on MPI-Sintel for FlowNetC, partly because our hyper-parameters are not optimized (see Section 6). Figure 5 shows an example visualization of a detected subset of anomalous locations in the feature space for the attacks of size p = 153 for KITTI 2015 (top) and MPI-Sintel (bottom). SADL successfully detects a subset of anomalous locations that align with the location of the patch attack.

6 Ablation Study

Performance across Layers Figure 4 shows the change in AUC for detecting 153 × 153 patch attacks as we go deeper in the four flow networks on KITTI 2015 dataset. For FlowNetC, we see better detection power as we go deeper into the network. This supports the claim made by Schrodi et al. [2022] that states that FlowNetC amplifies the effect of the patch attack going deeper into the decoder layers, making it easier to detect these anomalous behaviors in the deeper layers. The flow estimation by FlowNetC in Figure 1 also shows this pattern, where the effect of the patch attack is amplified to its neighborhood. For the other three networks, FlowNet2, PWCNet, and RAFT, we don't see this amplifying effect in the prediction, and they show better detection power in the earlier layers of the network. This observation aligns with Cintas et al. [2020], which shows that adversarial attacks can be better detected in the earlier layers of the network before their effects get saturated in the deeper layers.

Performance without the Proposed Components We show the performance changes in detection (Table 4) and localization (Fig. 5), without our proposed components, i.e., proximity constraint (PC) and spatial-channel optimization (SCO). Without PC, the detected subset of anomalous locations could be spatially far apart, which would make it hard to localize the patch attacks in the inputs. Without SCO, the anomalous score Fmax are computed using the activation across the entire channel in the detected locations, including the channels that are not anomalous. Overall, the use of the proposed components (SCO+PC) is shown to achieve the highest detection performance across different networks and patch sizes. Figure 5 further demonstrates the capability of the proposed PC component to successfully find the most anomalous local region where the attack occurs.

Impact of Local Region Size for Proximity Constraint Figure 6 shows the effect of k used in our proximity constraint on the AUC of attack detection on FlowNetC. Specifically, the x-axis of the plots show the ratio of the height of the feature we are considering and k, i.e. h/k, and the yaxis shows AUC for attack detection for KITTI 2015 (left) and MPI-Sintel (right). Generally, we see increasing detection performance as we decrease k. We selected k = h/3 for results in Section 5 to be similar to the largest patch attack size (p = 153). However, we can achieve even higher attack detection performances with other k values, especially for MPI-Sintel with p = 153 patch attacks (cyan in the right plot). However, using smaller k will sacrifice recall performances in attack localization as the detected k × k region will be smaller than the patch attack.

7 Conclusion

We propose SADL, the first method to detect and localize patch-based adversarial attacks in optical flow estimators. We use a spatially constrained subset scanning on the inner layer in an unsupervised manner without any training nor prior knowledge of the attacks. We further give insights on which layers are most affected by these attacks for various flow networks. We validated SADL with multiple flow networks and datasets, and compared it with a baseline method, across which SADL consistently performed well. Immediate next step could utilize detected and localized attacks to devise mitigation techniques for flow estimators. Future work also aims to explore various hyper-parameters and further analyze why the performances differ across various networks and datasets.