Boundary-Guided Camouflaged Object Detection

1 Introduction

Camouflage is an important defense mechanism in nature that helps certain species hide in the surroundings to protect themselves from their predators, through concealment by means of materials, coloration or illumination, or self-disguise as something else, such as imitating the appearance, colors, or patterns of the environment and disruptive coloration [Price et al., 2019]. This mechanism also affects human life, such as art, culture and design (e.g., camouflaged uniforms) [Stevens et al., 2009]. Recently, identifying a camouflaged object from its background, namely camouflaged object detection (COD), has attracted increasing research interest from computer vision community. It has promising prospects for facilitating various valuable applications in different fields, ranging from animal conservation, e.g., species discovery [Ricardo et al., 2012] and animal monitoring, to vision-related areas, including image synthesis [Fan et al., 2020a], medical image analysis [Fan et al., 2020b], and search-and-rescue. However, COD is a very challenging task due to the nature of camouflage, that is, the high intrinsic similarities between candidate objects and chaotic background, which make it difficult to spot camouflaged objects for humans and machines.

∗Corresponding Author.

To tackle this issue, numerous deep learning-based methods have been proposed for camouflaged object detection and have shown great potentials. These methods can be broadly divided into three types. One is to design targeted network modules/architectures to effectively explore discriminative camouflaged-object features for COD, such as C2FNet [Sun et al., 2021], UGTR [Yang et al., 2021]. One is to incorporate some auxiliary tasks into the joint learning/multi-task learning frameworks, such as classification task [Le et al., 2019], edge extraction [Zhai et al., 2021], salient object detection [Li et al., 2021], and camouflaged object ranking [Lv et al., 2021]. This kind of method can excavate valuable and extra cues from the shared features to significantly enhance the feature representation for COD. Another is the bio-inspired method, which mimics the behavior process of predators in nature or human visual psychological patterns to design the networks, such as SINet [Fan et al., 2020a; Fan et al., 2021a], MirrorNet [Yan et al., 2021], and PFNet [Mei et al., 2021].

Although these recently proposed methods have made significant progress, there are still some major issues. Existing methods are often difficult to effectively and completely identify the structure and details of objects due to the edge disruption or body outline disguising, thus providing unsatisfactory predictions with coarse/incomplete object boundaries. As shown in Fig. 1, the ambiguities, caused by the high similarities between the butterfly head/heron legs and its background surroundings, make the features extracted by the recently state-of-the-art JCSOD [Li et al., 2021] and MGL [Zhai et al., 2021] (including single-stage version and recurrent version) models indistinguishable. Thus these models are incapable of recovering the boundary details of butterfly head and heron legs. As mentioned in [Zhao et al., 2019], the edge prior is widely used as an effective auxiliary cue, which benefits the preservation of object structure, yet has been barely studied for COD. Empirically, it is necessary to study how to enhance object-related edge visibility for facilitating the feature learning of COD. To our best knowledge, the mutual graph learning model (MGL) [Zhai et al., 2021] is the first to explicitly exploit the edge information to improve the performance of COD. However, MGL encodes edge features into graph convolutional networks along with object features and boosts feature representation through graph interaction modules. It can be seen that MGL is a sophisticated model, which inevitably increases model complexity and suffers from the heavy computational burden. In addition, it is noted that, despite the introduction of edge cues, MGL still loses some fine boundary-related details and thus weakens the performance of COD. As shown in the 2nd row of Fig. 1, the MGL models lose many details about heron legs and introduces some obvious background noises in prediction results.

To this end, in this paper, we propose a novel boundaryguided network (BGNet), which explicitly employs edge semantic to enhance the performance of camouflaged object detection. Firstly, we design a simple yet effective edge-aware module (EAM), which integrates the low-level local edge information and high-level global location information to explore edge semantics related to object boundaries under explicitly boundary supervision. Then, the edge-guidance feature module (EFM) is introduced to incorporate the edge features with the camouflaged object features at various layers to guide the representation learning of COD. The EFM module can enforce the network to pay more attention to the object structure and details. After that, the multi-level fused features are aggregated gradually from top to bottom to predict camouflaged objects. To enhance the feature representation, we build a context aggregation module (CAM), which mines and aggregates multi-scale contextual semantics by a series of atrous convolutions to produce features with stronger and more effective representations. Benefiting from the welldesigned modules, the proposed BGNet predicts camouflaged objects with fine object structure and boundaries. Note that, compared with MGL, we design a simpler but more effective edge extraction module to excavate accurate object boundary semantics, and then guide the feature representation learning of camouflaged objects by means of the proposed EFM and CAM. Besides, our method achieves more accurate object localization and stronger preservation of object structure. To sum up, our main contributions are as follows:

• For the COD task, we propose a novel boundary-guided network, i.e., BGNet, which excavates and integrates boundary-related edge semantics to boost the performance of camouflaged object detection.
• We carefully design the edge-guidance feature module (EFM) and context aggregation module (CAM) to enhance boundary semantics and explore valuable and powerful feature representation for COD.

2 RelatedWork

Datasets. To boost deep learning-based camouflaged object detection, some annotation datasets are proposed. [Le et al., 2019] constructed the first camouflaged object dataset, namely CAMO, including 1,250 camouflaged images covering eight categories. [Fan et al., 2020a] collected a largescale challenging dataset, called COD10K, which contains 10,000 images covering 78 camouflaged object categories, with high-quality and hierarchical annotations. Recently, to support localization and ranking of camouflaged objects, [Lv et al., 2021] proposed a ranking-based testing dataset, named NC4K, which contains 4,121 images with extra localization annotation and ranking annotation.

Camouflaged object detection. In recent years, camouflaged object detection has attracted increasing attention in the computer vision community [Pang et al., 2022; Cheng et al., 2022]. Since the release of large-scale data sets (e.g., CAMO and COD10K), numerous deep learning-based camouflaged object detection models have been proposed and achieved great progress. These methods can be roughly divided into three types. The first type of method is to design advanced network modules/architectures to explore discriminative camouflaged features for COD. [Sun et al., 2021] designed an attention-induced cross-level fusion module and a dual-branch global context module to enhance feature representation. [Yang et al., 2021] incorporated the Bayesian learning into transformer-based reasoning, which can leverage both deterministic and probabilistic information for COD. The second type of method is to combine some auxiliary tasks into the joint learning frameworks to boost the performance of COD. [Le et al., 2019] proposed an anabranch network that performs an auxiliary classification network to help camouflaged object segmentation. [Zhai et al., 2021] exploited edge extraction as an auxiliary task and incorporated it into mutual graph learning for COD. [Li et al., 2021] presented a joint salient object detection and camouflaged object detection network to enhance the detection ability of both tasks. To segment and rank camouflaged objects, [Lv et al., 2021] designed a ranking-based COD model in a joint learning framework that can mutually boost the performance of each another. The last one is the bio-inspired method, which is inspired by the behavior process of predators in nature or human visual psychological patterns. [Fan et al., 2020a] presented a search identification network to gradually locate and search for the camouflaged object, inspired by the process of wild predators discovering prey. [Mei et al., 2021] proposed a positioning and focus network by mimicking the detection and identification stages of predation.

3 ProposedMethod

3.1 Overall Architecture

The overall architecture of the proposed BGNet is illustrated in Fig. 2. Specifically, we adopt Res2Net-50 [Gao et al.,

2019] as our backbone network to extract multi-level features from the input image, i.e., fi (i = 1, 2, . . . , 5). Then, an edgeaware module (EAM) is applied to excavate object-related edge semantics from the low-level features, which contain local edge details (f2), and the high-level features, which contain global location information (f5) under object boundary supervision. Following multiple edge-guidance feature modules (EFM) are leveraged to integrate the edge cues from EAMwith multi-level backbone features (f2-f5) at each level to guide feature learning, which enhances boundary representation. Finally, multiple context aggregation modules (CAM) are employed to progressively aggregate multi-level fused features in a top-down manner and discover camouflaged objects. In testing, we select the prediction of the last CAM as the final result. Noted that, we do not adopt the f1 backbone feature because it is too close to the input with much redundant information and a small receptive field.

3.2 Edge-aware Module

A good edge prior can benefit object detection in both segmentation and localization [Zhang et al., 2017; Zhao et al., 2019]. Although low-level features contain rich edge details, they also introduce many non-object edges. Thus, high-level semantic or location information is needed to facilitate the exploration of camouflaged object-related edge features. In this module, we incorporate the low-level feature (f2) and the high-level feature (f5) to model the object-related edge information, as shown in Fig. 3. Specifically, two 1×1 convolution layers are first used to change the channels of f2 and f5 to 64 (f ′2 ) and 256 (f ′5 ), respectively. Then we integrate the feature f ′2 and the up-sampled f ′5 by concatenation operation. Finally, we obtain the edge feature fe through two 3×3 convolution layers and one 1×1 convolution layer followed by the Sigmoid function. EAM is a simple yet effective module to extract specific edge features. As shown in Fig. 7, EAM perfectly learns the object boundary-related edge semantics.

3.3 Edge-guidance Feature Module

The edge-guidance feature module (EFM) is designed to inject boundary-related edge cues into the representation learning to enhance the feature representation with object structure semantics. As is known to all, different feature channels often contain differentiated semantics. Thus, to achieve good integration and obtain a powerful representation, we introduce a local channel attention mechanism to explore cross-channel interaction and mine the critical cues between channels.

As shown in Fig. 4, given the input feature fi (i ∈ {2, 3, ..., 5}) and the edge feature fe, we first perform the element-wise multiplication between them with an additional skip-connection and a 3×3 convolution to obtain the initial fused features f e i , which can be denoted as:

where D denotes down-sampling and Fconv is 3×3 convolution. ⊗ is element-wise multiplication and ⊕ is elementwise addition. To enhance feature representation, inspired by [Wang et al., 2020], we introduce local attention to explore critical feature channels. Specifically, we aggregate the convolution features (f e i ) using a channel-wise global average pooling (GAP). Then we obtain the corresponding channel attention (weight) by the 1D convolutions followed by a Sigmoid function. Unlike fully-connected operations, which capture dependencies across all channels but show high complexity, we explore local cross-channel interaction and learn each attention in a local manner, e.g., only consider its k neighbors of every channel. After that, we multiply the channel attention with the input feature f e i and reduce the channels by 1×1 convolution layer to obtain the final output f a i , i.e., f a i = Fconv1(σ(Fk1D (GAP(f e i ))) ⊗ f e i ), (2) where Fconv1 is 1×1 convolution, Fk1D is 1D convolution with kernel size k and σ denotes Sigmoid function. The kernel size k can be set adaptively as k = |(1 + log2(C))/2|odd , where | ∗ |odd denotes the nearest odd number and C is the channels of f e i . The kernel size is proportional to channel dimension. Obviously, the proposed attention strategy can highlight critical channels and suppress redundant channels or noises, thereby enhancing semantic representation.

3.4 Context AggregationModule

To integrate multi-level fused features for camouflaged object prediction, we design a context aggregation module (CAM) to mine contextual semantics for boosting object detection, as shown in Fig. 5. Unlike global context modules in BBSNet [Fan et al., 2020c], which only exploits several independent parallel branches to extract features of different scales separately without considering the semantic correlation among various branches [Wu et al., 2020], CAM takes cross-scale interaction into account to boost feature representation. Taking f a 4 and f a 5 as an example, we first upsample f a 5 and concatenate them followed by a 1×1 convolution layer to obtain the initial aggregated feature fm. Next, we evenly divide fm into four feature maps (f 1m, f 2m, f 3m, f 4m) along channel dimension, and then perform cross-scale interaction learning, that is, to integrate the features of adjacent branches to extract multi-scale contextual features by a series of atrous convolutions. It can be formulated as:

f j ′m = Fnj conv (f j-1′ m ⊕ f jm ⊕ f j+1 m ), j ∈ {1, 2, 3, 4}, (3) where Fnj conv indicates a 3×3 atrous convolution with a dilation rate of nj. In our experiments, we set nj = {1, 2, 3, 4}. Besides, for i = 1, there is only f 1m and f 2m; for i = 4, there is only f 4m and f 3′ m . Then, we concatenate these four multiscale features f j ′m followed by a 1×1 convolution, a residual connection and a 3×3 convolution, which can be denoted as:

where [∗] is concatenation operation, and f c i is output of CAM. Noted that, for i = {2, 3}, the output of the previous CAM (f c i+1 ) will be used together with f a i as the input of the next CAM to obtain the f c i . With another 1×1 convolution to change the channel number of feature f c i , we can obtain the prediction Pi (i ∈ {2, 3, 4} of camouflaged objects.

3.5 Loss Function

Our model has two kinds of supervisions: camouflaged object mask (Go) and camouflaged object edge (Ge). For mask supervision, we employ the weighted binary cross-entropy loss (LwBCE ) and weighted IOU loss (LwIOU ) [Wei et al., 2020], which pay more attention to hard pixels instead of assigning all pixels equal weights. For edge supervision, we adopt the dice loss (Ldice) [Xie et al., 2020] to deal with the strong imbalance between positive and negative samples. Noted the mask supervision is conducted on three camouflaged object predictions (Pi, i ∈ {2, 3, 4}) from CAM. Thus, the total loss is defined as: Ltotal = ∑4i=2 (LwBCE (Pi,Go) + LwIOU (Pi, Go)) + λLdice(Pe, Ge), where λ is a trade-off parameter and set λ = 3 in our experiments, Pe is the prediction of camouflaged object edges.

4 Experiments

4.1 Implementation Details

We implement our model with PyTorch and employ Res2Net50 [Gao et al., 2019] pre-trained on ImageNet as our backbone. We resize all the input images to 416 x 416 and augment them by randomly horizontal flipping. During the training stage, the batch size is set to 16, and the Adam optimizer [Kingma and Ba, 2014] is adopted. The learning rate is initialized to 1e-4 and adjusted by poly strategy with the power of 0.9. Accelerated by an NVIDIA Tesla P40 GPU, the whole training takes about ∼2 hours with 25 epochs.

4.2 Datasets

We evaluate our method on three public benchmark datasets: CAMO [Le et al., 2019], COD10K [Fan et al., 2020a] and NC4K [Lv et al., 2021]. We follow the previous works [Fan et al., 2020a], which use the training set of CAMO and COD10K as our training set, and use their testing set and NC4K as our testing sets.

4.3 EvaluationMetrics

We utilize four widely used metrics to evaluate our method, i.e., mean absolute error (MAE, M) [Perazzi et al., 2012], weighted F-measure (F w β ) [Margolin et al., 2014], structuremeasure (Sα) [Fan et al., 2017] and mean E-measure (Eϕ) [Fan et al., 2021b].

4.4 Comparison with State-of-the-arts

To demonstrate the effectiveness of our method, we compare it against 18 state-of-the-art methods, including 10 SOD models, i.e., PoolNet [Liu et al., 2019], EGNet [Zhao et al., 2019], SRCN [Wu et al., 2019], F3Net [Wei et al., 2020], ITSD [Zhou et al., 2020], CSNet [Gao et al., 2020], MINet [Pang et al., 2020], UCNet [Zhang et al., 2020], PraNet [Fan et al., 2020b] and BASNet [Qin et al., 2021], and 8 COD models, i.e., SINet [Fan et al., 2020a], PFNet [Mei et al., 2021], S-MGL [Zhai et al., 2021], R-MGL [Zhai et al., 2021], LSR [Lv et al., 2021], UGTR [Yang et al., 2021], C2FNet [Sun et al., 2021] and JCSOD [Li et al., 2021]. For fair comparison, all the predictions of these methods are either provided by the authors or produced by models retrained with open source codes.

Quantitative Evaluation. Table 1 reports the quantitative results of our method against 18 competitors on three datasets. It is obvious that our method outperforms all other models on three datasets under four evaluation metrics. Specifically, compared with the second-best JCSOD, our method increases Sα by 1.80%, Eϕ by 1.40% and F w β by 3.55% on average. Compared with the third-best C2FNet, our method increases Sα by 1.93%, Eϕ by 1.41% and F w β by 4.28% on average.

Qualitative Evaluation. Fig. 6 shows the qualitative comparisons of different COD methods on several typical samples from the COD10K dataset, covering four super-classes, i.e., aquatic, terrestrial, flying and, amphibious. These results intuitively show the superior performance of the proposed method. Noted that our method provides accurate camouflaged object predictions with finer and more complete object structure and boundary details.

Boundary Exploration. Fig. 7 shows the visual comparison of our model with MGL in terms of boundary-related edge extraction. It can be seen that, although MGL proposes an auxiliary edge detection network based on a complex graph model, it still loses many structural details, resulting in poor boundary localization in prediction. It is proven that our method achieves superior performance in object-related edge information mining and camouflaged object prediction.

4.5 Ablation Study

In order to validate the effectiveness of each key component, we design several ablation experiments and report the results in Tab. 2. For baseline model (B), we remove all the additional modules (i.e., EAM, EFM and CAM), and only retain the 1×1 convolution in four EFMs to reduce the channels of the backbone features (fi, i = {2, 3, 4, 5}) and use the initial aggregation operation in the CAM to fuse the multi-level features in the top-down manner.

Effectiveness of CAM. From Tab. 2, compared with B model, the B+CAMmodel provides better performance. Especially, our module has more advantages on the metrics F ω β that shows 1.50% performance increases averagely.

Effectiveness of Edge Cues (EAM). To verify the effectiveness of object-related edge cues, we keep the initial fusion operation and final 1×1 convolution in EFMs, and remove the local channel attention (LCA). From Tab. 2, the model c (B+EAM+EFM w/o LCA) achieves better overall performance compared with the baseline model a, especially in terms of F ω β with 1.15% performance gains on average for all datasets. Thus, the edge prior extracted by EAM is beneficial to boost detection performance.

Effectiveness of EFM. Then we add LCA on model c, that is, the complete EFM, to validate effectiveness of the integra- Table 3: Ablation studies of different inputs of EAM on three datasets. The best results are highlighted in Bold.

tion operation of edge cues and camouflaged object features. As can be seen in Tab. 2, the B+EAM+EFMmodel shows the performance improvement compared with model a and model c, demonstrating the effective contribution of LCA and the proposed EFM to the final predictions. Furthermore, combined with the designed EAM, EFM and CAM, the proposed BGNet achieves obvious performance improvements on all datasets, with the performance gains of 1.10%, 1.14% and 2.65% on average in terms of Sα, Eϕ and F ω β , respectively.

Input of EAM. We also test the effectiveness of different inputs for EAM, e.g., f1, f2 and f3 used to explore edges with f5 to help locate object-related edges. As shown in Tab. 3, the combination of f2 + f5 obtains the best performance for camouflaged object detection.

5 Conclusion

In this paper, we resort to edge priors to assist recovering object structure and boost the performance of camouflaged object detection. We propose a simple yet effective boundaryguided network (BGNet), which contains edge-aware module, edge-guidance feature module, and context aggregation module, to explore object-related edge semantics to guide and enhance representation learning for COD. By adopting edge cues, our BGNet provides accurate camouflaged object predictions with complete and fine object structure and boundaries. Extensive experiments show that our method outperforms existing state-of-the-art methods on three benchmarks.