From Language to Segmentation: Collaborative Category-Guided Unsupervised Camouflaged Object Detection with SAM3

1 Introduction

Camouflaged object detection (COD) aims to detect objects that are highly concealed within their surrounding environments [Fan et al., 2020a; Fan et al., 2021; Fan et al., 2023]. These objects exhibit visual appearances that are extremely similar to the background, particularly along object boundaries that almost blend into the surroundings, making the task highly challenging. Despite this difficulty, camouflaged object detection has broad application potential in realworld scenarios, such as rare species discovery [Fan et al., 2020a], medical image segmentation (e.g., polyp segmentation [Fan et al., 2020b], lung infection segmentation [Fan et al., 2020c]), and agriculture (e.g., locust detection to prevent invasion [Fan et al., 2020a]).

∗Corresponding Authors

Considering that annotating camouflaged objects is particularly time-consuming and labor-intensive due to their low visibility [Fan et al., 2020a; Chen et al., 2024b], unsupervised camouflaged object detection (UCOD) has attracted increasing attention as a promising alternative. Early UCOD methods require training and generally suffer from limited performance (e.g., FOUND [Sim´eoni et al., 2023], UCOSDA [Zhang and Wu, 2023]), resulting in low practical usability. With the rapid development of foundation models, new opportunities have emerged for UCOD. Many recent UCOD approaches (e.g., GenSAM [Hu et al., 2024a], ProMac [Hu et al., 2024b], CVP [Tang et al., 2024]) leverage foundation models to significantly improve performance without any training. However, a substantial performance gap with supervised models still remains, which to some extent limits their real-world applicability. Most of these UCOD methods follow a common paradigm of "MLLM + other foundation models + SAM". The core idea is to combine MLLMs with other foundation models to generate spatial prompts for target objects, which are then fed into SAM [Kirillov et al., 2023] for segmentation. We argue that this paradigm may suffer from fundamental bottlenecks in camouflaged scenarios. This bottleneck mainly arises from two aspects. First, the spatial localization capability of MLLMs is limited in camouflaged scenarios. The boundaries between camouflaged objects and the background are highly ambiguous, making it difficult for MLLMs to provide accurate spatial prompts. Second, SAM is highly sensitive to spatial prompts in camouflaged scenes. Inaccurate prompts can severely degrade segmentation quality, forming a vicious cycle between unreliable localization and fragile segmentation. Consequently, we argue that spatial information interaction between MLLMs and SAM is likely a key factor constraining performance. The recent emergence of SAM3 [Carion et al., 2025] offers a new opportunity by supporting category-level prompts for the first time. This inspires us to explore a new direction-bypassing unreliable spatial interactions and instead leveraging more robust categories for model interaction, potentially breaking the performance bottleneck of existing paradigms, as shown in Fig. 1.

Although SAM3 can be directly paired with MLLMs to provide a potential solution for UCOD, effectively leveraging it remains non-trivial. First, in UCOD, MLLMs are required to obtain the category via task-specific prompts (e.g., "Return the camouflaged object category"). However, this process is inherently uncontrollable, as MLLMs may return categories at varying levels of granularity. Correspondingly, SAM3 is highly sensitive to textual prompts in camouflaged scenarios, where even slight variations in category granularity for the same image can significantly affect segmentation results, as shown in Fig. 2. As a result, the category granularity produced by MLLMs may be suboptimal for SAM3, leading to segmentation failures. To address this issue, we propose Fine-grained Category Query, which explicitly instructs MLLMs to consider categories at multiple granularity levels in the prompt and dynamically return a full-granularity category chain for the camouflaged object. This design effectively mitigates the negative impact of category granularity on SAM3 and improves segmentation robustness.

After obtaining the full-granularity category chain, it is used as a prompt set and fed into SAM3 to generate segmentation masks. Unfortunately, SAM3 is not trained on COD datasets and has a limited understanding of the camouflage concept. As a result, two typical failure cases may occur: 1) even with correct category prompts, SAM3 may lack confidence when segmenting highly camouflaged objects, leading to segmentation failure; and 2) due to the absence of explicit constraints on the camouflage concept during segmentation, SAM3 may easily include surrounding background regions with similar appearances, as shown in Fig. 2. To address these issues, we first introduce Text Prompt Augmentation tailored for UCOD, which explicitly incorporates the camouflage concept into the prompts, thereby improving the segmentation of highly camouflaged objects. In addition, we propose Semantic-Geometric Dual Confirmation, which jointly evaluates the generated masks from both semantic and spatial perspectives to filter out masks that correspond to background regions.

Despite these designs substantially improving the performance of the proposed method, failure cases remain unavoidable. In particular, when MLLMs generate inaccurate responses due to the extreme ambiguity of camouflaged samples or inherent hallucination effects, such errors can propagate to the subsequent SAM3 stage and ultimately lead to erroneous segmentation results. To further enhance reliability, we introduce Semantic-Geometric Reasoning Injection that supplements MLLMs with critical semantic and geometric cues, enabling more reliable reasoning in camouflaged scenarios. The refined category information is then fed back to SAM3 to produce more robust segmentation results. Overall, this design allows the system to progressively correct errors and significantly improves segmentation stability under challenging camouflage conditions.

Our contributions are summarized as follows:

• We propose a novel training-free framework for UCOD. To the best of our knowledge, this is the first work to leverage SAM3 in the UCOD setting by interacting with MLLMs through category-level prompts, rather than spatial prompts.
• We introduce Fine-grained Category Queries, enabling MLLMs to generate full-granularity category chains, thereby mitigating the adverse impact of SAM3's sensitivity to category granularity on UCOD.
• We propose Semantic-Geometric Dual Confirmation and Semantic-Geometric Reasoning Injection. The former jointly validates segmentation results from semantic and geometric perspectives, while the latter injects semantic and geometric priors into MLLMs to enhance reasoning over failure cases, thereby improving suboptimal outcomes in challenging camouflaged scenarios.
• Experimental results demonstrate that our method significantly outperforms existing UCOD approaches, achieves performance comparable to WSCOD methods, and even approaches that of fully supervised methods.

2 RelatedWork

Camouflaged Object Detection. Camouflaged Object Detection (COD) focuses on detecting camouflaged objects within an image. Research on Camouflaged Object Detection (COD) has progressed rapidly, yet most existing methods rely on fully supervised annotations [Pang et al., 2022; Luo et al., 2024; He et al., 2025b], which incur substantial labeling costs. To reduce annotation overhead, many works explore weakly supervised settings, using annotations such as points [Chen et al., 2024a], bounding boxes [Chen et al., 2024b], and scribbles [He et al., 2025a; He et al., 2023b] as alternatives to pixel-level labels. Although these approaches alleviate labeling effort to some extent, they still require per-image annotation. Beyond weak supervision, a growing body of work investigates COD under unsupervised settings. Existing UCOD methods can be broadly categorized into two groups. The first group consists of training-based UCOD methods, such as UCOS-DA [Zhang and Wu, 2023] and UCOD-DPL [Yan et al., 2025], which follow a paradigm of generating pseudo labels and then training segmentation models. The second group includes training-free UCOD methods [Tang et al., 2024; Hu et al., 2024a; Hu et al., 2024b; Du et al., 2025], which rely on foundation models including MLLMs, diffusion models, DINOv2 [Oquab et al., 2023], CLIP [Radford et al., 2021], and SAM. These methods typically follow a paradigm of combining MLLMs with other foundation models to generate spatial prompts, which are then used by SAM for segmentation. Benefiting from foundation models, this category generally achieves better performance than training-based UCOD methods.

Multimodal Large Language Models for COD. Multimodal Large Language Models (MLLMs) achieve unified vision-language understanding through joint visual-textual token alignment. These MLLMs, like GPT-4V [Achiam et al., 2023], LLaVA-1.5 [Liu et al., 2024] and Qwen2.5VL [Bai et al., 2025], demonstrate unparalleled adaptability across diverse tasks. Many studies employ MLLMs for UCOD. CVP [Tang et al., 2024] proposes CoVP to improve MLLMs' perception in COD. GenSAM [Hu et al., 2024a] tries to map text prompts onto image heatmaps using MLLMs, acquiring visual prompts. ProMaC [Hu et al., 2024b] leverages hallucinations from large models to obtain instance-specific prompts. EASE [Du et al., 2025] utilizes MLLMs to extract category-level textual information from images, which is then input into a Diffusion Model to generate corresponding images.

Segment Anything Model for COD. Segment Anything Model (SAM) [Kirillov et al., 2023] is an interactive segmentation model that supports multiple types of prompts. Owing to this flexibility, SAM has been widely adopted in both weakly supervised and unsupervised COD [Chen et al., 2024b; Tang et al., 2024; Hu et al., 2024a; Hu et al., 2024b; Du et al., 2025]. In WSCOD, methods [He et al., 2025a; Chen et al., 2024b] use weak annotations as prompts to generate pseudo masks via SAM. In UCOD, approaches [Tang et al., 2024; Hu et al., 2024a; Hu et al., 2024b; Du et al., 2025] obtain spatial prompts by combining MLLMs with other foundation models and feed them into SAM for segmentation. Recently, SAM3 [Carion et al., 2025] further extends SAM by supporting category-level prompts, enabling segmentation to be conditioned on semantic categories rather than explicit spatial cues.

3 Approach

The overall architecture of the proposed framework is illustrated in Fig. 3. The Fine-grained Category Query (FCQ) explicitly guides MLLMs to comprehensively reason about the camouflaged object categories at multiple semantic granularities. Semantic-Geometric Dual Confirmation (SGDC) jointly validates the masks produced by SAM3 from both semantic and geometric perspectives. Semantic-Geometric Reasoning Injection (SGRI) further enhances the reasoning capability of MLLMs by incorporating prior knowledge from both semantic and geometric domains, enabling robust handling of hard samples caused by segmentation failures.

3.1 Fine-grained Category Query

We employ a task prompt as the input to MLLMs to obtain the camouflaged object category, which is then used as the prompt for SAM3 to generate segmentation masks. However, the granularity of the categories predicted by MLLMs is highly variable, and SAM3 is sensitive to category specifications in camouflaged scenarios. This mismatch often leads to sub-optimal segmentation results, as illustrated in Fig. 2. To address this issue, we propose Fine-grained Category Query (FCQ), which explicitly enforces MLLMs to reason about camouflaged object categories at multiple levels of granularity within the task prompt, as shown in Fig. 3. Specifically, conventional UCOD methods adopt a simple task prompt for MLLMs, e.g., "Return the camouflaged object localization and category." In contrast, we augment the prompt by explicitly requiring MLLMs to identify camouflaged objects at different granularity levels and to dynamically return a complete chain of categories across these levels according to each input image. This design effectively mitigates the negative impact of category-granularity sensitivity on SAM3. In addition, we further require MLLMs to output background category and localization information in the prompt, which will be utilized by subsequent modules. The detailed prompt design is provided in the supplementary material.

3.2 Semantic-Geometric Dual Confirmation

After obtaining the full-granularity category chain, it is used as a prompt set for SAM3 to generate segmentation masks. However, since SAM3 is not trained on COD datasets, it has a limited understanding of the concept of camouflage, which leads to two major issues: 1) SAM3 exhibits low confidence when segmenting highly camouflaged objects. 2) Due to the lack of explicit constraints on the camouflaged concept, it tends to mistakenly segment visually similar background regions. To address these problems, we propose Text Prompt Augmentation and Semantic-Geometric Dual Confirmation (SGDC). The former enhances SAM3's ability to segment camouflaged objects by strengthening its understanding of the camouflage concept, while the latter performs a comprehensive quality assessment of the generated segmentation masks.

Specifically, MLLMs take the task prompt and the input image described in the previous section as input, and output a full-granularity category chain, the background category, and the corresponding localization coordinates:

where I denotes the input image, Ptask is the task prompt, B represents the bounding box, Cfg denots the category chain, and Cbg represents the background category. The category chain Cfg is further augmented in a COD-specific manner:

where ⊕ denotes string concatenation, i represents the index of the granularity level in the full-granularity category chain Cfg = {C 1 fg , C 2 fg , . . . ,C N fg }, ordered from coarse to fine. ˆ C ifg and Cbg are used as category prompts for SAM3, yielding multiple foreground masks and a single background mask, respectively. The corresponding segmentation masks are obtained as:

where Mfg and Mbg denote the sets of foreground masks and the background mask obtained from SAM3, respectively. It is worth noting that Mfg may contain zero or more masks, as SAM3 may produce no output for a given category prompt.

For the non-empty cases of Mfg, we perform a semantic and geometric dual assessment. From the semantic perspective, well-segmented samples should exhibit complementary foreground and background masks with low overlap. Therefore, we use the Intersection-over-Union (IoU) to evaluate the degree of overlap betweenMfg andMbg:

where mifg denotes the i-th foreground mask in Mfg, τ is a threshold controlling the allowed overlap, and Sigisem indicates whether mifg satisfies the semantic complementarity constraint. From the geometric perspective, although the bounding boxes predicted by MLLMs are not perfectly aligned at the boundaries, they can still roughly indicate the location of the identified camouflaged objects. Therefore, we extract the enclosing bounding box of each foreground mask inMfg as the predicted object location, and compare it with the box provided by MLLMs:

where Box(·) denotes the bounding box of mask, η is a threshold, and Sigigeo indicates whether mifg is geometrically consistent. Only masks with Sigisem = 1 and Sigigeo = 1 are retained as valid candidates, and all valid masks are finally fused by mean aggregation to generate the final output mask.

3.3 Semantic-Geometric Reasoning Injection

Although most samples are successfully segmented after SGDC, a subset of hard cases still fails. This is mainly due to inaccurate predictions from MLLMs, which consequently lead to failures in SAM3. To address this issue, we propose Semantic-Geometric Reasoning Injection (SGRI), which injects semantic and geometric priors into MLLMs to enhance their reasoning ability on hard samples. For semantic priors, we collect the category information from successfully segmented samples to construct a category library, which is then provided to MLLMs as a set of highly probable candidate categories. For geometric priors, we extract the logits maps from SAM3 for the failed samples. Although these logits maps may be noisy and incomplete, they still contain informative boundary cues that are valuable for identifying camouflaged objects, as illustrated in Fig. 3. We therefore extract boundary information from the logits maps and combine it with the original image to inject geometric priors into MLLMs. With both semantic and geometric priors, MLLMs are prompted to re-reason and generate a new foreground category chain. To ensure a reliable final mask, the category chain is fed back into SAM3 to obtain a set of logits maps. We then group the logits maps according to their IoU similarity. Logits maps within the same group are first fused to form a group-level mask. The fused masks from different groups are subsequently concatenated to construct a composite mask image. This is further downsampled to reduce computational cost and is concatenated with the input image as the input to MLLMs, which performs a final selection to produce the ultimate segmentation result.

4 Experiments

4.1 Experimental Setup

Datasets. We evaluate our method on three widely used camouflaged object detection benchmarks: CAMO [Le et al., 2019], COD10K [Fan et al., 2020a], and NC4K [Lv et al., 2021], which contain 250, 2,026, and 4,121 test images, respectively.

Evaluation Metrics. We employ four commonly used metrics for evaluation: Mean Absolute Error (MAE), S-measure (Sm) [Fan et al., 2017], E-measure (Em) [Fan et al., 2018], weighted F-measure (Fwβ ) [Margolin et al., 2014].

Implementation Details. We implement our method in PyTorch and conduct all experiments on a single NVIDIA RTX A6000 GPU. We adopt the official release of SAM3 [Carion et al., 2025] for all experiments. For the MLLM component, we use Qwen2.5-VL-3B [Bai et al., 2025], and the detailed prompts are provided in the supplementary material. The logits maps are binarized with a fixed threshold of 128 before being used.

4.2 Compare with State-of-the-art Methods

Quantitative Comparison. As shown in Tab. 1, we conduct a comprehensive comparison with COD methods of different learning paradigms. Compared with fully supervised methods, our approach outperforms several strong fully supervised baselines. Compared with weakly supervised methods, our method achieves comparable performance without any training. Compared with training-free unsupervised methods, our approach delivers superior performance while using fewer foundation models and significantly fewer parameters. Compared with unsupervised methods that require training, our method not only eliminates the need for training but also substantially surpasses these methods.

Qualitative Evaluation. We visualize the predicted camouflaged object segmentation masks produced by our method and competing approaches under several challenging scenarios, including small objects, large objects, complex backgrounds, and blurry boundaries. As shown in Fig. 4, our method consistently achieves higher segmentation accuracy than existing UCOD and WSCOD methods, producing clearer object boundaries and more detailed representations of camouflaged objects.

Computational Cost Analysis. As shown in Tab. 1, although our model relies on foundation models, it requires fewer foundation models and has a smaller overall parameter footprint compared with other high-performing UCOD methods. Furthermore, unlike ProMac and GenSAM, which require six and twelve full iterations respectively, our method adopts a more targeted strategy by dynamically selecting objects for subsequent optimization based on the input sample, as shown in Tab. 2. These optimization targets constitute only a small subset of the dataset, and the additional optimization involves only two extra MLLM inferences. Since the foundation models do not require training, the memory footprint is approximately ∼11 GB, and inference on a single image takes only ∼4.14 s, which is substantially lower than the training cost of supervised methods (e.g., SAM-COD requires ∼21 GB of memory and ∼7 hours of training). Additionally, although each sample involves multiple category queries to SAM3, the image encoder is executed only once, and its features are reused across different queries. The additional computational overhead introduced by the text encoder and mask decoder is relatively small compared to that of the image encoder.

4.3 Ablation Study

To verify the effectiveness of our modules, we conduct ablation studies on the most challenging dataset, CAMO, where all methods achieve the lowest scores according to Tab. 1. Effectiveness of Fine-grained Category Query. As shown in Tab. 3, compared with directly using a task prompt that prompts the MLLM to return a single category of camouflaged object, our fine-grained category queries significantly improve the final performance.

Effectiveness of Semantic-Geometric Dual Confirmation. The Semantic-Geometric Dual Confirmation (SGDC) module identifies low-quality segmentation masks and applies subsequent refinement to these cases, thereby improving the final segmentation performance. In the ablation study of this module, we disable the subsequent refinement operation (SGRI) for the identified masks. Instead, we use bounding boxes predicted by the MLLM as prompts for SAM3 to generate replacement masks. Even under this setting, the overall performance still improves significantly, as shown in Tab. 3. Effectiveness of Semantic-Geometric Reasoning Injection. As shown in Tab. 3, applying Semantic-Geometric Reasoning Injection to the low-quality masks identified by SGDC leads to a substantial improvement in the final performance.

4.4 Further Analysis

Hyperparameter sensitivity. Two hyperparameters are involved in our model, namely τ and η. We evaluate different values of these hyperparameters, as reported in Tab. 4. Based on the results, we set τ = 0.3 and η = 0.3 in all experiments. Notably, the model performance is relatively insensitive to the specific values of these two hyperparameters.

Comparison across Different MLLMs. As shown in Tab. 5, we evaluate the performance of our method with MLLMs of different scales. While the performance generally improves as the model scale increases, the gains are relatively modest. In addition, we examine the performance across different types of MLLMs. As shown in Tab. 6, our method still achieves strong results when LLaVA is used as the MLLM.

4.5 Transferability Studies

To evaluate the transferability of our proposed method to related downstream tasks, we conduct experiments on Anomaly Detection (AD) and Transparent Object Segmentation (TOS). Extension to Transparent Object Segmentation. To demonstrate the strong generalization capabilities, we evaluated our model on the transparent object segmentation (TOS) task. Specifically, we evaluate our proposed model on the GSD [Lin et al., 2021] dataset for TOS. As shown in Tab. 7 and Fig. 5, our method also performs well on the TOS task.

Extension to Anomaly Detection. For the anomaly detection task, we conduct experiments on the CDS2K benchmark dataset [Fan et al., 2023], following the experimental settings of [Fan et al., 2023]. As shown in Tab. 8 and Fig. 5, our method also achieves competitive performance on anomaly

detection.

5 Conclusion

In this work, we presented a novel training-free framework for unsupervised camouflaged object detection that leverages SAM3 via category-level interaction with MLLMs. By introducing Fine-grained Category Queries, Text Prompt Augmentation, and Semantic-Geometric Dual Confirmation with Reasoning Injection, our method effectively mitigates the limitations of spatial prompts and improves segmentation robustness in challenging camouflaged scenarios. Extensive experiments demonstrate that our method consistently outperforms existing UCOD approaches and achieves performance comparable to weakly supervised methods, all without requiring any training. Beyond its empirical success, our method provides a new paradigm for integrating foundation models in UCOD, highlighting the potential of robust category-level interactions for tackling highly ambiguous visual tasks.

Acknowledgements

This work was supported by the Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (grant no. JYB2025XDXM902), the National Natural Science Foundation of China (grant no. 62502200), the Jiangsu Provincial Science and Technology Major Project (grant no. BG2024042), and the Natural Science Foundation of Jiangsu Province (grant no. BK20251203).