HyperTrans: Efficient Hypergraph-Driven Cross-Domain Pattern Transfer in Image Anomaly Detection

1 Introduction

Image anomaly detection aims to identify abnormal images and segment anomalous subregions, playing a vital role in industrial quality control [Defard et al., 2021; Roth et al., 2022]. Traditional methods have predominantly relied on the RGB modality, achieving significant progress [Zuo et al.,

∗Co-Correpsonding Authors.

2024b; Li et al., 2024]. However, advancements in technology have facilitated the adoption of image acquisition devices with multiple modalities, introducing pervasive crossdomain [Torralba and Efros, 2011] challenges. Among these, the shift to 3D is the most representative, as 3D sensors provide unique advantages for detecting depth-related defects [Zavrtanik et al., 2024]. This transition has increased demand for model upgrades. Various approaches have been proposed from a non-cross-domain perspective, including point cloud-based methods [Wang et al., 2023] and depth map-based methods [Chen et al., 2023; Liu et al., 2023], applied in both single-modal and multi-modal scenarios.

However, these non-cross-domain methods encounter inherent challenges. Multi-modal approaches, which rely on the collaborative input of both source and target data, impose high demands on acquisition devices and modality alignment. Single-modal methods, which retrain the sourcespecific model, tend to discard valuable information embedded in the original source model-information, optimized with extensive validated data. Furthermore, all existing approaches encounter substantial challenges due to the limited availability of target datasets, which are characterized by extreme scarcity and insufficient labeled samples. This limitation hinders the effective training of large-scale target models, particularly during the initial phases when acquiring adequate labeled data remains a resource-intensive endeavor.

To address these challenges and preserve the effective supervisory signals in the source model, we introduce and explore the feasibility of cross-domain (CD) adaptation [Wilson and Cook, 2020; Zhang et al., 2022] in industrial anomaly detection. This approach transfers and fine-tunes the source model for other domains. To further mitigate the issue of limited target domain data, we emphasize few-shot learning (FS) scenarios [Huang et al., 2022; Xie et al., 2023a]. Specifically, we conduct the first dedicated study on the problem of Cross-Domain Few-Shot Image Anomaly Detection (CDFSAD), which transfers knowledge from the source domain and leverages few-shot samples from the target domain, providing an innovative solution to anomaly detection.

Cross-domain adaptation in defect detection poses significant challenges. As illustrated in Figure 1, images captured by different sensors exhibit significant disparities. Achieving domain alignment is impeded by three key factors: 1) changes in image style (CS), which hinder feature extraction; 2) blurred boundaries (IB), which degrade anomaly segmentation accuracy; and 3) inter-class variance (ICV) [Fu et al., 2025], which compromises defect classification precision. To systematically address these challenges, we propose an innovative perturbation-rectification-scoring framework named HyperTrans, which maximizes the exploitation of source-domain information by hierarchically aligning and scoring patches across domains, achieving cross-domain generality and defect-detection specificity.

Specifically, a domain perturbation module injects channel-level perturbations to simulate new domain styles, followed by a lightweight adapter to align these features with the original domain, improving generalization. Furthermore, limited samples exhibit inherent relational information, as different patches from both domains may share similar properties, such as reflecting translation and rotation invariance of the same object attributes. To capture this higherorder correlations and enhance boundary delineation, we construct cross-domain hypergraphs [Hamilton et al., 2017; Gao et al., 2020; Kipf and Welling, 2022; Feng et al., 2023] that link patches globally across samples and domains. A residual hypergraph convolutional network (ResHGNN) further integrates and refines these latent relationships, with pseudo-features designated for target domain features, enabling effective cross-domain feature alignment.

While improving target domain adaptability, the previous training strategy also brings the source and target domain patches closer, thereby enhancing the utility of the source domain bank. Consequently, we incorporate the source domain bank into inference and propose a cross-domain scoring module leveraging subhypergraphs to assign anomaly scores and segmentation maps. By synthesizing differential outcomes between normal patterns across domains and query features, the module offers a unified solution to ICV and enables more precise anomaly classification and segmentation.

Our main contributions are listed as follows:

1. We pioneer the application of domain adaptation to image anomaly detection and propose an innovative perturbation-rectification-scoring framework tailored for cross-domain few-shot scenarios, facilitating
2. efficient knowledge transfer between domains with minimal labeled data.
3. We design a compact hypergraph cross-domain anomaly detection module. Centered on hypergraph, three key functions are implemented: a residual hypergraph convolutional network to model cross-domain higher-order correlations, a pseudo-feature generation method to guide the alignment and a cross-domain scoring method leveraging cross-domain normal patch patterns to facilitate anomaly detection.
4. Comprehensive experiments validate the superiority of the proposed model, significantly outperforming other state-of-the-art non-cross-domain methods in few-shot anomaly detection tasks and establishing a benchmark for cross-domain few-shot anomaly detection.

2 RelatedWorks

2.1 Image Anomaly Detection

Anomaly detection identifies rare data points that indicate defects in industrial production [Defard et al., 2021; Roth et al., 2022]. Similar to surface anomaly detection [Zavrtanik et al., 2022], both aim to recognize unexpected patterns in data or images. Methods for anomaly detection can be broadly classified into two categories: contrastive reasoning based methods and meta-learning based methods. Among the contrastive methods, image-level contrastive reasoning focuses on comparing entire images, such as in SimpleNet [Liu et al., 2023], which learns simpler representations for anomaly detection through image-level reconstruction. Feature-level contrastive reasoning compares features extracted from images, and includes methods like PatchCore [Roth et al., 2022], which performs anomaly detection by comparing feature patches. Instead, DSR [Zavrtanik et al., 2022], which trains a general VQ-VAE [Van Den Oord et al., 2017] and a specialized decoder to output anomaly maps, can be categorized as a meta-learning-based method. Similarly, AST [Rudolph et al., 2023] is also a meta-learning based method, as it employs a student-teacher architecture and utilizes knowledge distillation for anomaly detection. However, neither method can solve the cross-domain problem, as its model cannot adapt well to the diverse styles of features on the target domain.

2.2 Few-shot and Cross-domain Learning

In few-shot defect detection, many methods have been explored. For example, CLIP-FSAC [Zuo et al., 2024a] leverages the high generalization ability of the CLIP model and uses an alternately trained adapters to enable few-shot defect classification, demonstrating that adapters are effective for transferring the model to new domains. Additionally, graphbased methods have shown promising results by exploiting the correlation information between samples. The graph structure can effectively reduce the reliance on the number of samples by leveraging information from a few samples [Xie et al., 2023b]. In this work, we generalize this idea to hypergraph and extend it to model cross-domain feature patterns. Although cross-domain approaches have not been exclusively explored in defect detection, many general crossdomain frameworks [Muandet et al., 2013; Liu et al., 2024;

Fu et al., 2025] have emerged in recent years and can serve as paradigms. For instance, PDA [Bai et al., 2024] also inherits the high generalization capability of the CLIP model and utilizes cross-domain memory banks with a cross-attention enhancement mechanism, addressing the domain adaptation problem and more relevantly, demonstrating the effectiveness of memory banks in cross-domain tasks. DR-Adapter [Su et al., 2024] achieves top results in cross-domain few-shot segmentation through perturbations and a novel cyclic perturbation correction model. However, it is a one-to-one correction mode and lacks targeted exploration of relationships in small sample settings. By leveraging these approaches and uncovering high-order patch correlations, we introduce crossdomain few-shot learning to anomaly detection via the HyperTrans framework, which is tailored for defect detection and holds broader cross-domain applicability.

3 Methodology

3.1 ProblemDefinition

A typical defect detection task includes anomaly classification (AC) and anomaly segmentation (AD) [Li et al., 2024]. The dataset is divided into a normal sample set N and an anomaly sample set A. The goal is to train a model f(·) in an unsupervised manner, using only normal samples from dataset x ∈ N, to produce output consisting of a binary classification result for defect detection (denoted as a) and a binary semantic segmentation result for defect localization (denoted as S), where the classification result a is more primary. The whole process can be expressed as:

Cross-domain few-shot anomaly detection aims at generalizing and transferring the applicability from the source domain to the target domain, where only a few normal samples are available for training. Formally, given a model f(·) trained with data from the source domain x ∈ Ns and fewshot target domain normal samples N few-shot t , the task is to align the model to mitigate the issue of domain discrepancy between Ns ∪ As and Nt ∪ At. The training process is conducted on the source domain followed by fine-tuning on the target domain, and the inference process is performed on the target domain, which can be denoted as:

3.2 Hyper-graph Cross-domain Few-shot Anomaly Detection Framework

Overview

To accomplish CD-FSAD, we propose a novel framework called HyperTrans, with its architecture illustrated in Figure 2. The training process is firstly conducted on the sourcedomain, where the domain adapter is optimized with perturbed patches and a source-domain feature bank is generated. Correspondingly the target feature bank is generated with few-shot target samples. The two banks are united to construct a cross-domain hyper-graph, representing the higher-order correlations between patches. Subsequently, a pseudo-feature is assigned to each feature through the hypergraph, and the distances between them are reduced by the hyper-graph residual network to correct the cross-domain feature shift. During the inference process, the the source and target domain subhypergraphs are separately contracted along hyperedges to compute the anomaly score of patches, and the vertice-level results are weighted and integrated to form the image-wise anomaly score and pixel-wise anomaly map.

Domain Perturbation

Previous studies [Zhou et al., 2021; Su et al., 2024] have shown that perturbing channel statistics can effectively simulate different domain styles, thereby enhancing the domain adaptability. Building on this insight, we apply this technique to simulate diverse image patches extracted by the backbone.

For clarity, we denote the feature extractor symbolically as B(·), which derives patch features F = B(Ns) from the source samples Ns. Given the feature set F ∈ Rn×c×h×w , the mean avg(·) and variance std(·) are first calculated to represent feature channel statistics:

where n is the number of patches, c is the number of channels, h and w represent the height and weight of the patches. In defect detection, changes are typically localized and abrupt. Therefore, instead of using Gaussian noise, we design an alternative mechanism based on Perlin noise [Perlin, 1985] to better approximate smooth local variations in spatial structures and perturb the channel statistics (mean and variance) of feature maps. Specifically, two perturbation factors, Pavg and Pstd, are generated using Perlin noise, with dimensions matching the feature channels. These factors are applied to adjust the original mean and standard deviation as follows:

The perturbed statistics are then integrated into the Adaptive Instance Normalization (AdaIN) formula [Huang and Belongie, 2017] to compute the modified feature map:

To simplify, this can be rewritten as:

Such perturbations are accommodated by a learnable adapter of the deep layers of the backbone. To bring the simulated features Fp closer to the original features Fo, a domain perturbation loss with Mean Squared Error (MSE) is developed as the objective function, formally expressed as:

where ∥ · ∥2 denotes calculating the 2-norm of the matrix. n is the number of elements in the patch.

This approach dynamically adjusts the perturbation of the mean and variance using Perlin noise, leveraging its smooth, spatially coherent properties. Unlike direct noise addition, this method seamlessly integrates the perturbations with the feature map's channel statistics, resulting in spatially consistent and locally adaptive feature variations. Subsequently, through Ldp, the adaptability of the original backbone to different domain styles is augmented.

Hyper-graph Domain Rectification

To mitigate the domain discrepancy of patch features across different domains, we designed a cross-domain hypergraph feature correction module (HDR) centered on hypergraphs. Samples from the source and target domains represent different manifestations of the same object, with intra-domain and inter-domain patches exhibiting correlations-such as describing the same attributes of an object. Hypergraph is utilized to bridge such high-order correlations. Unlike edges in simple graphs, hyperedges in hypergraphs naturally simulate the attribute features shared among multiple vertices.

Each core patch in the banks from different domains is treated as a vertex. If let Fs denote the set of patches from the source domain and Ft denote the patches from the target domain, the cross-domain hypergraph can be represented as G = {Fs ∪ Ft, E,W}, where E is the hyperedge set. W ∈ R|E|×|E| is a diagonal matrix, where Wi,i denotes the weight of the i-th hyperedge.

The hyperedges are constructed based on the nodes with the reciprocal of Euclidean distance being the similarity metric. A hyperedge is created for each node, connecting it to other nodes based on similarity. For a source feature bank containing |Fs| patches and a target bank with |Ft| patches, a total of |Fs|+|Ft| hyperedges are constructed in this manner.

For computational convenience, the hypergraph is represented in matrix form as H ∈ [0, 1]|Fs∪Ft|×|E| where each row represents the vertex and each column represents a hyperedge. The value of Hi,j ∈ [0, 1] indicates the correlation from i-th vertex to the j-th hyperedge. To leverage the highorder relational information embedded in the hypergraph to guide feature rectification, a hypergraph convolution [Gao et al., 2022] (HGNNConv) is employed. It is based on hypergraph convolution layers c(·), defined as follows:

where Dv and De are the diagonal degree matrices for vertex and hyperedge, respectively. Θ ∈ R is the trainable parameter for the HGNNConv layer. By applying HGNNConv on the hypergraph constructed from cross-domain patches, the generated structure-aware embeddings can capture latent collaborative information from both the source and target domains.

The final hypergraph network is constructed with two layers of HGNNConv. Additionally, a residual structure is equipped to preserve the original features and reduce the compromise of feature correction on the original information. The overall output of the Res-HGNN is expressed as:

where α is a hyperparameter to balance the corrected and original features. The dropout rate is set to 0.5.

The pseudo feature generation is then developed to assign a prototype to each patch by leveraging the correlation information within the hypergraph. Specifically, for each node in the hypergraph, the feature of the most correlated node within its corresponding hyperedges is selected as the pseudo feature. In matrix form, the generation process can be denoted as:

where g(a, b) represents extracting elements from matrix a based on index list b and max index(·) outputs the index of the largest component for each row. InHc , non-cross-domain connections are set to 0.

To align the output features with the pseudo features and the original ones, Mean Squared Error is adopted:

Cross-domain Scoring

The feature bank from the source domain stores extensive information about normal samples. Relying on the feature bank for inference solely on target domain samples is insufficient, as it neglects the valuable information from source domain. Initially, due to domain shift, the differences between the source domain's feature bank and the original normal features of the target domain are significant, greatly impacting the effectiveness of defect scoring. However, the preceding domain perturbation and rectification modules align the target domain features generated by the model with the source domain features, mitigating the effects of domain shift. This alignment enhances the utility of the source domain bank for comparative scoring in the target domain. To further capitalize on this, we propose a cross-domain scoring module.

For each image I from the target domain requiring inference, we denote its patch feature set as F = R(B(I)), from which f is a specific patch feature. In the first step, the patches to be inferred are incorporated alongside the source and target domain banks to build the hypergraph H. To separately leverage the similarity information from both the source and target domains, the hypergraph matrix H is split. The source domain subhypergraph Hs ∈ [0, 1]|F|×|Fs| preserves the correlations between target features to be inferred and source feature bank, while the target domain subhypergraph Ht ∈ [0, 1]|F|×|Ft| preserves correlations of query target features and target feature bank. Subsequently, a defect score is assigned to each patch leveraging the subhypergraph from the source domain Hs , and the maximum score among all patches is taken as the image-level anomaly score:

where max(·) returns the max element of a vector and Hs (f) is the row with f being the vertex in the subhypergraph matrix in the source domain Hs . The pixel-wise anomaly map is obtained by rearranging patch anomaly scores based on their respective spatial positions:

The procedure is also adopted to assign anomaly score and segmentation map with the target domain subhypergraph:

The final image-wise anomaly score a and pixel-wise anomaly map S are obtained by combining the results of multiple domains with a parameter β to balance the impact of information from source and target domain:

To match the original input resolution, the final anomaly map is upsampled using bilinear interpolation. Additionally, a Gaussian function with a kernel width of σ = 4 is applied to smooth the results.

3.3 Model Training

Benefiting from the above observation, we propose a twostage learning strategy to train the whole model. The first stage will employ the domain perturbation loss Formula (7) to optimize the domain adapter in DP. Then, the model switch into the second stage with the domain rectification loss, i.e., Formula (11), to optimize the parameters in Res-HGNN. The overall procedure is illustrated in Algorithm 1.

Algorithm 1 Optimization algorithm for HyperTrans

Input: Backbone B(·), domain perturbing module ω(·), normal samples from source domain Ns, few-shot normal samples from target domain Nt, original parameters in domain adapter Θ, initialized parameters of Res-HGNN {Θ1, Θ2}

Parameters: epochsP, epochsR, ...

Output: Memory banks {Ms,Mt}, optimized parameters {Θ, Θ1, Θ2}

1: while training do 2: for e = 1 → epochsP do 3: F ←- B(Ns), Fp ←- ω(F) 4: Θ optim ←---- Ldp(F, Fp) 5: end for 6: while constructing feature banks do 7: Fs ←- B(Ns), Ft ←- B(Nt) 8: Ms store ←--- Fs, Mt store ←--- Ft 9: end while 10: for e = 1 → epochsR do 11: F ←- B(Nt) ∪Ms, 12: Fr ←-R(B(F)), Fpse ←- S(F) 13: {Θ1,Θ2} optim ←---- Ldr (Fr , Fpse) 14: end for

• 15: end while

4 Experiments

4.1 Experimental Settings

Dataset: The MVTec-3D AD dataset [Bergmann et al., 2021] is a standard 3D anomaly detection dataset, on which our experiments are performed. It consists of 10 categories, a total of 2656 training samples, and 1137 testing samples. The 3D scans were acquired by a high-resolution industrial 3D sensor using structured light. Due to the scarcity of defective samples, only normal samples are contained in the training set of MVTec-3D AD, while the testing set contains both normal and defective samples. The following experiments are carried out on it, with RGB data as the source domain and the XYZ data preprocessed into depth images to be the target domain.

Implementation Details: We implement HyperTrans using PyTorch2.4.0 and conduct evaluations on an NVIDIA A100. The ViT base patch8 224 dino [Caron et al., 2021] serves as the backbone. The domain perturbation module is integrated into layers 3 to 10 of the backbone, employing Perlin noise with a mean of 0 and a standard deviation of 0.75 for domain perturbations. 1

EvaluationMetrics: Models are evaluatedwithROCAUC in image and pixel orientation (Receiver operating characteristic - Area under the curve), which is a metric to evaluate the performance of a classifier by calculating the area under the ROC curve at various thresholds. The image-wise ROCAUC evaluates the performance based on the classification of entire images as defective or non-defective, while the pixel-wise ROCAUC assesses the ability to correctly classify each individual pixel as part of a defect or not. Initial values of other parameters are listed in Table 2.

1 https://github.com/raRn0y/HyperTrans

4.2 Comparision Experiments

Statistics: To verify the few-shot general anomaly detection performance, we compare HyperTrans with advanced depth methods. The image-wise and pixel-wise ROCAUCs are reported in Table 1. It can be concluded that source models generally have limited domain adaptability. When handling tasks with domain shift, most methods suffer from great accuracy loss, most of which come from the model design targeting source characteristics and the deep fitting of network to source data. With only a few depth samples (1, 2 and 5 samples for each classes) given, HyperTrans achieves the SOTA depth-domain performance in the task of cross-domain anomaly classification and segmentation. The advantage is more significant on the primary metric of image-wise ROCAUC with an improvement of 13%, 12% and 15% on 1, 2 and 5 shot settings compared with the best non-cross-domain methods. Tracing the image-wise results, HyperTrans improves the distinguishability on anomaly samples and alleviates the issue of inter-class variance. From the pixel-wise results, it enhances feature clarity and, through cross-domain joint inference-making, mitigates the problem of IB.

Visualizations: Figure 3 shows intuitive examples of predicted segmentation maps. HyperTrans achieves the best segmentation, effectively addressing the issue of IB.

4.3 Parameter Analysis

To evaluate the impact of α (residual on hypergraph crossdomain feature alignment), β (the influence of the source and target domains during inference), and the cross-interaction between feature alignment and inference, we conducted a series of parameter analysis experiments on the two parameters. The results are shown in Figure 4. As α increases, the crossdomain enhancement component increases, while the residual component decreases. From Figure 4 (Left), it can be seen that due to the fitting of HDR module on the source domain, the results relying on source domain patterns show an increase in fluctuations, while the results on the target domain gradually decrease. Therefore it's necessary to introduce β to balance the weight of source and target domain for inference and study the cross impact of α and β. From Figure 4 (Right), it can be concluded that HyperTrans achieves the best average ROCAUC on the test set when α = 0.3, β = 0.6 and the introduction of both components is essential.

4.4 Ablation Studies

Impact of EachModules: To measure the impact of each losses on the results and the effectiveness of each component in the framework, we conduct an ablation analysis with results shown in Table 3. We observe a consistent performance improvement with the introduction of each proposed modules, and the combination of them improves the averaged results by 0.018. This improvement is attributed to the effectiveness of cross-domain alignment, which not only mitigates the issues highlighted in Figure 1 but also brings the source and target domain patches closer. This proximity enhances the utility of the source domain bank, further improving the inference performance of the CDS mechanism.

Impact of Domain Perturbation: The DPmodule aims to simulate samples from various domains with channel statistic noise injection, guiding the model to adapt to non-existent domain styles in advance. t-SNE [Van der Maaten and Hinton, 2008] in Figure 5 shows that the perturbed features, after undergoing correction through this module, become closer to the original ones, demonstrating its effectiveness.

Impact of Hypergraph Domain Rectification: The HDR module provides a global correction across domains. We visualize its impact with PCA [Jolliffe and Cadima, 2016] in Figure 6. Comparing the distributions of the original patches with those of the HDR-enhanced patches, the representation spaces of the source and target samples become closer and the features are more aligned. This reduces the domain shift and enhances the utility of source bank during inference.

5 Summary

In this paper, we pioneer the exploration of cross-domain learning in image anomaly detection and introduce CDFSAD, a task designed to enable the effective transfer of knowledge for anomaly detection in few-shot learning scenarios. A novel perturbation-rectification-scoring solution named HyperTrans is then developed. The domain perturbation module simulates target domain styles using statistical Perlin noise, while the domain rectification module employs hypergraphs to capture higher-order cross-domain feature correlations, integrating residual hypergraph convolution and pseudo-feature generation for alignment. During inference, a cross-domain scoring method jointly leveraging multi-domain subhypergraph similarity information for multi-level anomaly reasoning. Extensive experiments validate that HyperTrans achieves SOTA performance, significantly outperforming non-cross-domain methods in classification and segmentation tasks, and providing a scalable, efficient and universal solution for image anomaly detection.

Acknowledgements

This work was supported in part by the Grants of Guangdong Major Project of Basic and Applied Basic Research (2023B0303000009) and the National Major Scientific Instruments and Equipments Development Project of National Natural Science Foundation of China under Grant (62327808) and the Shenzhen University 2035 Program for Excellent Research (00000224), and the National Natural Science Foundation of China Project (62403326).