Dual Encoder Contrastive Learning with Augmented Views for Graph Anomaly Detection

1 Introduction

An attribute network is a graph structure in which both nodes and edges encapsulate topological information and incorporate rich attribute data, enhancing the depth and context of the graph mining and analysis. Such networks are prevalent in a variety of domains, including social networks [Latah, 2020; Zhang et al., 2019], financial networks [Motie and Raahemi, 2024], and recommendation systems [Ying et al., 2018; Yu et al., 2022]. A critical aspect of analyzing these networks is anomaly detection, which aims to identify instances that significantly deviate from the majority of the network's elements. Graph anomaly detection (GAD) plays a crucial role in numerous applications, including fraud detection and social spam detection [Ding et al., 2019; Ma et al., 2021].

The shallow anomaly detection methods, such as AMEN [Perozzi and Akoglu, 2016] and Radar [Li et al., 2017], have been widely used in various applications, particularly for tasks where domain expertise can aid in crafting effective features. However, these approaches have notable limitations, especially when applied to complex and high-dimensional data. This challenge has led to a shift towards deep learning techniques that automatically extract meaningful patterns and representations. [Ding et al., 2019] introduced an unsupervised autoencoder-based method that effectively detects abnormal patterns by leveraging reconstruction errors. Building on this, [Liu et al., 2021] pioneered a contrastive learning framework for graph anomaly detection, which effectively utilizes local contextual information as supervision. Further advancements have been made by those who employed patchlevel and context-level contrastive learning using two GNNbased models. Currently, models for graph anomaly detection ([Zheng et al., 2021; Zhang et al., 2022a]), which combine both reconstruction-based and contrastive-based learning techniques, have attracted significant attention due to their effectiveness in identifying anomalies.

However, this joint paradigm also has some limitations. The contrastive learning approach relies on fixed-size subgraph sampling, primarily focusing on local information while neglecting the global structure. This results in incomplete subgraph capture and the potential for abnormal nodes to undermine the reliability of neighborhood information. Additionally, the current paradigm mainly emphasizes attribute reconstruction while disregarding topological information, weakening the detection capability for topological anomalies and failing to capture the complex global relationships within the graph structure effectively. Although Graph Convolutional Networks (GCNs) [Kipf and Welling, 2016] can model both attribute features and structural information effectively, the learned representations are not always wellsuited for anomaly detection. Furthermore, the lack of effective dependency modeling between distant nodes remains a significant gap in existing graph contrastive learning methods, limiting their ability to capture long-range interactions.

To address these limitations, we propose a novel method for graph anomaly detection called DECLARE (Dual Encoder Contrastive Learning with Augmented Views for Graph Anomaly Detection). DECLARE integrates a dual encoder framework with multi-view contrastive learning to detect anomalies in attributed networks. Specifically, our approach leverages a dual encoder architecture comprising a Graph Neural Network (GNN) and a Graph Transformer (GT). This dual encoder framework enables us to learn more expressive representations of both structural and attribute information within the graph. The GNN encoder focuses on learning node-level features by capturing local neighborhood information, while the Graph Transformer encoder models long-range dependencies and captures global interactions using self-attention mechanisms. This complementary encoding allows for a holistic representation of the graph that incorporates both local and global structural insights. In contrast to previous joint learning paradigms that focus solely on attribute reconstruction, our method introduces a reconstruction module that simultaneously reconstructs both structural and attribute information. This dual reconstruction process enhances the model's ability to detect a wide range of anomalies, including those related to graph structure and node attributes. Finally, an anomaly score is computed for each node based on the learned representations, facilitating the identification of anomalous nodes in the graph. Through this approach, DECLARE effectively addresses the challenges posed by previous methods, providing a more comprehensive and accurate solution for anomaly detection in attributed networks. In summary, the key contributions of this work are:

• We introduce DECLARE, a novel framework that combines a dual encoder architecture using GNN and Graph Transformer to learn expressive and informative representations of graph data, which has not been thoroughly explored in the field of Graph Contrastive Learning.
• Design a novel dual reconstruction approach that simultaneously reconstructs both graph structures and attribute information, thereby enhancing the model's anomaly detection capabilities and overall performance.
• We demonstrate the effectiveness of DECLARE through extensive experiments on benchmark datasets, showing significant improvements over state-of-the-art methods in detecting various types of graph anomalies.

2 RelatedWork

2.1 Graph Anomaly Detection

[Perozzi and Akoglu, 2016] proposes normality, a measure combining structure and attributes to evaluate neighborhood consistency and separability, and introduces AMEN, which outperforms traditional methods in anomaly detection on attributed graphs. [Li et al., 2017] addresses anomaly detection in attributed networks by proposing a framework that models residuals of attribute coherence with network structure, effectively identifying anomalies without prior knowledge. Recently, with advancements in deep learning, Graph Neural Networks (GNNs) have been widely adopted for anomaly detection on graphs, often coupled with diverse learning paradigms to enhance their effectiveness. [Ding et al., 2019] introduced a framework leveraging GCNs and autoencoders to detect anomalies by jointly evaluating reconstruction errors in the adjacency and attribute matrices, offering a solution to challenges like network sparsity and data nonlinearity.

Contrastive learning approaches often utilize the similarity between target nodes and their surrounding context for model pre-training. Building on this idea, [Liu et al., 2021] developed a contrastive self-supervised learning framework specifically designed for graph anomaly detection. This framework effectively models the interactions between nodes and their subgraphs to identify anomalies. [Hu et al., 2023] introduced a method for detecting anomalous nodes through subgraph-aligned contrastive learning across multiple graph views. Similarly, [Duan et al., 2023a] leveraged contrastive learning to compute anomaly scores by exploring multi-scale relationships, including node-to-node, nodeto-subgraph, and subgraph-to-subgraph interactions, between the original graph and its augmented version.

2.2 Graph Augmentation

Existing graph augmentation techniques encompass a variety of strategies. [You et al., 2020] propose a node-dropping strategy for graph augmentation, while [Qiu et al., 2020] and [Zhang et al., 2023] focus on edge perturbation techniques. [Zhu et al., 2021a] and [Zhang et al., 2022b] suggest attribute masking approaches, and [Hassani and Khasahmadi, 2020] advocate for subgraph extraction. [Zhu et al., 2020] introduce GRACE, which employs random edge perturbation and node feature masking to generate two augmented graph views. Subsequently, [Zhu et al., 2021b] extend this idea with GCA, an adaptive augmentation strategy that combines structural and attribute information. [Wei et al., 2023] develop GCS, which further integrates structural and semantic data for adaptive graph augmentation. [Tan et al., 2024] propose CI-GCL, a framework designed to preserve graph community structures during augmentation. Despite their success, these methods often disrupt semantic relationships, potentially introducing biases into the learning process.

3 Preliminaries

In this section, we present the notations, formal definitions, and the problem statement that form the conceptual and mathematical foundation of our work.

Notaions. (AttributedNetwork) Let G = (V, E,X) denote an attributed network, where V = {v1, v2, ..., vn} represents the set of nodes and E is the set of edges. Here, n = |V| denotes the number of the nodes in the G. Each node vi ∈ V is associated with a feature vector xi ∈ Rm(i = 1, 2, ..., n), where m is the dimensionality of the feature space. The attribute matrix X ∈ Rn×m contains the feature vectors of all nodes. Additionally, the adjacency matrix A ∈ Rn×n is used to represent the graph structure, where Aij = 1 if there is an edge between nodes vi and vj , otherwise Aij = 0.

Definition 1. (Encoders) To effectively capture the complex structural and semantic patterns in G, we utilize two complementary encoders: a Graph Neural Network (GNN) Encoder and a Graph Transformer (GTR) Encoder. The GNN encoder, denoted as GNN(·) ∈ Rn×d , is used for localized aggregation, while the Graph Transformer encoder GTR(·) ∈ Rn×d captures long-range dependencies, where d represents the dimensionality of the hidden embeddings.

Problem Definition (Graph Anomaly Detection) Given an attributed network G with adjacency matrix A, the goal is to learn an anomaly score function f(·) to measure the degree of the abnormality for each node in G.

4 Methodology

In this section, we outline the overall framework of the proposed DECLARE method, as illustrated in Figure 1. DECLARE comprises four main modules: a graph enhancement module, a graph contrastive learning module, a graph reconstruction module, and an anomaly scoring module. The original graph is treated as the base view, while a second view is generated via graph diffusion. For each view, both the GNN and Graph Transformer (GTR) encoders process the sampled subgraph and the target node to learn informative hidden representations. The contrastive learning module identifies anomalies by distinguishing positive and negative pairs frommultiple perspectives. Meanwhile, the reconstruction module independently reconstructs graph attributes and structures, with anomalies detected based on their respective reconstruction errors in both attribute and structural spaces.

4.1 Dual Encoders

The issue of smoothing in GCNs has been discussed in previous works [Chen et al., 2020], highlighting the need for effective solutions. To obtain a more comprehensive graph representation, we employ dual encoders: GCN and Graph Transformer. The layer of the GCN encoder are designed to capture local graph structures, while the Graph Transformer encoder enhances the model's ability to capture long-range dependencies and intricate relationships. By combining these two encoders, we aim to overcome the smoothing problem and improve the overall graph representation. The formulation of each graph convolutional layer is given as follows:

where H(l) and H(l+1) are the input and output node feature matrices at layer l, W(l) is the learnable weight matrix, ˆA= ˜D- 12 ˜A˜D- 1 2 is the symmetrically normalized adjacency matrix with self-loops ˜A, and σ(·) is a nonlinear activation.

Simultaneously, we use NodeFormer [Wu et al., 2022] as the Transformer encoder, which can be represented as:

where ϕ(·) denotes a kernel function, the kernel function is approximated using Random Features, making the kernel mapping in Eq. 2 a random feature map from Rd to Rm. We adopt widely used Positive Random Features for this mapping: ϕ(x) = exp( -||x||22 2 ) √m [exp(wT 1 x, ..., exp(wTmx)]. And qvi =W (l) Q h(l) vi , kvj =W (l) K h(l) vj and vvj =W (l) V h(l) vj .

4.2 Contrastive Learning with Augmented Views

Building on the effective embeddings obtained earlier, we further enhance the graph's structural representation for contrastive learning by employing graph diffusion to generate augmented views, providing a global perspective of the structure [Hassani and Khasahmadi, 2020]. The graph diffusion process is defined as:

where S is the diffused adjacency matrix, {θk}k≥0 are nonnegative weighting coefficients satisfying ∑∞k=0 θk = 1, and T ∈ Rn×n is the row-normalized transition matrix. To power the graph diffusion process, we utilize Personalized PageRank (PPR) [Page et al., 1999]. The diffusion matrix S can be reformulated as:

where ρ denotes the teleport probability in the random walk process. The resulting matrix S represents a globally diffused view of the original graph structure.

The contrastive learning framework is applied across two views-base and augmented. For each node vi, we sample a local subgraph Pi of size t from the original graph using random walk with restart (RWR) [Tong et al., 2006]. From this base view, we construct its positive pair. The augmented-view subgraph is then derived by applying the same procedure to sample a subgraph. The subgraph of vi in both the base and augmented views serves as the positive pair, while the subgraph of another node becomes the negative pair.

To enhance the discriminative power of node representations, we mask the target node's attributes within the sampled subgraph before feeding it into both branches of the dual encoder, consisting of a shared Graph Convolutional Network (GNN(·)) and a Graph Transformer GTR(·). These representations serve as the foundation for downstream contrastive and reconstruction tasks. To illustrate, the two encoders compute low-dimensional node representations as follows:

Since the target node is masked, its representation is mapped into a shared embedding space using the GCN weights. The output is then given by:

where ϕ represents an activation function such as ReLU, and hl-1 i is the previous layer representation (either from GNN(·) or GTR(·). These steps yield the node representations Hi and ˜Hi for the local and global subgraphs, respectively, and hi for the target node vi.

To ensure consistency between the node and its subgraph representation, we apply an average pooling function as the readout module to generate the subgraph-level embedding vector. For simplicity, we use GNN(·) as an example, where the subgraph-level embedding vector is computed by averaging the node representations:

where ti is the number of nodes in the subgraph Pi. The discriminative score for the positive pair is then computed as:

where Ws is a learnable matrix and T represents the transpose operation. A similar score ˜ sGNN dis,i is calculated for the negative pair. The base-view contrastive loss for the pair vi,Gi, ˜Gi) is:

A similar loss LGNN aug is computed for the augmented view. The combined intra-view contrastive learning objective is:

Next, we define the inter-view contrastive loss to align the representations across the two views. The inter-view loss is calculated as LGNN inter = ∥sGNN base,i - sGNN aug,i ∥2F where sGNN base,i and sGNN aug,i are the discriminative score vectors for the positive pairs in the base and augmented views, respectively. For the GNN encoder, the contrastive learning loss with augmented views is defined as:

Similarly, the GTR encoder uses the same formulation for its contrastive learning loss.

4.3 Graph ReconstructionModule

Using the embeddings obtained from the aforementioned Dual Encoder, we perform separate reconstructions for both the structure and attributes. The structure decoder aims to reconstruct the structural information and detects structural anomalies by measuring the reconstruction error between the output and the input structure. We employ a GCN-based decoder, which can be formulated as follows:

where HGNN , WGNN Tdec denote the potential node embedding matrix after the decoder and the weight matrix of the topology decoder. ̂AGNN denotes the reconstructed adjacency matrix obtained from the inner product of HGNN .

The attribute reconstruction decoder plays a critical role in regressing node attributes and is an effective tool for detecting contextual anomalies in graphs. A GCN-based decoder is used for this purpose. Nodes whose attributes are wellapproximated by the decoder are considered less likely to be anomalous, whereas significant reconstruction errors suggest that a node's attributes deviate from the patterns typical of normal nodes. The attribute reconstruction process is expressed as:

where ̂XGNN raw is the reconstructed attribute matrix, and WGNN Adec represents the decoder's weight matrix. The graph reconstruction module encompasses two essential tasks: topology and attribute regression. Reconstruction errors from these tasks are combined to form the objective function, which is defined as:

where β is a balancing parameter that adjusts the relative importance of topology and attribute reconstruction. This approach ensures a comprehensive understanding of both structural and attribute-level inconsistencies, aiding in anomaly detection on graphs.

4.4 Graph Anomaly Score Module

Anomaly detection combines scores from the graph reconstruction and discrimination modules. For node vi, the reconstruction score is calculated as:

where β ∈ [0, 1] balances topology and attribute errors. Higher scores indicate greater anomaly likelihood. To complement this, we compute two discriminative scores, sGNN dis,i and sGTR dis,i , derived from the contrastive learning objectives under the dual encoders. And the final composite anomaly score combines all three components, formulated as:

where α ∈ [0, 1] balances the contributions of the two discriminative scores, and γ controls the influence of the reconstruction-based anomaly signal. To ensure robustness against stochastic variations such as subgraph sampling and training noise, we repeat the entire scoring process over R independent runs. The final anomaly score for each node is then computed by combining the average and variability of its per-round scores:

5 Experiment

5.1 Datasets

To evaluate our model, we use six widely recognized benchmark datasets commonly employed in graph anomaly detection. These datasets are categorized into two types: citation network datasets and social network datasets [Liu et al., 2021;

Zheng et al., 2021]. The citation network datasets include Cora, Citeseer, ACM, and Pubmed, while the social network datasets consist of BlogCatalog and Flickr. Since ground truth anomalies are not available, we adopt the anomaly injection method proposed by [Ding et al., 2019; Liu et al., 2021] to simulate realistic anomalous behaviors. This approach ensures a controlled and consistent evaluation environment across all datasets. The detailed statistics of these datasets are provided in Table 1.

5.2 Experimental Setup

Baselines And Evaluation Metrics

In our experiments, we compare our model with seven deep learning-based anomaly detection methods. DOMINANT [Ding et al., 2019] employs a deep graph autoencoder to detect anomalous nodes by utilizing both graph structure and features. CoLA [Liu et al., 2021] leverages a GNNbased contrastive learning approach at the node-subgraph level to compute anomaly scores for nodes based on positive and negative instance pairs. ANEMONE [Jin et al., 2021] utilizes multi-scale patch and context-level contrastive learning with GNNs to identify anomalous nodes. SL-GAD [Zheng et al., 2021] is a self-supervised method that integrates generative and structural spaces. Sub-CR [Zhang et al., 2022a] is a contrastive learning-based anomaly detection framework that uses graph diffusion to augment the original graph for both inner and outer views. GRADATE [Duan et al., 2023a] applies multi-view contrastive learning across node-node, node-subgraph and subgraph-subgraph relations to detect anomalies. Lastly, NLGAD [Duan et al., 2023b] is a multi-scale contrastive learning network that incorporates high-confidence nodes into the normality pool, achieving superior results through training based on these nodes. All methods are comprehensively evaluated using the AUC metric to ensure a fair and reliable comparison.

Hyperparameter Settings

In this study, we set the subgraph size to 4. The GNN encoder and the attribute decoder each consist of two layers of GNN, while the structure decoder uses a single-layer GCN. The hidden layer embedding dimension is fixed to 64. We optimize the model using the Adam optimizer, with a batch size of 300 across all datasets. For the Cora, Citeseer, Pubmed, and ACM datasets, the learning rate is set to 0.001, whereas for BlogCatalog, the learning rate is 0.003, and for Flickr, it is 5e-4. The model is trained for a total of 300 epochs.

5.3 Result And Analysis

As shown in the Table 2, DECLARE performs exceptionally well across all datasets, especially on more common datasets like Cora, Citeseer, and Pubmed, where it achieves high scores, particularly in Citeseer and Pubmed, with scores of 0.9772 and 0.9901, respectively. This indicates that DECLARE excels in these specific tasks. In contrast, DECLARE performs slightly worse on the Flickr and BlogCatalog datasets but still maintains stable performance, with scores of 0.7933 and 0.8189. Considering that these datasets usually contain more noise and complex multimodal information, DECLARE's performance remains acceptable. Cora and Citeseer are traditional citation network datasets, where DECLARE shows a clear advantage over other methods. For instance, on the Cora dataset, compared to CoLA (score 0.8941), DECLARE's score of 0.9433 represents an improvement of about 5.5%, and compared to SL-GAD (score

0.9001), the improvement is around 4.8%. On the Citeseer dataset, DECLARE scores 0.9772, which is an improvement of around 10.3% compared to CoLA (score 0.8864), and about 5.4% compared to ANEMONE (score 0.9268). On these datasets, DECLARE's performance improvement ranges from 5% to 10%, showing a significant advantage. On the Pubmed dataset, DECLARE also demonstrates substantial improvement, with a score of 0.9901, significantly outperforming other methods. Compared to CoLA (score 0.8984), DECLARE shows an improvement of around 10.2%, and compared to SL-GAD (score 0.9575), the improvement is about 3.4%, further highlighting its strong performance.

Although DECLARE's scores are relatively lower on the Flickr and BlogCatalog datasets, it still demonstrates stable performance with notable improvements. On the Flickr dataset, DECLARE scores 0.7933, which reflects an improvement of about 6.5% compared to CoLA (score 0.7449)

and around 13.0% compared to NLGAD (score 0.7021). On the BlogCatalog dataset, DECLARE scores 0.8189, showing an improvement of about 10.3% compared to CoLA (score 0.7421) and around 1.9% compared to SL-GAD (score 0.8039). While the improvements on these datasets are smaller, DECLARE still exhibits significant gains, with improvements ranging from 6% to 13%. However, DECLARE performs relatively weaker on complex multimodal datasets like Flickr and BlogCatalog. This is likely due to the intricate relationships between graph structures and textual features in these datasets, which DECLARE may not handle as effectively as models specifically designed for these tasks, such as NLGAD or GRADATE. Additionally, the smaller improvements observed in some datasets suggest that DECLARE's generalization ability could benefit from further targeted optimization in specific challenging scenarios.

5.4 Ablation and Analysis

The ablation experiments, summarized in Table 3, analyze the contributions of key components in DECLARE. Specifically, we conducted an ablation study on the reconstruction and contrastive learning modules, as well as on the use of GCNs and Transformers for modeling local and global structures. The results demonstrate that combining GCNs and Transformers yields better performance than using either alone. Furthermore, the results show that both the reconstruction and contrastive learning modules are essential for improving performance. Removing the reconstruction module (DECLARERec) causes performance drops across all datasets, with the largest declines on BlogCatalog (9.7%) and Flickr (10.5%). Similarly, removing the contrastive learning module (DECLARECon) reduces performance, especially on Cora (3.3%) and Citeseer (4.7%). These findings underscore the critical role of each component and motivate further studies to refine and optimize the framework.

5.5 Sensitivity analysis

Subgraph Size. As shown in Figure 3, the AUC rises from t = 1 to t = 4, peaks at t = 4, and then stabilizes or slightly decreases from t = 5 to t = 10, indicating that t = 4 is the optimal subgraph size across datasets.

Balance Parameter. We further analyzed the impact of the balance parameters α, β, and γ on performance. As shown in Figure 4, increasing α and β generally improves performance, with notable gains when α, β ∈ [0.5, 0.9]. In practice, we set α = [0.9, 0.9, 0.6, 0.8, 0.9, 0.9] and β = [0.9, 0.4, 0.7, 0.8, 0.8, 0.8] for Cora, Citeseer, ACM, Pubmed, Flickr, and BlogCatalog, respectively. Similarly, as shown in Figure 5, performance generally improves with higher γ, with optimal values typically in the range [0.6, 0.9]. We observe that γ = 0.7 works best for Cora, Citeseer, ACM, and Pubmed, while γ = 0.9 yields the best results on the more complex Flickr and BlogCatalog datasets.

6 Conclusion

In this paper, we propose the DECLARE framework as a comprehensive and robust solution for graph anomaly detection. It seamlessly integrates four key modules-graph augmentation, contrastive learning, reconstruction, and anomaly scoring-that work synergistically to improve detection accuracy. By generating multiple complementary views through graph diffusion and utilizing both GNN and Graph Transformer encoders, DECLARE effectively captures rich and informative embeddings of the graph's structure and attributes. The contrastive module enhances detection by distinguishing positive and negative pairs, while the reconstruction module identifies outliers via structural and attribute reconstruction errors. Experimental results demonstrate the superior effectiveness of DECLARE across various challenging datasets.

Acknowledgments

This work was supported by the National Key R&D Program of China (No.31400), NSFC (U2468204), Key R&D and Transformation Plan of Qinghai Province (No.2022QY-218), Science and Technology Plan of Shenzhen (KCXFZ20211020172544004), the Key R&D project in Hainan province, China (NO.ZDYF2024SHFZ051) and the Major Science and Technology Program of Yazhou Bay Innovation Institute of Hainan Tropical Ocean University (2023CXYZD001).

Contribution Statement

Nannan Wu and Hongdou Dong contribute equally to this work.