Gaussian Mixture Model for Graph Domain Adaptation

1 Introduction

The problem of labeling scarcity has long been a challenge in deep learning, as collecting large amounts of labeled data is often expensive, time-consuming, and even infeasible [Pan and Yang, 2009; Zhang et al., 2023]. Unsupervised domain adaptation (UDA) has emerged as an effective solution, which leverages knowledge from a source domain with abundant labeled data and transfers it to a target domain to compensate for the lack of labeled data in the target domain. While source and target domains typically share similar semantics, they differ in data distributions [Zhang et al., 2023; Ganin and Lempitsky, 2015; Du et al., 2024]. In this case, the knowledge from the source domain may not be directly applicable to the target task, leading to poor transfer performance.

†Corresponding Author

To address the distribution discrepancy between the different domains, UDA methods often reduce the feature distribution gap to learn domain-invariant [Long et al., 2017; Wang et al., 2025; Wang et al., 2024c; Wang et al., 2024b; Wang et al., 2024d] feature representations. This approach can effectively improve the performance of models in the target domain, even with a lack of labeled data, by utilizing the labeled data from the source domain. However, despite achieving some success, traditional UDA [Li et al., 2021a; Li et al., 2024] methods still face challenges due to significant differences between the two domains. These challenges include overly simplifying domain differences and overlooking domain-specific features. Therefore, how to better preserve the feature information of the target domain while reducing the distribution discrepancy has become a key research focus.

In recent years, UDA methods [Liu et al., 2024; Shi et al., 2024; Zhang et al., 2024b] have begun to explore incorporating additional prior knowledge, structured information, and graph models to further enhance domain adaptation performance. For example, GCAN [Ma et al., 2019] proposes an end-to-end graph convolutional adversarial network that jointly models data structures, domain labels, and class labels within a unified network framework to achieve UDA. AdaGCN [Dai et al., 2022] develops a novel and principled framework for graph transfer learning, effectively combining adversarial domain adaptation and graph convolution techniques to improve transfer learning efficiency. StruRW [Liu et al., 2023] estimates pseudo-node labels on the target graph to compute edge probabilities between different classes, and uses these probabilities to guide the bootstrapping of neighbors in the source graph's Graph Neural Network (GNN) computation, effectively reducing conditional shift between neighborhoods. However, these methods generally assume homogeneous clusters and rely on Euclidean distances between points in a given feature space for learning, which somewhat limits the efficiency of representation learning and clustering. The homogeneous clustering assumption overlooks the inherent complexity of the data, especially in domain adaptation tasks where there can be significant differences in data distributions across domains. In such cases, relying solely on Euclidean distances may fail to capture the complex relationships between data points, leading to suboptimal clustering results and poor performance in crossdomain transfer, making it difficult to achieve desirable generalization performance.

To handle this issue, we propose the use of the Gaussian Mixture Model (GMM) for graph domain adaptation. The GMM, with its ability to model data distributions as a mixture of multiple Gaussian components, offers a more flexible approach compared to traditional clustering methods. It can effectively capture the heterogeneity of clusters, thus allowing for a more accurate representation of complex, multi-modal distributions inherent in real-world data (see Figure. 1). Additionally, when applied to graph-based domain adaptation, the GMM can leverage the graph structure to encode relationships between data points more efficiently, allowing for improved domain alignment and more robust feature representations. By integrating GMM into the UDA framework, we can better model the domain shift and reduce the risk of negative transfer. The proposed approach not only enhances clustering performance but also facilitates smoother domain adaptation, enabling more reliable and scalable solutions in tasks such as cross-domain classification and data analysis. Our main contributions can summarized as follows:

• We propose a novel unified framework for graph domain adaptation, called GMM, which is capable of handling both heterogeneous graph structures and domain distribution shifts simultaneously. The framework effectively integrates the power of Gaussian mixture models for probabilistic distribution modeling and graph neural networks for structural representation learning.
• Incorporating graph structural information into the adaptation process, maintaining consistency in relational dependencies.
• We conduct extensive experiments on real-world information networks to verify the effectiveness of our model, which demonstrates its superior performance compared with state-of-the-art baselines, impressive label efficiency, and good model robustness against distribution discrepancy.

2 RelatedWork

2.1 Unsupervised Domain Adaptation

Current UDA techniques concentrate on acquiring feature representations that are domain-invariant. UDA techniques [Long et al., a; Long et al., b] have as one goal reducing the disparity between several domains. To quantify the difference between the source and target domains, early methods use combined MMD and MK-MMD. Additionally, various well-designed discrepancies and higher-order statistics are used [Zellinger et al., 2017]. The intra-class and interclass domain discrepancies introduced by CAN [Zellinger et al., 2017] are used to leverage category information for domain alignment. An alternative approach involves creating an adversarial optimization goal for a domain discriminator and using adversarial learning to acquire domain-invariant representations. Through the use of a progressively disappearing bridge mechanism, GVB-GD [Cui et al., 2020] encourages adversarial domain adaptation. Using several adversarial discriminators, GSDA [Hu et al., 2020] implements hierarchical domain alignment. In contrast to STAR [Lu et al., 2020], which samples classifiers from Gaussian distributions without additional parameters, MCD [Saito et al., 2018] maximizes the discrepancy between two classifiers.

2.2 Graph Neural Networks (GNN)

Graph Neural Networks (GNNs) are a natural extension of convolutional neural networks in non-Euclidean spaces, designed to process graph-structured data. GNNs can capture complex relationships and structured information, making them suitable for applications in social networks, molecular graphs, traffic networks, and more. The concept of GNNs was first introduced in [Gori et al., 2005; Scarselli et al., 2008; Wang et al., 2024a; Wang et al., 2025] as a trainable recursive message-passing model, where node representations are updated based on information passed between nodes, thereby learning the structural features of the graph. Subsequent research enhanced the model's expressive power by introducing gating mechanisms [Li et al., 2015; Sukhbaatar et al., 2016], improving its performance on complex graph structures. Spectral graph convolutional networks, proposed in [Bruna et al., 2013; Henaff et al., 2015], aim to capture global information by learning spectral multipliers of the graph Laplacian. However, these methods come with high computational costs. To address this issue, [Defferrard et al., 2016] introduced a method that learns polynomials of the Laplacian to replace the computation of eigenvectors, improving computational efficiency and reducing complexity, allowing GNNs to handle larger-scale graph data. Overall, the continuous optimization and innovation of GNNs have made them a powerful tool for processing graph data. They are widely applied in modeling and analyzing various complex systems. As research continues, the capabilities of GNNs will further improve, offering efficient solutions for a broader range of domains.

Our works is based on the spectral perspective line. The proposed model exploits the GCN to operate on a denseconnected instance graph so that data structure information can be jointly complemented with domain adaptation in a unified deep network.

3 Method

In unsupervised domain adaptation (UDA), we are given ns labeled samples {(x(i) S , y(i) S )}ns i=1 from the source domain DS, where x(i) S ∈ XS and y(i) S ∈ YS, with XS and YS representing the source data space and label space, respectively. Additionally, we are also provided with nt unlabeled target samples {(x(i) T )}nt i=1 , where x(i) T ∈ XT , and XT denotes the target data space. It is assumed that XS and XT are different but related. Our goal is to use the model trained on the source domain to predict the unlabeled samples in the target domain.

The proposed graph domain adaptation framework is shown in Figure 2. In the UDA task, when domain shift occurs, the label prediction function f is trained by minimizing the overall objective function, as detailed in Eq. 1:

The LC (XS, YS) denoted the classification loss. LDA (XS,XT ) and LGAA (XS, YS,XT ) present the domain alignment loss and graph-aware alignment loss, respectively. The details are introduced as follows.

3.1 Domain Alignment

We use the domain adversarial similarity loss as the domain alignment loss, as shown in Eq. 2. In this process, we introduce an additional domain classifier D to distinguish whether the features output by the feature extractor G come from the source or target domain, while the goal of G is to make it impossible for D to differentiate between them. This two-player minimax game is expected to reach an equilibrium, where the

3.2 Graph-aware Alignment

Existing domain alignment mechanisms typically focus on aligning the global statistical information between the source and target domains, often neglecting the structural information within mini-batch samples. Traditional methods mainly align domains by comparing global features, which may fail to capture subtle relationships, thus limiting the performance of UDA tasks. In UDA, the local relationships and structural information between samples are often more critical than simple global statistics, particularly when faced with complex cross-domain transfers. Ignoring these details can lead to performance degradation. Increasing research has demonstrated that the structural information of data plays a crucial role in UDA tasks. Many advanced methods have successfully incorporated structural information modeling, significantly improving UDA performance. These methods not only focus on global statistical features but also emphasize modeling local data structures, leading to a better understanding and capturing of the complex relationships between the source and target domains [Ma et al., 2019; Zhang et al., 2024a].

To further enhance alignment performance in UDA, especially during deep learning-based training, we propose a novel graph-aware alignment mechanism. This mechanism effectively models the structural information of mini-batch samples from both the source and target domains by introducing a graph structure. Specifically, we use the graph structure to represent the similarity between samples and apply techniques like graph convolution to strengthen structural alignment between domains. This approach enables a finer-grained alignment process, allowing for better capture of the subtle differences between the source and target domains. The method not only improves domain alignment accuracy but also enhances the model's generalization ability in the target domain, particularly when there are significant data distribution differences between the domains, thus reducing negative transfer and improving task performance. Moreover, with the graph-aware alignment mechanism, the model can intelligently adjust and optimize when transferring across different domains, further expanding the application scope of UDA methods. This structural alignment strategy not only enhances the performance of traditional methods in complex tasks but also provides a new approach for broader cross-domain learning challenges. In particular, when data distributions are drastically different, this mechanism can reduce information loss during alignment through fine-grained structural alignment, further boosting the effectiveness and robustness of transfer learning.

In graph domain adaptation, we first use a Data Structure Analyzer (DSA) network to compute structure scores for each mini-batch sample. Then, we combine these scores with the features extracted from the samples through a Convolutional Neural Network (CNN) to construct densely connected instance graphs. Next, we apply a Graph Convolutional Network (GCN) to these instance graphs to learn features that effectively capture data structure information. Before detailing our method, we briefly review the GCN model introduced in [Kipf and Welling, 2016; Ma et al., 2019]. The core goal of GCN is to learn layer-wise propagation operations that can be directly applied to graph structures. Specifically, given an undirected graph with m nodes and edges connecting them, along with an adjacency matrix A ∈ Rm×m, the linear transformation of graph convolution can be represented as the product of the graph signal X ∈ Rk×m and the filter W ∈ Rk×c :

where ˆA = A+ I, I denotes the identity matrix, and ˆDii = ∑j ˆAij. In this formula, the output is a matrix Z of size c ×m. It is worth noting that the GCN can be constructed by stacking multiple graph convolutional layers, as shown in Eq. 3, with a non-linear operation applied after each layer.

Next, we will explain how to construct densely-connected instance graphs for the GCN, and how to derive the graph signal X and adjacency matrix A as shown in Eq. 3. Each node in the instance graph corresponds to the feature of a sample, which is extracted by a standard convolutional network. Thus, the graph signal X can be obtained as follows:

where Xb represent a mini-batch of samples. To construct the adjacency matrix ˆA, we feed the same mini-batch samples into a Data Structure Analyzer (DSA) network to generate structure scores, and then construct the adjacency matrix ˆA based on these scores, as follows:

where X ∈ Rb×d , b is the batch size, and d is the dimension of the structure scores. Then we can obtain the latent variable

Then, the mean vector µ and the variance vector σ are as follows:

Node embeddings Z can be obtained by sampling from the posterior distribution N ∼ (µ,σ). In deep learning models, sampling operations are typically non-differentiable, which prevents direct backpropagation through gradient descent, causing training difficulties. To address this issue, we introduce a random variable ε that follows a standard normal distribution, and transform the sampling process into a differentiable operation. Specifically, the sampling operation is transformed into the following form: Y = ε ∗ σ + µ, where ∗ denotes the dot product. This approach makes the sampling process linear and differentiable, allowing the model to perform backpropagation during training. This not only ensures the operability of the sampling process, but also effectively avoids training issues caused by non-differentiability in traditional sampling methods, significantly improving the model's training efficiency and stability.

Based on the above discussion, we can formalize the inference process achieved through the GCN encoder as follows:

where Zi is the i-th elements of Z,µ and σ are the i-th row of µ and σ, respectively.

An inner product between latent variables is adopted as the decoder to reconstruct the input graph's topological structure, More specifically,

where ˆA is the reconstructed adjacency matrix. δ is the logistic sigmoid function. We define Aij are the elements of A. The generative process can be formalized as,

The initial variational lower bound of GMM in our model consists of reconstruction loss and the KL divergence between the approximate posterior distribution q (Y | X,A) and the initialized prior distribution P (Y) i.e. single Gaussian distribution.

3.3 GaussianMixture Model

The Gaussian Mixture Model (GMM) is a parametric probability density function, represented as a weighted sum of multiple Gaussian component densities. It describes the overall distribution of data by combining multiple Gaussian distributions and is a widely used clustering algorithm in both academic research and industry. Due to its ability to effectively model complex data distributions, we use GMM to partition the feature representations of all classes into different clusters. The formalization of GMM in UDA is described as follows:

where x represents the input feature vector of the data, and wi denotes the mixture weights for each domain, satisfying wi ≥ 0 and ∑Mi=1 wi t = 1. µi,GMM t and Σi,GMM t represent the mean vector and covariance matrix, respectively, of the i-th Gaussian component in the two domains. λ = (µi,GMM ,Σi,GMM ), where M is the total number of Gaussian components. g (x | µi,GMM ,Σi,GMM ) denotes the Gaussian distribution density of the i-th component, where each component is a D-dimensional Gaussian function.

Furthermore, to estimate the parameters of the GMM, we use the Expectation-Maximization (EM) algorithm [Moon, 1996]. The GMM generates M clusters, denoted as Ki , i ∈ 0, 1, . . . ,M - 1, where each cluster represents a set of snapshots exhibiting similar structural and attribute characteristics.

The GMM can be accurately represented by a series of parameters, including the mean vector µGMM , the covariance matrix ΣGMM , and the weights w of the two domains. We update the prior distribution by adjusting the parameters of the GMM to achieve joint iterative training. This approach enables information sharing across multiple domains and optimizes the model parameters in each iteration, allowing it to better adapt to the data distribution of different domains and improve domain adaptation performance.

4 Experiments

4.1 Datasets

Office-31 [Saenko et al., 2010] is a widely used cross-domain dataset, particularly applied in office environments, consisting of 31 classes of images from three distinct domains: Amazon (A), Webcam (W), and DSLR (D). The dataset is imbalanced in terms of the number of images, with 2,817 images from Amazon, 795 images from Webcam, and 498 images from DSLR.

Office-Home [Venkateswara et al., 2017] is a dataset comprising 15,500 photos from both home and business settings, organized into 65 categories. The dataset consists of four distinct domains: Product (Pr), Clipart (Cl), Real-World (Rw), and Art (Ar). These categories include real-world camera captures, product photos, clipart images, and artistic renderings, respectively.

VisDA-2017 [Peng et al., 2017] contains over 280,000 images across 12 categories and presents a challenging benchmark for unsupervised domain adaptation (UDA). In our experiments, we use the validation set as the target domain and the training set as the source domain.

DomainNet [Peng et al., 2019] is a large-scale domain adaptation dataset with over 590,000 images spanning six domains: Painting (pnt), Infograph (inf), Real (rel), Quickdraw (qdr), Clipart (clp), and Sketch (skt). Each domain contains items from 345 categories and is the largest image benchmark for domain adaptation currently available.

4.2 Implementation Details

We use PyTorch to implement our technique, and the backbone network for all datasets is ResNet [He et al., 2016], which has been pre-trained on ImageNet [Russakovsky et al., 2015]. We used Pytorch [Paszke et al., 2019] to implement all of the experiments. Our Stochastic Gradient Descent (SGD) algorithm is used with momentum set to 0.9 and weight decay set to 0.001. For model optimization, we follow the learning rate annealing technique described in [Ganin et al., 2016]. In this study, we specifically specify λ = 0.5, γ = 1.0 and perform a sensitivity analysis to find out how hyper-parameter selection affects the results. We present the average accuracy from three random trials for each task. We compare with the recent popular methods,e.g.,ResNet-50 [He et al., 2016],DDC [Tzeng et al., 2014],DDC [Tzeng et al., 2014],DAN [Long et al., 2015],DANN [Ganin et al., 2016],SimNet [Zhang et al., 2019],CyCADA [Hoffman et al., 2018],CDAN [Long et al., 2018],TADA [Hung et al., 2023],BSP [Chen et al.,

2019],MCD [Saito et al., 2018], T2DA [Li et al., 2021c], SAFN [Xu et al., 2019],ADDA [Tzeng et al., 2017], MIMTFL [Braytee et al., 2022], MDD [Li et al., 2020], SCDA [Li et al., 2021b], CDAN+InterBN [Wang et al., 2021].

4.3 Results

We present our results in Table. 1 to Table. 4. Our GMM method significantly enhances the performance of the compared state-of-the-art methods, achieving average accuracy improvements of 4.5%, 4.4%, 5.1%, and 9.0% on the Office31, Office-Home, VisDA-2017, and DomainNet datasets, respectively. Notably, we observe substantial accuracy gains on some of the most challenging tasks, such as in the Office-31 dataset, where D→A improved from 78.2% to 83.5% and W→A improved from 78.5% to 86.7%. On the largest dataset, DomainNet, our GMM method also demonstrates impressive performance with a 9.0% increase compared to CDAN+InterBN. Importantly, across all four benchmark datasets, our proposed method outperforms all state-ofthe-art approaches. Additionally, the absence of adversarial networks in our method further underscores its effectiveness.

4.4 Empirical Analysis

Distribution Discrepancy. The A-distance [Ben-David et al., 2010] is a widely used metric for measuring the distance between two distributions, with a larger A-distance indicating a greater distinction between the source and target domains. However, the complexity of directly computing the A-distance prompts us to use the proxy A-distance, ˆ dA, defined as ˆ dA = 2(1 - 2ε). In our analysis, we compute this proxy A-distance for various models, including ResNet, CDAN, T2SA, and GMM, specifically focusing on the feature representations for the A→W task in Office-31. The results, shown in Figure 3(a), reveal that GMM has the lowest proxy A-distance among the evaluated methods.

Hyper-parameter Sensitivity. To further validate the effectiveness of these hyper-parameter settings, we conducted additional experiments under various conditions. Specifically, we tested the performance of GMM on multiple datasets with different characteristics, including varying sizes, noise levels, and data distributions. The results consistently showed that the optimal hyper-parameter combination of λ = 0.5 and γ = 1.0 maintained superior performance across these diverse scenarios. Furthermore, we compared GMM with several state-of-the-art methods under the same experimental settings. Overall, these findings suggest that GMM is a reliable and efficient approach for the given task, and the identified optimal hyper-parameter settings can be applied broadly to achieve consistent and high-quality results.

5 Conclusions

In this paper, we propose a novel method called Gaussian mixture model for graph domain adaptation. Our approach leverages the flexibility of GMM to model multi-modal data distributions, effectively capturing the inherent complexity and heterogeneity of real-world data. By integrating GMM into the UDA framework, we enhance clustering performance and improve domain alignment, thereby reducing the risk of negative transfer and improving the robustness and scalability of domain adaptation.

Acknowledgments

This work is supported by the National Natural Science Foundation of China under Grants No. 62406100, Tianjin Natural Science Foundation under Grants No. 24JCQNJC00320, Beijing Postdoctoral Research Foundation.