Dynamic Weighted Graph Fusion for Deep Multi-View Clustering

1 Introduction

Clustering represents a fundamental task in unsupervised learning and has been widely utilized across various domains, such as biology [G¨onen and Margolin, 2014], psychology [Burr et al., 2022], information retrieval [Bruno and Marchand-Maillet, 2009], etc. Conventional clustering methods typically work with single-view data [Peng et al., 2019]. However, with the advance of multimedia technology, real data are frequently collected from multiple sources or described from different perspectives, known as multi-view data. Numerous studies [Wang et al., 2023; Wang et al., 2022a; Zhang et al., 2015; Han et al., 2022; Cui et al., 2024; Wen et al., 2024] have shown that multiview data contains complementary information that could help the model to learn more comprehensive and more expressive representations. In the literature, multi-view clustering (MVC) has been extensively studied at both shallow and deep levels. Shallow MVC approaches include 1) cotraining approach [Nie et al., 2020; Hu et al., 2020], which seeks to maximize the mutual agreement across multiple views; 2) multi-kernel approach [G¨onen and Margolin, 2014; Lu et al., 2014]

∗Corresponding author.

views and linearly or non-linearly integrates them; 3) subspace clustering approach [Li et al., 2019a; Zheng et al., 2020], which aims at exploring shared representations of multiple views; and 4) graph-based approach [Tang et al., 2020; Lin and Kang, 2022], which attempts to deploy the relations between different instances with certain distance metrics or similarity. Deep MVC approaches include 1) deep embedded clustering (DEC) based approach [Xu et al., 2023], which adopts autoencoders to learn low-dimensional embeddings [Yang et al., 2021; Li et al., 2020]; 2) generative adversarial network (GAN) based approach [Zhou and Shen, 2020; Li et al., 2019b], which uses adversarial training to capture data distribution; and 3) graph neural network (GNN) based approach [Fan et al., 2020; Xia et al., 2021], which applies GNN to take full advantage of the characteristics of multiview graph data. In this study, we focus on DEC and GNN based MVC, as it allows to exploit the principles of consistency and complementarity as well as the topological information hidden in multi-view data.

Most existing graph-based MVC methods [Chen et al., 2022] often require explicit graph data as input, real-world data often lacks the original graph structure. Therefore, graph-based MVC methods [Wang et al., 2022a] with unknown graph structure is emerging in recent years, which consider graph structures in multi-view data by constructing local or global graphs. However, most existing methods suffer from the drawback that the constructed graphs are not clustering task-oriented. They also generally ignore the local structures of all views, and the constructed graph structure information is not properly utilized.

To address the above issues, in this paper we propose dynamic weighted graph fusion for deep multi-view clustering (DFMVC). DFMVC firstly learns the embedded features via deep autoencoders and constructs local latent graphs for multiple views seperately. Then, the embedded features of all views are concatenated to create a global feature to leverage complementary information and a fusion graph is generated via fusing all latent graphs to capture the topological information among samples.

The proposed DFMVC incorporates global features and the fusion graph as inputs to the graph convolution module, which serves to generate pseudo-label information for selfsupervision. This valuable information acts as a supervisory signal for all views, facilitating the learning of embedded features and latent graphs. To ensure the continual refinement of the pseudo-label information, these updated embedded features and latent graphs are cyclically utilized. Furthermore, in order to enhance fusion process and enhance the quality of the fused graph, a meticulous weighted graph fusion scheme is devised. This scheme effectively discerns and accounts for the distinct contributions made by each individual view.

In general, DFMVC effectively enhances the clustering performance by comprehensively considering both the latent sample attributes and the intrinsic topological structure of all views. Moreover, by refining the fusion graph dynamically, even if the learned graph structure is not suitable for clustering at the beginning. With the aid of all views' complementing information, it will be steadily optimized. The main contributions of this work are as follows:

• We propose a novel deep MVC framework by exploiting dynamic graph fusion. It utilizes both global features, offering complementary information, and latent graphs, which provide topological information, to enhance deep multi-view clustering capabilities.
• In a self-supervised manner, we dynamically update the fusion graph and pseudo-label information such that constructed graph structure favors clustering task.
• We design a weighted graph fusion scheme that adaptively evaluates the weight of each view to enhance the quality of graph fusion. Extensive experiments on realworld multi-view data demonstrate the superior performance of our proposed model.

2 RelatedWork

2.1 Multi-ViewClusteirng

By utilizing the complimentary information among multiview data, multi-view clustering can enhance the clustering performance. The core problem is how to make full use of the information among different views and give full play to each view's advantages while reducing each view's limitations to achieve accurate and robust clustering.

Bickel et al. [Bickel and Scheffer, 2004] used the strategy of stitching to connect features of multiple views into a new feature space in which traditional clustering algorithms are executed to achieve better clustering performance, leading to a series of works [Zheng et al., 2020; Yu et al., 2021]. However, this type of method ignores the complementary information hidden in multi-view data. To this end, MVC methods based on representation learning emerged, which can be further divided into multi-view graph clustering [Kang et al., 2020], multi-view subspace clustering [Zheng et al., 2020], and nonnegative matrix factorization (NMF) based multiview clustering [Ren et al., 2020].

Nowadays, MVC has also become a hot research topic in deep learning, and numerous deep MVC models [Xu et al., 2021b; Lin et al., 2022] have been proposed. Using autoencoders to learn low-dimensional feature representations for multi-view clustering [Chen et al., 2020; Xu et al., 2021a] allows for adequate learning of representative latent features, and these studies have yielded impressive results.

2.2 Graph-based DeepMulti-ViewClustering

Graph neural networks (GNNs) are widely used in unsupervised learning due to their powerful representation capabilities on graph-structured data, and many graph-based deep MVC models have emerged [Tang et al., 2020; Li et al., 2021; Wang et al., 2022b]. The graph convolution network (GCN), an important branch of GNN, which can considerably enhance the effectiveness of the clustering algorithm by extracting data from the properties of the graph and the nodes, attracts people's increasing attention [Ren et al., 2022; Du et al., 2023; Ren et al., 2024].

Many researchers have considered the explicit presence of graph structure in multi-view data, and constructed local and global graphs to perform multi-graph learning and consensus clustering [Zheng et al., 2021; Hao et al., 2021]. For example, Wang et al. [Wang et al., 2022a] used k-nearest neighbors (kNN) to construct the attribute graph structure for multi-view source data and then obtained attention weighted fusion graph based on graph encoders and attention network to complete the graph reconstruction and clustering tasks. Ma et al. [Ma et al., 2022] construct multiple learned graphs using multi-view data. This approach aims to effectively capture the intricate local manifold structures that are concealed within each view. By generating high-quality graph structures and consensus graphs, these components mutually influence and guide one another, leading to iterative refinement and improvement in the overall analysis process.

Although the above methods improve the MVC performance by constructing multi-view graphs and fusing graph structure, they typically ignore the latent graphs of different views. To address this, Fan et al. [Fan et al., 2020] pioneered the application of the Graph Convolutional Network (GCN) architecture to tackle a challenging multi-view clustering task. This novel approach incorporates a GCN encoder along with multiple graph decoders, enabling the effective encoding of multi-view attribute graphs into a low dimensional feature space. By leveraging this framework, the encoded representations capture essential information from diverse views, facilitating comprehensive and robust clustering of the data.

Xia et al. [Xia et al., 2021] used the Euler transform to extract features as view descriptors, and then performed representation learning and clustering by constructing a multiview self-supervised graph convolutional clustering network. However, the graphs in existing GCN-based MVC methods are typically fixed, which makes the predefined/constructed graphs a major determinant of clustering performance.

In conclusion, although a great deal of GCN-based MVC models have made encouraging progress, they still have inherent shortcomings. First, since the graph information is usually not given, GCN models that require explicit graph data as input are not practical in real-world applications. Second, the graphs in GCN models are typically fixed, which may not favor the clustering task. By contrast, our proposed model dynamically constructs the graph data by embedding the features of multi-view data, which is suitable for MVC problem without explicit graphs. In addition, we design a dynamic weighted graph fusion strategy to address the graph fixation issue, further improving the practicality and robustness of the MVC model.

3 ProposedMethod

Given a multi-view dataset X = {Xv ∈ RN×dv }Vv=1 , where Xv = [xv1 ; xv2 ; . . . ; xvN ] denotes the v-th view's data, N is the number of instances, V is the number of total views, and dv is the dimension of the v-th view. The goal of dynamic weighted graph fusion for deep multi-view clustering (DFMVC) is to partition the instances into K clusters. Figure 1 provides an overview of our proposed DFMVC framework.

3.1 Dynamic Graph Generation

To learn the representative latent features, we use the deep autoencoder [Rumelhart et al., 1986] to project the raw data of each view to a low-dimensional space. For the v-th view, f v θv and gvϕv denote the encoder and decoder respectively, where θv and ϕv are learnable parameters. The embedded feature can then be indicated aszv i = f v θv (xvi ), (1)

where zv i ∈ Rl denotes the l-dimensional embedded feature of xvi (l is the same for all views). After that, zv i is decoded as ˆxvi through the decoder gvϕv , and the reconstruction loss of all the views can be formulated as

To obtain expressive graph structure representation, we dynamically generate a graph in each view's latent space. It is mainly because when composing fixed graphs based on raw data [Wang et al., 2021; Tang et al., 2020], graphs constructed cannot be adjusted according to the clustering results. On the contrary, as the features extracted by the deep autoencoder are more representative than the raw features, we construct graphs in the latent space. In addition, during the dynamic update process, the fusion of graphs and the acquisition of pseudo-label information are carried out alternately and promote each other. Furthermore, unlike [Cheng et al., 2021; Ren et al., 2022] that necessitate explicit graph data as input, our approach introduces the capability to dynamically construct graphs within the latent feature space. This advantageous characteristic allows us to overcome the limitations posed by a scarcity of real-world data with graph structures.

Therefore, we choose to generate graph information in the latent feature space and update it dynamically.

Specifically, we define the latent graph Gv = {Z v ,Av } for the v-th view, where Z v = [zv 1 ; zv 2 ; . . . ; zvN ] denotes the node properties of graph Gv . Av is the adjacency matrix and Avij denotes the edge between the samples i and j in the v-th view. Samples i and j are connected if Avij = 1 and disconnected otherwise. We utilize the kNN graph [Peterson, 2009]

to construct Av :

where N v i is the set of k nearest neighbors of zv i in the v-th view according to the Euclidean distance in the latent space.

3.2 Graph Fusion with AdaptiveWeight

Our motivations for designing the weighted dynamic graph fusion mechanism are:

• The latent graphs of different views contain the discriminative and complementary information. Therefore, fusing these multi-view graphs into a more informative and robust global graph will be beneficial to the clustering task.
• Due to the noisy issue and the incompleteness of raw features, the generated graphs in certain views may not correctly reflect the actual topological structure among samples. Thus it is essential to adaptively assign different weights to the graphs with different qualities.

Building upon the reasons mentioned above, we approach dynamic graph fusion as follows:

Inspired by [Kang et al., 2020], we use the inverse distance of fusion graph A and each latent graph Av to obtain wv. In our model we assign the value of the exponential parameter of the inverse distance weighting method to 2 and then wv can be adaptively computed as

With the above weighting approach, a latent graph with high quality (which is close to the fusion graph) will be assigned a high weight. By this way, the influence of graphs reflecting clear topological structure and complementary information will be enhanced and correspondingly the graphs containing unclear information will play an inconsequential role in the cluster assignment.

Based on the obtained graph weights, in each iteration, the proposed DFMVC updates the fusion graph colmun-wisely through the following precedures:

By computing the derivative of Eq. (6) and setting it to zero, it yields

The graph A is a fusion of [A1 , ...,Av ] which is supposed to capture the ground-truth sample similarity hidden in the multi-view data. [A1 , ...,Av ] are generated by embeddings from each view, they may not be optimal for the clustering result at first. At the beginning of the training, latent graphs from different views are given the equal weight. Then, the graph that is close to the fusion graph should be assigned a large weight. A relative favourable fusion graph can be learned through weight update. Through the dynamic graph fusion, our method progressively obtain more comprehensive information and clearer clustering structure from the latent features as training process forwards. Meanwhile, by assigning smaller weights to the less reliable views, the negative affect of noise graphs is reduced effectively and thus the clustering performance can be further promoted.

3.3 Graph Convolution

To enhance the clustering results with both latent features and topological information, we further utilize the GCN structure to obtain better sample representations based on the fusion graph and global features.

Concretely, as the embedded features of different views are learned independently and each view can provide specific and complementary information, we concatenate the embedded features across all the views to generate the global feature zi = [z1 i , z2 i , . . . , z V i ] ∈ R1×∑Vv=1 dv . Let Z = [z1; z2; . . . ; zN] ∈ RN×∑Vv=1 dv denote all the global features. To leverage the graph structure information, we introduce a two-layer GCN module for dynamic fusion graph. By aggregating the multi-order neighbourhood information with stacking layers, this GCN module guides the learned common representation maintaining the adjacent relationship in each view. As the GCN module directly performs graph convolution on the global feature and dynamic fusion graph, the refined representation ˜ z is obtained by

where ˜ A = In +A, ˜Dii = ∑j ˜ Aij. Here, the number of layers of the graph convolution is always set to 2. Through this GCN module, the dynamic fusion graph term seeks to find the underlying relationships from samples that make the cluster structure explicit. Compared with other existing GCN-based methods, our adopted simple yet effective GCN module has better generalization capability.

3.4 Self-Training Clustering

Inspired by a popular single-view deep embedded clustering (DEC) method [Xie et al., 2016] that applies Student's t-distribution [Van der Maaten and Hinton, 2008] for selftraining, we construct clustering layer cvµv , where µv represents the learnable cluster centroids. For the v-th view, let Qv = [qv 1 ; qv 2 ; . . . ; qvN ] ∈ RN×K denote the cluster assignments of all samples, where qv ij is the probability that the i-th sample belongs to the j-th cluster. qv ij can be considered as a soft cluster assignment and is obtained by:

After obtaining the refined global features ˜ z via graph convolution module (Section 3.3), we adopt K-means [Mac- Queen, 1967] to compute the cluster centroids cj:

Then, the soft pseudo assignment tij between each global embedding and each cluster centroid with Student's tdistribution is defined as

In order to enhance the discriminability of the pseudo soft assignments, the global target distribution P can be computed by

The clustering loss for the v-th view is determined by the KL divergence between the unified target distribution P and the cluster assignment distribution Qv :

To learn an accurate assignment for clustering, it is necessary to introduce an integrated objective to guide the learning process. With the aim of achieving this goal, we undertake a joint optimization approach, simultaneously optimizing the deep autoencoder embedding and clustering learning. Thus, the overall objective function of DFMVC is defined as:

where 0 < τ < 1 is a trade-off coefficient controlling the degree of distorting embedded space, and is always set as τ = 0.5 for all experiments (which means Lvr and Lvc are equally important). Minimizing KL divergence between Q and P makes the distribution of Q sharper and mine the information in different views. Global fusion can iteratively lead all views to learn more comprehensive information. After obtaining soft cluster assignments from multiple views, the highly confident predictions will guide the training process, which can also prevent the disruption caused by a few erroneous predictions. When DFMVC stops, the final clustering prediction can be obtained by:

3.5 Optimization

The detailed optimization procedure is described in Algorithm 1.

First, we pretrain autoencoders f v θv and gvϕv of all views to obtain θv and ϕv by minimizing the reconstruction loss in Eq. (2). After that, we use K-means to initialize the cluster centroids of each view, and wt is initialized to 1 V . Then the fusion graph is obtained by Eq. (7). In the training process, Algorithm 1 Dynamic weighted graph fusion for deep multiview clustering (DFMVC)

Input: multi-view dataset X, cluster number K.

Initialization: {wv}Vv=1 ← 1 V .

Obtain {θv , ϕv , µv , Av }Vv=1 by pretraining autoencoders.

while not reach the maximum iterations T1 do Update P according to Eqs. (10) - (12).

while not reach the maximum iterations T2 do

Update {θv , ϕv }Vv=1 according to Eq. (14).

end while

Update {wv}Vv=1 according to Eq. (5).

Update A according to Eq. (7).

end while

Output: Cluster assignment y according to Eq. (15).

with the advantage of dynamic latent graph fusion in discovering clearer graph structure representation and global information, the target distribution P obtained by fusion graph can better improve the cluster distribution of data and robustness with adaptive weight wv. In addition, the weight of of each latent graph is updated iteratively during the training process.

4 Experiments

4.1 Experimental Setup

Datasets The following three real-world multi-view datasets are tested in our experiments. BDGP [Cai et al., 2012] contains 2500 samples of 5 different types of drosophila embryos. Each sample has two views corresponding to the visual and textual features. Fashion-MV [Xiao et al., 2017] collects images from 10 categories, i.e., t-shirt, trouser, pullover, dress, coat, sandal, shirt, sneaker, bag, and ankle boot. Following the literature [Xu et al., 2021b], we use 30000 samples to construct Fashion-MV. Three viewpoints of a single occurrence are represented by each of the three photographs picked from the same category. Handwritten Numerals (HW) encompasses 2000 samples distributed across 10 classes, each representing numerals ranging from 0 to 9. Each category comprises six distinct visual views. The details of datasets are shown in Table 1.

Comparison Methods The comparison methods include three single-view clustering methods: K-means [MacQueen, 1967], SC (spectral clustering [Ng et al., 2001]), DEC (deep embedded clustering [Xie et al., 2016]), and seven state-ofthe-art MVC methods: GFSC (multi-graph fusion for multiview spectral clustering [Kang et al., 2020]), GMC (graphbased multi-view clustering [Wang et al., 2019]), SAMVC (self-paced and auto-weighted multi-view clustering [Ren et al., 2020]), DEMVC (deep embedded multi-view clustering with collaborative training [Xu et al., 2021a]), SIMVC & COMVC (reconsidering representation alignment for multiview clustering [Trosten et al., 2021]), SDMVC (selfsupervised discriminative feature learning for deep multiview clustering [Xu et al., 2023]).

Implementation Details Across all datasets utilized, we employ an identical fully connected (Fc) autoencoder structure. Specifically, for each view, the encoder structure comprises the following layers: Input - Fc300 - Fc1000 - Fc2000 - Fc10. The decoders mirror the respective encoders of their corresponding views. To facilitate pre-training, all autoencoders undergo 3000 epochs. For our method, the trade-off coefficient τ is set at 0.5, while the kNN graph algorithm employs a value of 30 for the number of neighbors (k). It is worth noting that our approach necessitates constructing a graph encompassing all nodes, with the batch size equal to the total number of instances (N). For the comparing methods, we use the source codes provided by the authors with suggested parameter settings.

4.2 Clustering Results

Comparison with Baselines The quantitative comparison of DFMVC and baseline models is shown in Table 2 (raw datasets). The best result is shown in bold in each column, and the second-best result is underlined. We can observe from Table 2 that DFMVC achieves the best across all multi-view datasets on three evaluation methods. Especially on FashionMV, DFMVC makes more remarkable improvements than existing methods. This is mainly because by dynamically updating the weighted fusion graph, DFMVC can effectively capture the explicit global information and avoid the influence of low-quality natural features in one view, such as noise or incompletion.

The graph-based shallow multi-view clustering methods achieve better performance, indicating that graph structure information can make beneficial contribution to clustering.

Ablation Studies To further verify the contribution of each module in the proposed method, we conduct ablation studies on the Eq. (4) and Eq. (8), which represent dynamically updating fusion graph and training with graph structure respectively. Table 3 shows the clustering results with different variants included, where w/o means that variant is removed from the method. Concretely, (A) is optimized without graph structure, which achieves the basic purpose of learning the consistency. (B) is optimized with fusion graph but without updating dynamically.

From the results, (B) performs better than (A) indicating that fusion graph structure contains more accurate global information to enhance the latent representation. Besides, we can also find that the full version of the proposed model (C) performs better than (B). The reason is that with the dynamic weighted graph fusion mechanism, our method obtains a clearer and more robust graph structure to mine the global information accurately.

We also conduct ablation studies on the noisy dataset. The experiment shows the excellent performance on noisy datasets illustrates the effectiveness of dynamic update process to deal with the incompleteness of raw features (half the feature values are zero in one view). The specific experimental results are presented in the appendix.

Visualization of Learning Process We visualize the learning process of embedded features on BDGP via t-SNE [Van der Maaten and Hinton, 2008]. The color denotes the label of each sample. After the pretraining of autoencoders is finished, the embedded features of the two views of BDGP are shown in Figure 2. At the beginning of finetuning stage (T = 0), the embedded features are non-separable. However, as the training process forwards. As the embedded features progressively separate, the clustering structures become more clearer, coinciding with the gradual dispersion of their respective centroids, which visually demonstrates the effectiveness of the proposed DFMVC.

Parameter Sensitivity Analysis The main hyperparameters of the proposed DFMVC are the trade-off coefficient τ in Eq. (14) and the number of neighbors k in constructing the kNN graph. We test the average clustering performance of twenty independent runs on BDGP dataset and HW dataset. From Figure 3, we can observe that assigning k with a too small value will lead the graph become too sparse and thus lost some of graph structure information. As a consequence, with less information can be aggregated, the representation capability of the GCN module will be

τ

τ

Figure 3: Clustering performance w.r.t. different parameter values on BDGP dataset and HW dataset.

weaken. However, that does not mean an extremely larger k is better, since it will bring more time complexity to the composition and may introduce unnecessary noise. Besides, it is shown that the too large and too small values of the trade-off parameter τ also negatively impact on the clustering performance. This indicates that both the reconstruction loss and clustering loss in Eq. (14) are important for the training of DFMVC. The suggested default parameter values are k = 30 and τ = 0.5.

5 Conclusion

In this paper, we propose a dynamic weighted graph fusion framework for deep multi-view clustering (DFMVC), which effectively leverages both global features and latent graphs to enhance the clustering performance. Specifically, DFMVC firstly learns embedded features via deep autoencoders and then constructs latent graphs for each individual view. After that, it captures the complementary node attributes and topological information from different views by concatenating the embedded features of all views as well as fusing all the latent graphs. Extensive experiments on multiple real-world multi-view datasets and their noisy versions demonstrate the superior effectiveness and robustness of the proposed model. Since the computing complexity is a major issue of graphbased models, it is interesting to explore the deep multi-view graph clustering on large-scale datasets to further enhance our model's efficiency by utilizing techniques such as anchor graphs or encoding meaningful graph-related information.

Acknowledgments

This work is supported in part by Shenzhen Science and Technology Program under grants JCYJ20230807115959041 and JCYJ20230807120010021. Lifang He is partially supported by the NSF grants (MRI-2215789, IIS-1909879, IIS2319451), NIH grant under R21EY034179, and Lehigh's grants under Accelerator and CORE.