Multi-view Clustering via Multi-granularity Ensemble

1 Introduction

Multi-view clustering integrates complementary information from various modalities (e.g., images, texts, sensor readings) to produce more accurate and robust clustering [Chao et al.,

∗Corresponding author.

2021; Jiang et al., 2025b]. However, effectively leveraging multi-view information remains challenging due to inconsistencies in data quality, feature distributions, structural heterogeneity among views, and issues such as missing or redundant views [Jiang et al., 2025a]. Multi-view clustering can be broadly categorized based on their fusion strategies: (1) representation and graph-based fusion methods [Xu et al., 2023; Wang et al., 2024b; Huang et al., 2022; Huang et al., 2024], which construct a unified feature representation or similarity graph for clustering (commonly referred to as early fusion), and (2) ensemble-based methods [Pfeifer et al., 2023; Zheng et al., 2024; Xu et al., 2024], which combine clustering labels from different views to form a consensus result (referred to as late fusion). Some hybrid approaches combine feature-level and label-level integration for improved performance. Despite substantial progress, these categories face challenges such as information loss due to reliance on singlegranularity labels [Wang et al., 2025], sensitivity to heterogeneous view inconsistencies, and the need for manual tuning of view weights[Zhang et al., 2024a; Zhang et al., 2024b; Sun et al., 2025].

Representation and graph-based fusion methods aim to learn a shared representation or construct a consensus graph by aggregating information from multiple views [Chen et al., 2024b; Wen et al., 2024; Chen et al., 2024a; Huang et al., 2023b]. For instance, the consensus graph learning framework builds a robust consensus similarity graph by integrating spectral embeddings with a weighted tensor-based lowrank representation [Li et al., 2021]. The summarized multiview clustering approach reduces redundancy and enhances inter-view consistency by leveraging an information-theoretic variational lower bound [Cui et al., 2024]. Similarly, the robust multi-view clustering with noisy correspondence method employs a noise-tolerant contrastive loss to learn embeddings that remain robust even with misaligned views [Sun et al., 2024]. The multi-level feature learning framework for contrastive multi-view clustering independently learns low-level and high-level features, avoiding direct feature fusion and ensuring that private, view-specific information does not interfere with shared representations [Xu et al., 2022]. However, Data Multi-view and Multi-granularity Labels these methods often involve complex representation or graph alignment processes, which pose challenges for highly heterogeneous views, and they rely heavily on hyperparameter tuning, such as adjusting view weights and embedding dimensions, to achieve optimal performance [Wu et al., 2024; Lou et al., 2024; Liang et al., 2024; Wu et al., 2025].

Ensemble-based multi-view clustering methods aim to combine the strengths of multiple base clustering results to achieve robust and consistent consensus outcomes [Liang et al., 2025]. For example, the Parea hierarchical clustering ensemble framework leverages late-stage fusion to integrate clustering solutions from heterogeneous biomedical datasets, enhancing disease subtype discovery [Pfeifer et al., 2023]. The low-rank and sparse decomposition approach formulates ensemble clustering as a tensor decomposition problem, capturing high-order correlations across views [Zhang et al., 2023]. Similarly, the hybrid multi-view clustering ensemble method employs diverse view transformations and hybrid subspace learning to enhance the diversity of base clusterings [Yu et al., 2020]. The Fast Multi-view Ensemble Clustering (FMVEC) approach introduces a hybrid earlylate fusion strategy with random view groups, achieving nearlinear time complexity for large-scale datasets [Huang et al., 2023a]. However, most ensemble-based methods rely on single-granularity clustering labels, which often fail to capture critical hierarchical structures and limit their ability to learn more discriminative representations. Additionally, directly aggregating clustering labels into a final consensus label set can amplify errors, particularly when individual viewspecific labels are noisy, undermining their robustness and effectiveness [Wang et al., 2024a].

In this paper, we propose a novel multi-view clustering framework called Multi-granularity Ensemble (MGE), which addresses the limitations of both representation/graph fusionbased and ensemble-based methods by incorporating multigranularity clustering labels and constructing a co-association similarity matrix that encodes rich multi-view and multi-level information. Specifically, a constrained hierarchical clustering is introduced and exploited to each view and their fused view to generate multi-granularity labels that capture both fine-grained and coarse-grained cluster structures. The cross-view and cross-granularity fusion is then performed on these labels to construct a co-association matrix, encoding discriminative representations across local and global, multigranularity, and multi-view levels. As a result. MGE effectively avoids the challenges of heterogeneous view alignment and view-weight hyperparameter tuning that are typically required in representation/graph fusion-based methods. Moreover, it mitigates the impact of noisy labels from individual views by avoiding direct label aggregation and instead employing secondary clustering on the collaborative representation. Compared to existing ensemble-based methods, MGE captures richer local and global clustering structures by integrating multi-granularity information, ensuring more discriminative representation learning. Figure 1 illustrates the basic framework of MGE and the main contributions of this paper are summarized as follows:

• We propose a novel multi-view clustering framework, i.e., Multi-Granularity Ensemble (MGE), which effectively addresses the inherent information loss caused by reliance on single-granularity labels in existing methods, achieving more robust and accurate clustering results.
• MGE tackles the challenge of heterogeneous views existing in multi-view data, which effectively explore the structure information from varying data distributions without requiring explicit view alignment, enabling more accurate and robust clustering.
• We enhance representation learning via the cross-view and cross-granularity fusion that seamlessly integrates fine-grained and coarse-grained clustering structures, such that the local and global patterns across multiple views are simultaneously considered, further improving the discrimination and robustness of representations.

2 The Multi-granularity Ensemble Framework

2.1 Multi-view Multi-granularity Label Generation

To comprehensively capture fine-grained and coarse-grained clustering structures across multiple views, the proposed MGE framework introduced the Constrained Hierarchical Clustering (CHC) [Yang and Lin, 2024] to accommodate multi-view data, generating multi-granularity labels for each view and the fused view. Let X = {X1,X2, · · · ,Xv} be the multi-view dataset, where Xi represents the i-th view, and v denotes the number of views. The fused view Xf is defined based on the average of the similarity matrices Si derived from each view:

where Si is the similarity matrix corresponding to the i-th view. The matrix Sf serves as the similarity matrix for the fused view Xf , preserving cross-view consensus information.

To generate multi-granularity label sets for each view Xi (including Xf ), CHC is applied to produce Li = {Li,1,Li,2, . . . ,Li,ki }, where Li,j represents the clustering labels at the j-th granularity level for view i. The labels are ordered such that Li,1 corresponds to the finest partition (with the most clusters), and Li,ki represents the coarsest partition (with fewer clusters). The total number of clustering levels ki is determined based on a granularity termination parameter λ (discussed below).

Constrained Hierarchical Clustering

Hierarchical clustering (HC) constructs dendrograms to generate multi-granularity labels, enabling flexible exploration of data structures. However, traditional HC methods, such as single, complete, and average linkage, rely solely on 1nearest neighbor statistics, often merging sub-clusters from different categories, leading to low-purity multi-granularity labels. CHC addresses these limitations by introducing adjacency constraints, prioritizing sub-clusters that align with ground-truth structures, and using larger clusters to guide smaller ones. As illustrated in Figure 2, for the current clusters A, B, and C, a parallel merge based on 1-nearest neighbor relationships would result in A merging with B, and B merging with C. In contrast, CHC applies the constraint that the size of A is larger than that of B, no merge them; whereas the size of C is smaller than B, merge them, avoiding A merging with B while allowing B merging with C. This perfectly matches the ground truth and significantly improves the purity of the merging process at each iteration.

For each view Xi (including Xf ), CHC initializes each sample as its cluster. A graph G = (V,E) is constructed, where V represents the set of clusters (i.e., nodes), and an edge (Ci, Cj) ∈ E exists if cluster Cj is the 1-nearest neighbor (denoting as 1NN) of cluster Ci and |Ci| ≤ |Cj|. The adjacency matrix A is defined as:

where I(·) is the indicator function that returns 1 if the condition is true and 0 otherwise. The connected components of the graph G after each iteration represent the clusters at a specific granularity level. By iteratively merging clusters based on adjacency constraints, CHC produces hierarchical partitions corresponding to various levels of granularity.

When a target cluster number K is specified, CHC constructs a hierarchy and removes the K - 1 strongest edges from the graph G. The weight of each edge is computed as:

where d2 (Ci, Cj) denotes the squared distance between clusters Ci and Cj, and |Ci| represents the sizes of the clusters.

The K-1 edges with the highest weights are removed to disconnect the graph into K connected components, resulting in precisely K clusters.

Granularity Termination Hyperparameter λ

For each view Xi (including Xf ), CHC generates mi clustering partitions ranging from fine-grained to coarse-grained labels. The granularity termination parameter ki for view i determines the number of labels selected from the hierarchy:

where λ ∈ (0, 1] is a shared hyperparameter across all views, ensuring consistency in the number of clustering levels. Here, ⌊·⌋ denotes the rounding operation to the nearest integer. A smaller λ results in fewer clustering partitions (i.e., lower ki), corresponding to a coarser analysis. Conversely, a larger λ produces more clustering partitions (i.e., higher ki), corresponding to a finer breakdown with more detailed clustering levels.

By performing CHC to each view Xi and the fused view Xf , the proposed framework generates multi-granularity label sets Li = {Li,1,Li,2, . . . ,Li,ki }, where Li,ki corresponds to the clustering partition at the granularity level determined by ki. This ensures that both fine-grained and coarsegrained clustering structures are captured, enabling richer multi-view and multi-level representations for the subsequent integration process.

2.2 Cross-view and Cross-granularity Fusion

Previous ensemble methods, such as the Ensemble Learning via Propagation of Cluster-wise Similarities (ELPCS) [Huang et al., 2018], are restricted to single-view contexts, integrating labels of the same granularity from different algorithms. This design overlooks the complementary information inherent in multiple views and the enriched discriminative power provided by multi-granularity labels, thereby limiting clustering performance. To overcome these limitations, we extend ELPCS to support multi-view and multi-granularity data by introducing a cross-view and cross-granularity fusion strategy. This strategy facilitates a seamless global interaction between views and granularities, synergizing complementary insights and hierarchical structures to extract the most discriminative representations. The resulting co-association matrix effectively captures intricate relationships across views and granularities, serving as a robust foundation for accurate and reliable consensus clustering.

Cluster-wise Similarity Graph Construction

Given the multi-granularity label Li = {Li,1, · · · , Li,ki } for each viewXi (including the fused viewXf ), we construct a combined cluster-wise similarity graph G = (V, E), where: V represents the clusters across all views and granularities, and E represents edges weighted by the Jaccard coefficient:

where Cp and Cq are clusters from different views and/or different granularities, reflecting the proportion of overlapping data points. In this combined graph, nodes correspond

Algorithm 1 MGE framework

Input: Multi-view data X = {X1, · · · ,Xv}, and the cluster number c, and the granularity control parameter λ;

• 1: for each viewXi (including the fused viewXf );
• 2: Apply CHC to Xi to generate the multi-granularity label set Li by Eqs. (1)-(4);
• 3: end for
• 4: Construct the cluster-wise similarity graph using all label sets Li from all views by Eq. (5);
• 5: Construct the transition probability matrix P by Eq. (6);
• 6: Propagate cluster-wise similarities by Eq. (7);
• 7: Compute the cluster-wise similarity matrix Z by Eq. (8);
• 8: Construct the co-association matrix B by Eq. (9);
• 9: Apply CHC to B to obtain the final clustering label set L by Eqs. (2) and (3);

Output: The clustering result L.

to clusters from different views and granularities, and edges represent the similarity between these clusters based on the overlap of their constituent points.

Propagation of Cluster-wise Similarities

To capture both direct and indirect connections among clusters, a random walk propagation process is applied to the combined graph G. Let P denote the transition probability matrix for G, where each entry pp,q represents the probability of transitioning from cluster Cp to Cq in one step:

where ep,q is the Jaccard similarity between clusters Cp and Cq. The random walk propagation matrix P(t) after t steps captures indirect relationships:

The cluster-wise similarity matrix Z is computed by taking the cosine similarity of the random walk trajectories:

where P(1:t) (p, :) represents the trajectory of cluster Cp during the propagation process. The value of t is chosen to ensure clustering stability and efficiency, guided by practices commonly observed in related works.

Construction of the Co-association Matrix

To map the cluster-level similarities back to the object level, we construct the co-association matrix B as follows:

where v is the number of original views, and "+1" accounts for the fused viewXf , ki is the number of granularities in the i-th view. z(j) p,q represents the propagated similarity between clusters Cp and Cq at the j-th granularity level of the i-th view. x ∈ Cp and y ∈ Cq indicate that data points x and y belong to these clusters.

Figure 1 illustrates the proposed MGE framework. For a multi-view dataset, the CHC is applied to each view (including the fused view) to generate multi-view, multi-granularity labels. These labels are then integrated through an ensemble method that performs cross-view and cross-granularity fusion, resulting in the co-association similarity matrix B. As a result, the co-association matrix B averages the cluster-wise similarities from all views and all granularity levels, capturing direct and indirect object-level relationships across all views and granularities, and it serves as input for the secondary clustering to generate the final clustering results. Algorithm 1 presents the procedures of the MGE framework.

2.3 Computational Complexity Analysis

The computational complexity of MGE is primarily determined by three stages: multi-view multi-granularity label generation, cross-view and cross-granularity fusion, and secondary clustering. In the first stage, applying CHC to each view (including the fused view) incurs a complexity of O(n2 ), where v is the number of views and n is the number of data points. In the second stage, constructing the cluster-wise similarity graph requires computing pairwise Jaccard coefficients between clusters, resulting in a complexity of O(K2 total ), where Ktotal is the total number of clusters across all views and granularities. The subsequent random walk propagation step involves matrix multiplications with a complexity of O(tK2 total ), where t is the number of propagation steps. Finally, the secondary clustering step applies CHC to the co-association matrix, contributing an additional complexity of O(n2 ). Therefore, the computational complexity of MGE is O(n2 + tK2 total ). By utilizing approximate methods such as kd-tree to accelerate the 1-nearest neighbor search in CHC, the complexity can be further reduced to O(n log n + tK2 total ).

3 Experiments

In this section, we present the experimental studies of the proposed MGE on a synthetic data and six real-world datasets, in which three views of the synthetic data are shown in Figures 3 (a)-(c), and the detailed information of real-world multi-view datasets are reported in Table 1.

3.1 Experimental Settings

MGE is compared with the state-of-the-art competitors, including three ensemble-based methods: Fast Multi-View Ensemble Clustering (FMVEC) [Huang et al., 2023a], Matrix Multi-View Ensemble Clustering (MMEC) [Zhang et al.,

(h)

Figure 3: Results on the synthetic data. (a)-(c) depict the original data distributions with three views, (d)-(f) show the co-association matrices learned by MGE with different granularities (i.e. 3 granularities, 6 granularities and 9 granularities), (g)-(i) illustrate the results of FMVEC, MMEC, and GMC, respectively.

2023], and Coordinate Descent Ensemble Clustering (CDEC) [Li et al., 2024], as well as seven graph/representation fusionbased methods, such as Graph-based Multi-View Clustering (GMC) [Wang et al., 2019], Multi-view Subspace Clustering on Topological Manifold (TMMSC) [Huang et al., 2022], View Variation and View Heredity for Multi-View Clustering (V3H) [Fang et al., 2020], Large-Scale Multi-View Subspace Clustering (LMVSC) [Kang et al., 2020], Multi-View Clustering via Adaptively Weighted Procrustes (AWP) [Nie et al., 2018], Co-Regularized Multi-View Spectral Clustering (CoReg) [Kumar et al., 2011], and Weighted Multi-View Spectral Clustering (WMSC) [Zong et al., 2018]. We evaluate the performance using two widely used external validation metrics: Accuracy (ACC), and Normalized Mutual Information (NMI) [Strehl and Ghosh, 2002]. For all methods, parameters are tuned according to the default settings in their original publications to ensure optimal performance.

3.2 Experiments on Synthetic Dataset

To intuitively demonstrate the advantages of MGE, we generated a synthetic dataset with three heterogeneous views, as illustrated in Figures 3(a)-(c). Differing from other ensemblebased methods, MGE integrates multi-granularity labels, capturing both fine-grained and coarse-grained structures to encode local and global patterns within data. This manner facilitates the learning of more discriminative ensemble representations. As shown in Figures 3(d)-(f), increasing the number of granularities (and λ is set to 0.3, 0.6, and 0.9, progressively covering granularities from fine to coarse) from three to nine progressively clarifies the clustering structures in the co-association matrices, providing a stronger foundation for subsequent clustering. Figures 3(g)-(i) further demonstrate that FMVEC and MMEC produce weaker representations due to their reliance on single-granularity label integration, while GMC fails to fuse these heterogeneous views, manifesting the limitations of graph fusion-based methods in such scenarios.

It should be pointed out that MGE can effectively leverage its advantages to enhance the clusterability of the coassociation matrix even when a single view exhibits nearperfect clustering potential, as seen in View 3. By employing a constrained 1NN merging strategy, the multi-granularity labels generated by MGE accurately capture proximity relationships across both local and global scales. In contrast, the single-granularity methods (e.g., FMVEC and MMEC) and the fusion-based methods fail to utilize the multi-layered, near-ground-truth information provided by such views, resulting in co-association matrices with significantly diminished discrimination.

3.3 Experiments on Real-world Datasets

The results of the proposed MGE and other state-of-theart methods across six datasets are presented in Table 2, from which we can find that MGE achieves the best performance across all datasets on the ACC and NMI, Specifically, MGE achieves improvements in the average ACC of 17.62% and 10.07% over the best ensemble-based method (i.e., FMVEC) and the best graph fusion-based (i.e., CoReg), respectively. For the average NMI metric, MGE likewise achieves 7.38% and 5.70% improvements over the best ensemble-based method (i.e., FMVEC) and the best graph fusion-based (i.e., CoReg), respectively. The superior performance of MGE can be attributed to its innovative clustering mechanism that addresses key limitations of existing methods. Unlike ensemble-based approaches like FMVEC and MMEC, which rely on single-granularity label aggregation and are sensitive to noisy labels, MGE employs multigranularity clustering to preserve fine-grained and coarsegrained structures. Its secondary clustering stage further mitigates the impact of noisy labels, ensuring robust results. Compared to graph fusion-based methods like GMC and CoReg, which require explicit view alignment and extensive parameter tuning, MGE leverages multi-granularity information for consistent and scalable clustering across heterogeneous views. Additionally, MGE surpasses representation fusion-based methods like TMMSC and LMVSC by utilizing clustering labels and encoding both local and global patterns, delivering comprehensive and discriminative representations.

Figure 4 shows the runtime of MGE with several representative methods on the 100Leaves dataset, in which the number of samples varies from 200 to 1600. We note that MGE consistently exhibits the shortest runtime across different data sizes, attributed to its efficient two-stage process, i.e., generating multi-granularity clustering labels first and then clustering on the co-association matrix. This manner minimizes computational overhead by avoiding iterative optimization and graph alignment, which demonstrates the scalability of MGE and positions it as an effective solution for relatively large-scale clustering tasks.

3.4 Visualization

To demonstrate the advantages of the cross-view and crossgranularity fusion method in MGE, we compare its coassociation similarity matrix with those generated by two ensemble-based methods, MMEC and FMVEC, on the 100Leaves dataset. As shown in Figure 5, the first row presents the similarity matrices of the original three views, and the second row displays the co-association matrices learned by MGE, MMEC, and FMVEC. Specifically, the matrix of MGE exhibits a much clearer and more coherent clustering structure than those of MMEC and FMVEC. Moreover, the matrix of MMEC shows indistinct clustering patterns with significant noises, while the matrix of FMVEC, though clearer than MMEC, likewise suffers from low intracluster similarity (indicated by lighter diagonal regions), resulting in less compact clusters. This indicates that making full use of multi-granularity information facilitates capturing both fine-grained and coarse-grained structures as well as learning a more comprehensive representation for clustering.

3.5 Ablation Study

The MGE framework involves two clustering processes: the first generates multi-view, multi-granularity clustering labels, and the second performs secondary clustering on the coassociation matrix to produce the final result. In both stages, CHC is employed as the clustering method. To assess the impact of this choice, we replaced CHC with other hierarchical clustering algorithms, including single-linkage and averagelinkage, and named the corresponding variants MGE-1 and MGE-2, respectively. As shown in Figure 6, the ACC scores of MGE-1 and MGE-2 across three datasets were significantly lower than that of MGE, indicating the importance of using CHC in both stages of the proposed MGE framework.

3.6 Hyperparameter Sensitivity Analysis

The MGE framework contains only one hyperparameter λ, which controls the granularity of the clustering labels generated for all views, including the fused view. Figure 7 illustrates the ACC scores of MGE on three datasets when adjusting λ within the range of [0.1:0.1:0.9]. It can be observed that setting λ around 0.5 achieves the best average performance across the three datasets. A larger λ produces an excessive number of coarse-grained multi-view labels, reducing the discriminative ability between different classes in the ground truth. Conversely, a smaller λ generates insufficient types of multi-granularity labels, leading to significant information loss and diminishing the ability of the co-association matrix to distinguish ground-truth classes.

4 Conclusion

In this paper, we introduce Multi-view Clustering via Multigranularity Ensemble (MGE), a novel framework designed to

200

400

600

800

1000

1200

1400

1600

1600

address the challenges posed by view-specific inconsistencies and single-granularity limitations in ensemble-based multiview clustering methods. The proposed framework leverages the CHC to generate multi-granularity clustering labels for each view, including the fused view, capturing both finegrained and coarse-grained cluster structures. By performing cross-view and cross-granularity fusion, we constructed a co-association similarity matrix that integrates local and global clustering patterns across views, enabling robust secondary clustering. Extensive experiments conducted on a synthetic dataset with heterogeneous views and real-world multi-view datasets demonstrated the superior performance of MGE compared to state-of-the-art multi-view clustering algorithms across key evaluation metrics (ACC and NMI). Notably, MGE consistently achieved the highest clustering accuracy, highlighting its resilience to noisy or low-quality views and its ability to provide comprehensive representations of complex data structures. In future work, we aim to extend MGE to scenarios with incomplete or missing views and explore its scalability for large-scale, high-dimensional datasets. Additionally, we plan to investigate adaptive granularity control mechanisms to further enhance its generalization across various domains.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 62306171), the Science and Technology Major Project of Shanxi (No. 202201020101006).