One-Step Self-Aligned Anchor Learning for Multi-View Clustering

1 Introduction

Real-world data are often collected from multiple sources with different modalities, such as visual signals, textual descriptions, and audio tracks. To discover the underlying knowledge of such data, multi-view clustering (MVC) has emerged as a pivotal unsupervised learning paradigm [Wen et al., 2019]. Among various MVC approaches, multi-view graph clustering (MVGC) is particularly effective [Wen et al., 2022], as it constructs view-specific similarity graphs to capture pairwise relationships and then integrates them into a unified consensus graph.

While MVGC achieves promising performance, its computational complexity is typically O(n3 ) or O(n2 ) with respect to the number of samples, rendering it infeasible for largescale scenarios. Therefore, multi-view anchor graph clustering (MVAGC) has attracted increasing attention [Kang et al., 2020]. Instead of calculating affinities between all sample pairs, MVAGC selects a small set of representative points, i.e. anchors, to capture the data distribution [Deng et al., 2025]. By constructing an anchor graph between samples and anchors, the complexity is significantly reduced to a linear scale O(nm), making graph-based clustering feasible for large-scale datasets.

Despite the efficiency of MVAGC, the independent generation of anchors across different views leads to the critical Anchor-Unaligned Problem (AUP) [Wang et al., 2022], as shown in Figure 1 (a). The early studies avoid alignment issues by sampling raw instances with shared indices. For example, SFMC [Li et al., 2020] applies alternate sampling to identify anchors from aggregated similarity scores. However, these methods restrict anchor flexibility. Consequently, recent studies have turned to establishing correct correspondences for flexibly generated anchors across different views. For example, FMVACC [Wang et al., 2022] introduces a postalignment module that attempts to match anchors based on the similarities of their generated anchor graphs. Building on this, 3AMVC [Ma et al., 2024] further investigates the complex correspondence relationships among varying numbers of anchors across different views.

However, existing multi-view anchor matching approaches suffer from two critical limitations. First, the disjoint optimization of anchor alignment and anchor graph generation inevitably leads to suboptimal clustering performance. In such a disjoint paradigm, independently generated anchors cannot adaptively adjust to subsequent anchor alignment and anchor graph fusion, hindering high-quality consensus graph learning. Second, the indirect alignment strategy based on viewspecific anchor graphs is often unstable, as shown in Figure 1(b). This process relies heavily on the quality of the initial anchor graphs. More critically, selecting an optimal baseline view without prior knowledge is extremely challenging. Fixing a single view as the baseline essentially forces other views to conform to its specific distribution. This strategy ignores

(a) Anchor-Unaligned Problem

(b) Difference of Alignment Methods

Figure 1: Illustration of the AUP (Left) and comparison of alignment methods (Right). (a) The anchors generated by two different views may be semantically unaligned. (b) Comparison of alignment strategies: Traditional methods perform indirect alignment by utilizing the anchor graph of a single pivot view as the baseline, making the process highly sensitive to the quality of the baseline graph. In contrast, our OSAA-MVC aligns view-specific anchors with learned consensus anchors, effectively avoiding the dependency on any single view.

the complementarity among different views, which hinders the learning of the consensus anchor graph.

To overcome these limitations, we propose a novel unified framework, named One-Step Self-Aligned Anchor Learning for Multi-View Clustering (OSAA-MVC). The framework of our proposed OSAA-MVC is shown in Figure 2. Instead of existing multi-stage approaches, we integrate anchor generation, anchor alignment, and consensus anchor graph construction into a unified framework. Specifically, we learn view-specific anchors and global consensus anchors in a shared low-dimensional latent space, enabling direct selfalignment during the anchor learning process. This mechanism resolves the dependency on the quality of initial anchor graphs and effectively avoids the suboptimal clustering performance caused by post-alignment. Furthermore, to avoid the dilemma of selecting a robust baseline view, we design a mixed anchor strategy. By fusing the global consensus anchors with aligned view-specific anchors for data reconstruction, our method effectively avoids the bias introduced by a fixed baseline view while capturing both cross-view consistency and view-specific complementarity. Our primary contributions are outlined as follows:

• We propose a unified one-step framework for large-scale multi-view clustering to effectively tackle the AnchorUnaligned Problem, where anchor alignment and consensus anchor graph generation are seamlessly integrated without separation.
• We design a mixed anchor reconstruction mechanism that fuses the consensus anchors with aligned viewspecific anchors. It can effectively avoid the dilemma of baseline view selection.
• We employ a direct anchor self-alignment strategy, which effectively mitigates the dependency on the quality of initial anchor graphs.
• Extensive experiments demonstrate that our algorithm consistently outperforms state-of-the-art competitors.

2 RelatedWork

2.1 Multi-viewAnchor Graph Clustering

To address the scalability bottleneck of traditional graph clustering, multi-view anchor graph clustering (MVAGC) utilizes m representative anchors (m ≪ n) to construct anchor graphs, reducing the complexity to a linear scale O(nm) [Ji and Feng, 2023; Xu et al., 2025]. Therefore, the selection of representative anchors is critical.

Traditional methods typically select anchors using static strategies based on random sampling or heuristic algorithms and keep anchors fixed for subsequent graph construction [Shi et al., 2021; Yu et al., 2023]. For example, FMVPG [Shi et al., 2021] adopts direct alternating sampling to obtain anchors. FMCNOF [Yang et al., 2020] uses k-means to obtain cluster centers as anchors, and HBG-MVSC [Zhou et al., 2025] advances the selection process by applying bisecting k-means. The performance of these methods is highly dependent on the quality of the initial anchors.

To address the issue, recent works have adopted dynamic strategies that alternately optimize anchors and anchor graphs to obtain more representative anchors [Sun et al., 2024]. In practice, existing dynamic anchor optimization strategies can be categorized into two main types: consensus anchor learning [Ji and Feng, 2025] and view-specific anchor learning [Liu et al., 2024b; Wen et al., 2023]. The former methods learn a shared anchor set in a latent space and then map it to each view. For example, SMVSC [Sun et al., 2021] learns projected consensus anchors to generate a common anchor graph. MVSC-HFD [Ou et al., 2024] learns a shared anchor set and employs hierarchical dimensional projection matrices to reduce inter-view discrepancies. The consensus anchors may fail to fully capture view-specific information. In contrast, the latter methods learn anchors independently for each view. For example, CAMVC [Zhang et al., 2024] learns anchors separately for each view based on the anchor-cluster assumption. LASD [Zhang et al., 2025] learns anchors with distributions similar to the original data through a multi-view optimal transport framework. However, generating anchors from independent views leads to mismatched anchor sets, which complicates graph fusion.

2.2 Anchor-Unaligned Problem

Although anchor strategies are widely adopted to reduce computational complexity, the semantic correspondence of anchors across different views has long been overlooked. A potential solution to AUP is to select raw samples with identical indices across views as anchors [Li et al., 2020]. For example, TBGL [Xia et al., 2022] leverages a variance-based de-correlation anchor selection technique to choose anchors from concatenated view representations. However, this strategy inherently compromises the representational flexibility of the anchors. Recent advances focus on establishing accurate correspondences for flexibly generated anchors. For example, FMVACC [Wang et al., 2022] proposes an anchor alignment module that combines feature and structure information to align anchors across views. However, it simplistically adopts the first view as the alignment baseline, which poses the risk of being influenced by a noisy view. 3AMVC [Ma et al., 2024] further evaluates view quality based on clustering compactness to automatically identify an optimal baseline. This strategy still relies on a single view, neglecting the complementarity among different views. Consequently, AEVC [Liu et al., 2024a] generates a unified anchor graph from concatenated multi-view features to serve as common guidance for alignment, but unfortunately, such simple concatenation overlooks the inherent distribution discrepancies across views.

3 Method

3.1 Formulation

In multi-view learning, a fundamental assumption is that all views share a unified underlying data distribution despite representing different feature spaces [Ren et al., 2021;

Dong et al., 2023]. Consequently, it is reasonable to learn a set of representative anchors in a shared latent space. Let {Xi ∈ Rdi×n }vi=1 denote a multi-view dataset with n instances and v views, where di is the feature dimension of the i-th view. According to [Wang et al., 2021], the general framework for multi-view anchor graph clustering based on global consensus anchor learning can be formulated as

whereWi ∈ Rdi×k denotes the projection matrix for the i-th view, A ∈ Rk×m represents the consensus anchor matrix in the latent space, Z ∈ Rm×n is the consensus anchor graph, k and m denote the numbers of clusters and anchors, respectively. However, the consensus anchor matrix A fails to capture view-specific details. We thus introduce view-specific anchors Ai ∈ Rk×m and align them with A to integrate both common and specific information. Specifically, we obtain the alignment matrix Pi by minimizing the distance between Ai and A as follows

where ΩPi denotes the set of permutation matrices defined as

By integrating the mixed anchor representation into the reconstruction loss and imposing the alignment constraint, the final objective function is formulated as

where α is the balance factor for mixed anchors and λ is the regularization hyperparameter. The orthogonality constraints are enforced to enhance the diversity and discriminability of the learned anchors. For the final clustering partition, we directly apply k-means to the right singular vectors of the consensus anchor graph Z [Guan et al., 2025].

3.2 Optimization

Considering that the objective function in Eq. (4) is not jointly convex with respect to all optimization variables, we adopt an alternating iterative strategy.

Update Ai. Fixing other variables, the optimization problem for Ai is given by

The above problem is equivalent to the following trace maximization,

where Bi = αW⊤ i XiZ⊤P⊤i -α(1-α)AZZ⊤P⊤i +AP⊤i . According to [Hu et al., 2022], this problem admits a closedform solution, which can be derived using singular value decomposition (SVD).

Update A. Fixing other variables, the sub-problem for the consensus anchor matrix A can be equivalently transformed into a trace maximization problem as follows

where C = ∑vi=1 AiPi - α(1 - α) ∑vi=1 AiPiZZ⊤ + (1 - α) ∑vi=1 W⊤ i XiZ⊤. Analogous to the optimization of Ai, this problem can be solved by performing SVD on C.

UpdateWi. Similarly, the problem can be reformulated in trace form as follows,

where Di = XiZ⊤(αAiPi + (1 - α)A)⊤. This problem can be solved by performing SVD on Di.

Algorithm 1 OSAA-MVC algorithm

Input: Multi-view data matrices {Xi}vi=1 , cluster number k, anchor number m, parameters α and λ.

Output: Clustering result.

• 1: Initialize {Ai}vi=1 ,A, {Wi}vi=1 as zeromatrices, and set Z, {Pi}vi=1 as identity matrices.
• 2: while not converged do
• 3: Update {Ai}vi=1 by solving Eq. (6);
• 4: Update A by solving Eq. (7);
• 5: Update {Wi}vi=1 by solving Eq. (8);
• 6: Update Z by solving Eq. (10);
• 7: Update {Pi}vi=1 by solving Eq. (12);

• 8: end while

Update Z. Fixing other variables, the sub-problem for optimizing Z is formulated as

Since the columns of Z are independent, we can decompose this matrix optimization problem into n separate quadratic programming (QP) tasks. For the j-th column zj of Z, the problem is rewritten to

where G = 2(vα2 + v(1 - α)2 + λ)I + 2α(1 - α) (∑vi=1 P⊤i A⊤i A+A⊤ ∑vi=1 AiPi), f ⊤ = -2 ∑vi=1 X⊤i[:,j] Wi(αAiPi +(1-α)A), and Xi[:,j] represents the j-th column of Xi. The optimal solution is obtained by solving these sub-problems.

Update Pi. Fixing other variables, the optimization problem for updating variable Pi can be reduced to

where Amix = αAiPi + (1 - α)A. The problem (11) can be equivalently rewritten as

where Ei = αZX⊤i WiAi -α(1-α)ZZ⊤A⊤Ai +A⊤Ai. According to [Wang et al., 2025], it can be efficiently solved using standard linear programming toolboxes. The overall optimization procedure is summarized in Algorithm 1.

3.3 Discussion

Time Complexity. The optimization process involves updating five variables: {Ai}vi=1 , A, {Wi}vi=1 , Z, and

{Pi}vi=1 . Their corresponding computational complexities are O(nmd + vmk2 ), O(nmd + mk2 ), O(nmd + dk2 ), O(nm3 + nmd), and O(nmd + vm3 ), respectively, where d = ∑vi=1 di represents the total feature dimension. Consequently, the overall complexity is O(nmd + nm3 ). Given that m≪ n, the total time cost remains linear with respect to the number of samples n.

Space Complexity. The storage for multi-view data {Xi}vi=1 requires O(nd). The space complexities for the five optimization variables {Ai}vi=1 , A, {Wi}vi=1 , Z, and {Pi}vi=1 are O(vmk), O(mk), O(dk), O(nm), and O(vm2 ), respectively. Consequently, the overall space complexity is O(n(d + m) + dk + vm2 ). The memory requirement is linear with respect to the sample size n, ensuring the proposed OSAA-MVC method is memory-efficient for largescale applications.

Convergence Analysis. As detailed in the optimization section, we solve each sub-problem optimally by fixing the remaining variables. This iterative updating scheme ensures a monotonic decrease in the objective function value defined in Eq. (4). Therefore, consistent with the findings in [Bezdek and Hathaway, 2003], our proposed algorithm is theoretically guaranteed to converge to a local optimum.

4 Experiments

4.1 Experimental Setting

Datasets. To validate the proposed framework, we conduct extensive experiments on ten widely used multiview datasets, including Yale, Dermatology, Caltech101-20, Scene15, CCV, COIL100, Fashion, Hdigit, VGGFace2, and MNIST. The detailed descriptions of these datasets are listed in Table 1.

Baseline Algorithms. We benchmark the proposedOSAAMVC against eleven representative large-scale MVC algorithms, including FMVACC [Wang et al., 2022], 3AMVC [Ma et al., 2024], AEVC [Liu et al., 2024a], MVCAGAF [Wang et al., 2025], Orth-NTF [Li et al., 2023], TC-MVSC [Chang et al., 2024], SMVAGC [Wang et al., 2024], MVSC-HFD [Ou et al., 2024], NpGC [Yu et al., 2024], MCHBG [Zhao et al., 2025], LASD [Zhang et al., 2025]. To ensure an equitable comparison, all selected competitors feature linear computational complexity with respect to the number of samples. It is worth mentioning that the first four methods address the Anchor-Unaligned Problem (AUP) by incorporating anchor alignment modules. The experimental environment consists of a 1.40 GHz Intel Core Ultra 7 CPU and 32GB RAM.

Experimental Details. To ensure a fair comparison with baselines, we optimize their hyperparameters by traversing the search spaces recommended in their papers and report the best results. Our method involves three key parameters: anchor number m, balance factor α, and regularization parameter λ. We adopt a grid search strategy to determine the optimal values. Specifically, m is selected from {k, 2k, 3k}, α is tuned within the range of [0.1, 0.5] with a step size of 0.2, and λ is searched from {10, 100, 1000}. Four clustering evaluation metrics are employed in our paper, including accuracy (ACC), normalized mutual information (NMI), Purity, and Fscore. To mitigate the impact of random initialization, we repeat the clustering process 50 times and report the average results.

4.2 Clustering Performance

To evaluate the effectiveness of the proposed method, we compare OSAA-MVC with eleven state-of-the-art multi-view clustering algorithms on ten benchmark datasets, as shown in Table 2. Based on the experimental results, we have the following observations:

• OSAA-MVC achieves the best performance on all ten datasets across four evaluation metrics. Focusing on ACC, our method consistently surpasses the second-best method, with relative improvements of 6.05%, 4.06%, 23.29%, 10.43%, 0.27%, 2.42%, 3.87%, 0.74%, 6.60%, and 0.20%.
• Compared with methods that address the AUP, such as FMVACC, AEVC, and 3AMVC, our method achieves better clustering performance across datasets. The performance improvement validates the effectiveness of our one-step self-alignment strategy. Furthermore, the superior results confirm that our direct alignment paradigm outperforms indirect alignment by anchor graphs, avoiding the dependency and instability associated with baseline view selection.
• OSAA-MVC exhibits remarkable scalability and memory efficiency on large-scale datasets like Hdigit and MNIST. It delivers the best performance on large-scale benchmarks while several baselines suffer from out-ofmemory errors, highlighting the suitability for largescale applications.

4.3 Running Time

To evaluate computational efficiency, we report the average execution times of all compared methods on ten benchmark datasets, as summarized in Table 3. Specifically, on smallscale datasets, OSAA-MVC maintains competitive efficiency comparable to other efficient baselines. However, its advantages become pronounced on large-scale datasets. The proposed method achieves the fastest runtime on Scene15, CCV, COIL100, Fashion, VGGFace2, and MNIST. Furthermore, while competitors like Orth-NTF, MCHBG, and TCMVSC suffer from out-of-memory errors on large datasets, our method handles them efficiently, demonstrating favorable scalability in both time and memory.

4.4 Ablation Study

To comprehensively evaluate the contribution of each component within our proposed framework, particularly the mixed anchor strategy and the self-alignment mechanism, we conduct ablation studies by comparing OSAA-MVC with three variants: (1) Ai: using only view-specific anchors; (2) A: using only the shared consensus anchors; and (3) Amix: utilizing the mixed anchor strategy but removing the alignment constraint. The results reported in Table 4 show the performance differences among different variants. First, both Ai and A underperform compared to the mixed variants, indicating that relying solely on specific or consensus information is insufficient for complex multi-view data. Second, while Amix improves upon the view-specific and consensus anchors, it is consistently outperformed by the proposed method. For instance, on the Dermatology dataset, the proposed method increases the clustering performance from 93.85% to 97.21% compared to Amix. This performance improvement validates the necessity of the self-alignment mechanism, demonstrating that simply mixing anchors without alignment introduces semantic mismatch, whereas our onestep alignment strategy effectively resolves the AUP.

4.5 Parameter Sensitivity

Sensitivity of anchor number m. We investigate the impact of the anchor number m on clustering performance by varying it from k to 5k, as shown in Figure 3. From the results, the performance improves as m increases from k to 2k and reaches the best performance at m = 2k. However, further increasing m beyond 3k leads to a noticeable decline in clustering performance, indicating that too few anchors cannot sufficiently cover the data distribution, while too many anchors may introduce redundancy and degrade the consensus graph. Therefore, we set the search space of m to {k, 2k, 3k} in the main experiments.

Sensitivity of parameters α and λ. To evaluate the stability of OSAA-MVC, we analyze the sensitivity of the balance factor α and the regularization parameter λ. As shown in Figure 5, we report the clustering performance in terms of ACC on the Yale and Dermatology datasets. The proposed OSAAMVC is relatively insensitive to α but more sensitive to λ. Based on these empirical observations, we suggest selecting α from a coarse candidate set {0.1, 0.3, 0.5} and tuning λ within the range of {10, 100, 1000} for practical applications.

4.6 Convergence

To empirically investigate the convergence properties of the proposed optimization algorithm, we record the objective function values at each iteration on the Yale and Dermatology datasets. As illustrated in Figure 4, the objective value monotonically decreases as the number of iterations increases. Notably, the curve drops sharply within the first few iterations and stabilizes rapidly, typically reaching convergence in less than ten iterations. This observation verifies that our iterative optimization scheme is computationally efficient and capable of finding a local optimum quickly.

5 Conclusion

In this paper, we propose a novel unified one-step framework, named OSAA-MVC, to tackle the challenging AnchorUnaligned Problem in large-scale multi-view clustering. Instead of traditional multi-stage approaches, OSAA-MVC integrates anchor generation, anchor alignment, and anchor graph construction into a coherent joint optimization process. By aligning the learned view-specific anchors with the learned global consensus anchors in the latent space, our method achieves direct self-alignment, effectively eliminating the dependency on the quality of initial anchor graphs. Furthermore, the proposed mixed anchor reconstruction strategy fuses common and view-specific information, successfully circumventing the dilemma of baseline view selection and the associated bias. Extensive experiments on various benchmark datasets validate that OSAA-MVC consistently outperforms state-of-the-art methods while maintaining high computational efficiency.

Acknowledgements

This work was supported in part by the Natural Science Foundation of Hunan Province, China, under Grant 2026JJ20079, and the National Natural Science Foundation of China (NSFC) under Grants 62376279 and 62306324.

Contribution Statement

• ∗ These authors contributed equally. † Corresponding author.