Progressive Adversarial Multi-View Alignment for Unsupervised Embedded Feature Selection with Linear Complexity

1 Introduction

As object collections continue to expand, modern datasets increasingly involve objects being assigned multiple distinct feature sets, forming multi-view representations [Xu et al., 2024; Khalafaoui et al., 2025]. Such views typically appear as images paired with text descriptions [Feng et al., 2024] or as sensor data drawn from different modalities [Fadadu et al., 2022]. By providing complementary information, they enable a more comprehensive and profound understanding of underlying data structures, making multi-view representation learning a key research focus in computer vision, bioinformatics, and natural language processing [Li et al., 2018; Fang et al., 2023b]. However, the high dimensionality and inherent redundancy nature of raw multi-view data lead to high costs and challenges in fitting. Unsupervised multi-view feature selection (UMFS) addresses these challenges by selecting a compact and informative feature subset from all available views to enhance the performance and efficiency of downstream tasks [Dong et al., 2025].

∗Corresponding author.

The fundamental challenges in UMFS involve achieving cross-view consistency while preserving intra-view diversity [Zhang et al., 2024a; Liu et al., 2025]. While maintaining consistency facilitates learning unified representations from cross-view knowledge, this process should not come at the expense of diversity, which constitutes the core discriminative value of individual viewpoints. Current approaches to cross-view issues predominantly rely on a single static objective to resolve this conflict, yielding unsatisfactory outcomes. For instance, canonical correlation analysis maximizes statistical correlations between views while compromising discriminative information [Xu et al., 2019b; Xu et al., 2019a], whereas joint matrix factorization reconstructs single-view spaces without considering inter-view spatial coherence, thereby undermining consistency [Hsieh et al., 2019; Cao and Xie, 2024]

objectives prevents models from effectively addressing both consistency and diversity challenges, thereby limiting their practical applications.

Another challenge stems from the massive scale of modern datasets, which creates a disconnect between theoretical algorithm design and practical implementation. Standard methods inherently involve super-linear complexity computations, facing scalability issues in both memory allocation and runtime. These limitations are particularly evident in fundamental algorithms like constructing pairwise similarity graphs, where quadratic expansion in computational complexity poses significant challenges for handling large datasets [Wang et al., 2021]. Furthermore, it is also manifested in algorithms requiring iterative updates of complete data matrices, which demand computationally intensive matrix factorization [Zhou et al., 2023a; Wu et al., 2024]. Consequently, reliance on complex computations leads to nonlinear growth, creating a mismatch between theoretical algorithmic design and scalable practice.

This study proposes a Progressive Adversarial Feature Selection (ProAd-FS) framework to address computational and structural challenges. To be specific, ProAd-FS utilizes an adversarial framework to assign competing objectives into a new decoupled architecture, which consists of an interview game to learn view-invariant representations and an intra-view game to retain feature diversity. We further develop a Robust Gated Curriculum (RGC) to impose an easyto-hard learning trajectory throughout the training process. This enables ProAd-FS to construct a robust shared baseline from basic data, progressively expanding through complex sample details, yielding stronger comprehensive discrimination. The entire framework is optimized via an endto-end objective function embedding structured sparsity regularization through ℓ2,1-norm, demonstrating linear complexity for scalability across large datasets. Experimental results show ProAd-FS outperforms baseline methods on large-scale benchmark datasets of up to 70,000 samples, while running over more than an order of magnitude faster than baselines. In summary, our contributions are as follows.

1. We present a novel UMFS framework that leverages the adversarial game to formulate the static trade-off between cross-view consistency and intra-view diversity as a dynamic competitive process.
2. We design a gated curriculum, which imposes a hierarchical feature space discovery process to progressively guide the model towards a robust subset of features.
3. ProAd-FS demonstrates linear computational complexity in both sample size and feature dimensionality, with high scalability.

2 RelatedWorks

Unsupervised Multi-view Feature Selection. Numerous UMFS methods enforce cross-view consistency by learning shared latent representations or pseudo-label spaces. Early studies used pseudo-class labels as a bridge between views [Tang et al., 2013] or decomposed the label space into consensus and view-specific components [Tang et al., 2021]. Other prominent studies employ adaptive learning to derive a unified similarity matrix that reflects cross-view commonalities. Building upon this paradigm, numerous methods focus on learning a common similarity matrix with view-specific weights [Hou et al., 2017; Wan et al., 2020], integrating latent adjacency representations [Zhou et al., 2023b], or employing non-linear kernels and hashing to generate weak supervision [Hu et al., 2025]. In terms of scalability, [Zhang et al., 2024b] constructed bipartite graphs to reduce complexity to near-linear. Our work also achieves linear complexity, but unlike previous studies, ProAd-FS innovatively modifies the objective function by introducing a dynamic adversarial game mechanism, enabling a more scalable and robust feature selection process, especially in complex data environments.

Adversarial Representation Learning. Generative Adversarial Networks (GANs), proposed by [Goodfellow et al., 2014], represent a powerful framework for representation learning. Existing applications are diverse, including learning rich visual representations with bidirectional GANs [Donahue and Simonyan, 2019], decoupling through mutual information maximization [Chen et al., 2016], generating challenging training samples to learn robust encoders [Pan et al.,

2021], and ensuring discriminative representation in hybrid architectures [Zuo et al., 2023]. While these methods typically rely on a single game to achieve their goals, ProAd-FS introduces an adversarial decoupled framework. By assigning consistency and diversity as distinct games to two independent processes, it transforms the static trade-off in UMFS into dynamic competition.

Curriculum and Self-Paced Learning. The Curriculum Learning (CL) strategy proposed by [Bengio et al., 2009] for progressively presenting training samples from simple to complex has been proven effective in improving model robustness. Self-Paced Learning (SPL) automates this training by dynamically selecting samples based on current loss values [Kumar et al., 2010]. Other improvements to SPL have focused on encouraging sample diversity [Jiang et al., 2014], establishing self-adaptive difficulty ratings [Kong et al., 2021], mitigating outliers in feature selection [Li et al., 2022], and ensuring high-difficulty training is learned during the learning process [Jin et al., 2025]. These methods require an absolute loss-value threshold. Otherwise, they fail to function effectively during training. Inspired by the principle of robust learning, ProAd-FS employs a ranking method to adjust loss values while maintaining their fundamental stability, providing a robust gate controller for feature discovery. These methods require a predefined absolute loss threshold, which is fragile in the volatile training dynamics of adversarial settings. Inspired by the principle of robust learning, ProAd-FS employs a ranking method to adjust loss values. This design is inherently invariant to monotonic shifts in the loss distribution, providing a stable gating mechanism for feature discovery.

3 ProposedMethodology

3.1 Overall Objective Function

Let a multi-view dataset be denoted by X = {X(v) ∈ Rn×dv }Vv=1 , where n is the number of samples and dv is the feature dimensionality of the v-th view. Our goal is to learn a set of linear projections W = {W(v) ∈ Rdv×k }Vv=1 such that the resulting k-dimensional latent representations H(v) = X(v)W(v) are learned by a set of competing objectives: they should be sufficiently cross-view consistent to confuse a discriminator trained to detect their origin, while also being sufficiently intra-view diverse to appear indistinguishable from a prior distribution to another set of discriminators.

The ProAd-FS framework is formulated as a min-max optimization problem. The minimizer controls the projection matrices W and a sample-weighting vector s, while the maximizer consists of two distinct discriminators: a consistency discriminator Dc tasked with identifying the origin view of latent samples, and a diversity discriminator Dd tasked with distinguishing these representations from synthetic prior distributions. To ensure the correctness of the learning trajectory, we introduce Robust Gated Curriculum (Section 3.3), which progressively selects easier-to-train samples. This process is implemented by the indicator function ICt (s), with hard constraints imposed on the vector s. The constraint set Ct = {s ∈ {0, 1}n | ∥s∥1 = Nt} specifies that exactly nt samples must be selected during the t-th iteration. The sample budget Nt is not fixed but varies over time, enabling the model to learn from more complex samples. The overall objective function is defined as follows:

where Li is the sample-level adversarial loss, while β is the hyperparameter that balances the loss with ℓ2,1-norm sparse regularization. We now define each component in detail.

3.2 Competing Adversarial Objectives

For each sample loss Li that comprises two adversarial games, we didn't use a single objective function to balance their consistency and diversity. Instead, we assign these objectives to specific competition mechanisms to solve the problem. The relationship between the generator and discriminator is shown in Figure 1.

Inter-View Consistency Game. The task of the consistency game is to enforce distributional alignment between latent representations, leading to a view-invariant learned latent space. To achieve this, we design a multi-class consistency discriminator Dc, parameterized by θc. This attempts to predict the view of origin for any given latent sample h. The projection matrices W act as a single multi-part generator, trained to produce latent representations that are so similar across views that Dc cannot distinguish them. This is formulated as a Min-Max game over the negative log-likelihood of the correct view prediction:

where Dc(y = v|h) denotes the discriminator's probability pc that latent sample h belongs to view v. This is computed by the Softmax function from the logit score vector ac = hθc. The discriminator Dc aims to maximize this objective (correct classification), whereas the generator W aims to minimize it (induce misclassification).

Intra-View Diversity Game. In the diversity game within the view, to prevent all samples from being mapped to a trivial solution, the model ensures the latent representation to preserve the rich structure of the original data. For each view v, a binary diversity discriminator Dd(v) is defined by a parameter θ(v) d , which discriminates between "real" samples drawn from the latent distribution h(v) ∼ PH(v) and "fake" samples drawn from the standard Gaussian prior distribution p(z) = N(0, I). The Min-Max game is governed by the cross-entropy loss function.

The discriminator Dd is designed to estimate the probability scalar pd that real samples are labeled as 1 while fake samples are labeled as 0. This scalar is generated by the Sigmoid function for the Logit score ad = hθ(v) d . The generatorW(v) minimizes this expression by transforming X(v) to H(v) , with its distribution indistinguishable from p(z).

Total Adversarial Loss. The total adversarial loss is computed with a single loss value Li for each sample during the training process. From the generator's perspective, the algorithm achieves loss minimization through a weighted sum of consistency loss and diversity loss, as detailed in Algorithm 1. Instead of balancing fixed objectives, γ acts as a scaling factor for the gradient signals that arise from two adversarial games. This trade-off is dynamically adjusted by the relative strength of Dc and Dd throughout the training process.

Algorithm 1 Per-Sample Adversarial Loss Computation

Input: Sample X(v) i ; Projection matrix W(v) ; Discriminators Dc,Dd; Scaling factor γ, Small epsilon ϵ. Output: Per-sample loss Li.

3.3 Robust Gated Curriculum

The RGC is designed to impose a hierarchical learning path to resist the volatile optimization dynamics in adversarial training. It consists of two interconnected components: gating to regulate training data volume and a robust selection criterion. In adversarial scenarios, features form a non-stationary loss function landscape where absolute loss values fluctuate during iterations. Therefore, content selection should prioritize samples with relative lower complexity over those exhibiting extreme volatility.

This strategy exhibits inherent robustness, as the monotonicity of the loss distribution remains unchanged. The gating mechanism consistently selects the simplest samples at each stage, while progressively increasing the active sample size Nt with each iteration t, thereby achieving a deterministic annealing process. The transition from simple to complex samples is smooth, with the sample selection ratio δt determined by the following sigmoidal annealing function:

where t denotes the current iteration count, δstart represents the initial sample proportion, T indicates the total iteration count, σ is the slope of the sigmoidal curve, and Nt = ⌊δt ·n⌋ is the sample size. For the step of calculating and selecting the easiest samples, see Algorithm 2.

Algorithm 2 Robust Gated Curriculum Update

Input: Samples {xi}ni=1 ; Current iteration t; Total iterations T ; Empty list L.

Output: Curriculum weights s.

δt ←Compute annealing percentage by Eq. (4);

for i = 1 to n do

Li ← Compute per-sample loss using Algorithm 1; Append (Li, i) to L;

end for

Sort L based on loss values in ascending order; Initialize sample weight vector s ← 0 ∈ Rn ;

end for

return s

3.4 ℓ2,1-NormRegularization

To achieve feature selection, we impose a group sparsity constraint on the rows of the projection matrices W(v) . The ℓ2,1-norm of a matrix W(v) is defined as ∥W(v) ∥2,1 = ∑dv j=1 ∥w(v) j ∥2, where w(v) j is the j-th row vector [Nie et al., 2010]. Minimizing this norm encourages entire rows of W(v) to become zero, effectively reducing redundant features.

4 Optimization

The Min-Max objective function of ProAd-FS (Eq. 1) involves multiple interdependent variables and does not possess a closed-form solution. Therefore, we devise an iterative alternating optimization strategy to find a stable equilibrium.

4.1 Minimizer Optimization

Updating SampleWeights s. Given the fixed generatorW and discriminator D, the constraint on the sample weights s from our overall objective function J (Eq. (1)) is a constrained optimization problem:

where the sample loss function Li measures sample complexity, and the optimal solution is provided by the following theorem.

Theorem 4.1. Let {Lj, idxj}nj=1 be the sequence of persample losses and their original indices, sorted such that L1 ≤ L2 ≤ · · · ≤ Ln. The optimal solution s∗ to the problem in Eq. (5) is given by:

Proof. The proof is by a standard exchange argument. Assume there exists an optimal solution s′ that is different from s∗ . This implies there must be at least one index k ∈ {idx1, . . . , idxNt } for which s′k = 0, and at least one index m ∈ {idxNt+1, . . . , idxn} for which s′m = 1. By the definition of our sorted losses, we have Lk ≤ Lm.

Let us construct a new solution s′′ by swapping these two assignments: let s′′ k = 1, s′′m = 0, and s′′ i = s′i for all other i. The new solution s′′ remains feasible, as ∥s′′ ∥1 = ∥s′ ∥1 = Nt. The objective value for s′′ is ∑ni=1 s′′ i Li = (∑ni=1 s′iLi) - Lm + Lk. Since Lk ≤ Lm, we have ∑ni=1 s′′ i Li ≤ ∑ni=1 s′iLi. Therefore, the objective value of s′′ is no worse than s′ . By repeating this exchange for all such differing pairs, we can transform s′ into s∗ without increasing the objective value.

Updating Projection Matrices W. With the curriculum s∗ and the discriminators D fixed, we update the projection matrices W by performing a single step of gradient descent. The optimization of each projection matrixW(v) is achieved by descending along the gradient of J :

where each component of the gradient is a weighted sum over the samples, detailed as follows:

where ⊙ is the Hadamard product. The diagonal matrix Λ(v) contains the inverse ℓ2-norms of the rows ofW(v) .

Aggregating these components, the update rule for the projection matrices is executed with a learning rate ηw:

4.2 Maximizer Optimization

Updating Consistency Discriminator Dc. The parameters θc are updated using the gradient of the weighted crossentropy loss. The gradient is derived from the prediction error (Yc -Pc), where Yc represents the one-hot view indices and Pc the predicted probabilities. This error is scaled by the curriculum weights s∗ , and the resulting weighted error is backpropagated. The update rule is:

where ∇θc Jcons is proportional to (Hc)T (s∗ ⊙ (Yc -Pc)).

UpdatingDiversityDiscriminatorsDd(v) . The update for the view-specific parameters θ(v) d relies on a composite batch Hd and an asymmetric weighting strategy. The batch Hd comprises both real samples in the form of latent representations H(v) and fake samples consisting of synthetic vectors z from a Gaussian prior. The corresponding authenticity labels Yd are generated on-the-fly (1 for real, 0 for fake). Crucially, the curriculum is applied asymmetrically, where the prediction error for real samples is weighted by s∗ in contrast to the error for fake samples, which is uniformly weighted by one. This is mathematically realized by constructing a composite weight vector s∗d = [s∗ ; 1]. This design fosters a more robust adversary by compelling it to learn the entire prior distribution. The update rule is:

where ∇θ(v) d Jdiv is proportional to (Hd)T (s∗d ⊙ (Yd -Pd)), reflecting the backpropagation of weighted error.

5 Computational Complexity

We analyze the computational complexity of the ProAdFS algorithm per iteration. The forward pass commences with the projection of input data into the latent space. For each of the V views, this involves a matrix multiplication over all views between X(v) and W(v) , incurring a cost of O(∑Vv=1 ndvk). Subsequently, computing per-sample losses is dominated by the consistency discriminator, which has a complexity of O(V 2nk). The curriculum update then requires a sorting operation on the n loss values, which costs O(n logn). During the backward pass, the gradients for the projection matrices are computed. For each view v, it requires a multiplication of the form (X(v) )⊤(·), where a matrix of size dv × n is multiplied by a matrix of size n × k. The cost for all V views is therefore O(∑Vv=1 dvnk).

Aggregating these costs, the total complexity per iteration is O (∑Vv=1 dvnk + V 2nk + n log n). In typical largescale scenarios where multi-view data is used, the feature dimensions dv are significantly larger than the number of views V and logn. The term ∑Vv=1 dvnk dominates the others. As a result, the per-sample total is approximately O (nk ∑Vv=1 dv ). Therefore, the complexity of ProAd-FS is linear with respect to both the number of samples and the total feature dimensionality.

6 Experiments

6.1 Experimental Setup

Datasets. We selected six multi-view datasets that span diverse types and scales to ensure a comprehensive evaluation, including MSRC-v1 1 , ORL 2 , COIL20 3 , Animal 4 , ALOI 5 ,

1 http://www.cad.zju.edu.cn/home/dengcai/Data/data.html

2 https://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html

3 https://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php

4 https://opendatalab.org.cn/OpenDataLab/ANIMAL

5 https://aloi.science.uva.nl/

and FashionMNIST 6 . Detailed statistics for each dataset are summarized in Table 1.

Comparison Methods. We compared ProAd-FS against a variety of baselines covering a wide spectrum of UMFS methods. Our comparisons included advanced multi-view methods such as JMVFG [Fang et al., 2023a], CAHR [Song et al., 2024], CE-UMFS [Zhou and Song, 2024], and UKMFS [Hu et al., 2025]. We also included a leading single-view method adapted for this task (FSDK [Nie et al., 2023]) and a fullfeature baseline (AllFea) to establish an initial performance floor. The theoretical complexity of the methods is presented in Table 2.

Implementation Details. Following standard practice, we applied each method to select a feature subset. We then evaluated the quality of the selected features by performing Kmeans clustering and reported three standard metrics: Accuracy (ACC), Normalized Mutual Information (NMI), and Adjusted Rand Index (ARI) [Xu and Wunsch, 2005]. To ensure statistical significance, we averaged all results over 20 independent runs, and conducted statistical comparisons in accordance with the recommendations of [Zhou et al., 2025]. Furthermore, we tuned γ and β via a grid search over {0.01, 0.1, 1, 10, 100}, and set the learning rates to ηw = 10-4 and ηD = 10-5 . Finally, we scheduled the curriculum to complete at 40% of the total iterations.

6.2 Clustering Results

To validate the effectiveness of ProAd-FS, we conducted a clustering performance evaluation. Figure 2 demonstrates that our ProAd-FS framework consistently outperforms all baseline methods across four representative datasets, even surpassing the full dataset at low selection rates (5-10%). This highlights the effectiveness of our adversarial game in identifying discriminative feature subsets.

6 https://www.worldlink.com.cn/en/osdir/fashion-mnist.html

Table 3 details the performance at a more reasonable 15% feature selection rate. Our method still demonstrates significant advantages in the majority of cases. The statistical significance test results (marked with †) indicate a clear advantage of our approach over suboptimal methods (p < 0.05). Specifically, on the COIL20 dataset, our method achieves a distinct advantage, showing the efficacy of the decoupled adversarial architecture. On the ALOI dataset, while our method does not achieve the highest accuracy, it excels in both NMI and ARI metrics. This is because NMI and ARI are key indicators for evaluating clustering structure fidelity.

Our framework preserves the inherent data structure, which proves more valuable than simple label matching accuracy in certain unsupervised scenarios. The scalability advantage of ProAd-FS was validated on the large-scale FashionMNIST dataset. Compared to the multi-view baseline methods rendered "N/A" (Not Available) by their super-linear complexity, ProAd-FS maintains computational feasibility while achieving the highest scores across all metrics. These empirical results validate both the effectiveness and the scalable superiority of ProAd-FS.

6.3 Running Time

To validate the efficiency of the ProAd-FS framework, we measured the runtime of various baselines across four representative datasets. Table 4 demonstrates its practical benefits in linear complexity design, where ProAd-FS exhibits superior performance, especially on challenging datasets. On the ORL dataset, ProAd-FS is significantly faster than methods like CAHR, which have high complexity for feature dimensions. On the large-scale FashionMNIST dataset, while other competing methods fail due to Out-of-Memory (OM) errors, ProAd-FS operates with significantly higher efficiency. The experimental results aligned closely with our theoretical complexity analysis outlined in Table 2. The linear-complexity architecture of ProAd-FS is not just a theoretical benefit, but translates directly to practical implementations, confirming its efficiency and scalability for high-dimensional and largescale applications.

6.4 Ablation Study

To validate the necessity of our comprehensive design, we employed three variants of ProAd-FS: 1) without Consistency (w/o Cons.), which removes the inter-view consistency game; 2) without Diversity (w/o Div.), which removes the intra-view diversity game; and 3) without Robust Gated Curriculum (w/o RGC), which removes the Robust Gated Curriculum. As shown in Table 5, removing both consistency and diversity components resulted in performance degradation across all datasets. This occurs because the model cannot learn core shared information across views when alignment targets are absent, which is fundamental to multi-view learning. The significant performance drop caused by removing the RGC also demonstrates its critical role in the optimization process. This is because the RGC establishes the learning process in a progressive complexity order, which is crucial for achieving a high-quality solution. These three components collectively validate the effectiveness of our complete design.

6.5 Hyperparameter Robustness

We investigated the sensitivity of our ProAd-FS framework to its two main hyperparameters: the scaling factor γ and the sparsity regularization weight β. Figure 3 shows the ACC for varying pairs of these two parameters for the four benchmark datasets. As can be seen, the results reveal a large contiguous plateau of high performance across all datasets, rather than an isolated peak. This indicates that ProAd-FS is not sensitive to the exact choice of hyperparameters and does not require detrimental exhaustive fine-tuning. In fact, significant degradation in performance only occurs at the extremes of the grid investigated for the parameter values where the objectives are highly imbalanced. For our experiments, optimal parameters were easily found via a standard grid search over the set {0.01, 0.1, 1, 10, 100}.

7 Conclusions

This work introduced a novel ProAd-FS framework to overcome the limitations in modeling the competition between cross-view alignment and intra-view diversity. We formulated this static trade-off as a dynamic competitive process in an adversarial decoupled architecture, where these competing objectives are assigned to distinct games. To ensure a principled learning trajectory within this adversarial setting, we introduced the RGC strategy that guides the model from underlying shared structures to complex view-specific information. An important aspect of ProAd-FS is that we achieve linear computational complexity in both sample size and feature dimension. This makes ProAd-FS suitable for large-scale applications, which would presumably be a limiting factor in existing UMFS methods. In summary, ProAd-FS unites a game-theoretic formulation with a highly efficient implementation, establishing an effective new solution to modern large-scale UMFS.

Acknowledgments

This work was supported in part by Guangdong Basic and Applied Basic Research Foundation (NO. 2024A1515030022), National Natural Science Foundation of China (No. 62276103, 61906069), Innovation Team Project of General Colleges and Universities in Guangdong Province (No. 2023KCXTD002), the Fundamental Research Funds for the Central Universities, JLU (No. 93K172024K03, 93K172024K24, 2024ZYGXZR097), the Natural Science Research Project of Education Department of Guizhou Province (No. QJJ2023061), Analysis of Urban Events Based on Low-Altitude Drones (No. 2024BQ010011), the Research and Development Project on Key Technologies for Intelligent Sensing, Guangdong Provincial Philosophy and Social Sciences Planning 2023 Key Project on Special Research in Auditing Theory (No. GD23SJZ09).