Multiscale-adaptive and Size-adaptive PSO-based Feature Selection for Gene Expression Analysis

1 Introduction

Gene expression analysis and cancer diagnosis are representative and widely studied problems in the bioinformatics and medical research [Ravindran and Gunavathi, 2023]. These tasks typically involve numerous features, and the high cost acquisition severely limits the sample sizes, resulting in high-dimension low-sample size (HDLSS) datasets[Liu et al., 2017] and the curse of dimensionality [Chattopadhyay and Lu, 2019]. Despite the success of deep learning (DL) in vision and language domains, significant challenges remain in this fields [Noor and Ige, 2025]. The characteristics of HDLSS datasets make it difficult for most DL approaches based on manifold assumptions to learn informative and interpretable representations [Loaiza-Ganem et al., 2024]. Moreover, the class imbalance in biological data further increases the risk of overfitting [Min et al., 2017].

*Corresponding author: Fei Han; Code and datasets are available at https://anonymous.4open.science/r/MASA-PSO-15FE/.

In this context, feature selection (FS), particularly evolutionary algorithm (EA) methods, receives considerable attention due to its ability to significantly reduce dimensionality, enhance interpretability, and avoid manifold assumptions [Xue et al., 2015]. Among EAs, particle swarm optimization (PSO) has been widely applied to optimization tasks as its fast convergence and robustness in high-dimensional spaces [Bonyadi and Michalewicz, 2017]. However, EAs often suffer from high computational and memory costs in HDLSS biological datasets. To address this, Tran et al. [Tran et al., 2018] proposed VLPSO, which dynamically adjusts the search space via length-adaptive and mutual learning mechanisms, enabling more efficient discovery of compact feature subsets with better classification performance. It has inspired subsequent variable-length mechanisms EAs (VLEAs) in bioinformatics and medical applications. For instance, Qu et al. [Qu et al., 2023] designed a PSO with the explicit feature representation (ESAPSO) to further reduces computational cost and memory usage. Nonetheless, existing VLEAs [Tran et al., 2018; Qu et al., 2023; Zhou et al., 2022] typically construct different search space sizes through feature grouping, while overlooking the mismatch between population size and space size. A subgroup with the excessively small search space may impair collaborative search and miss the global optimum, while a subgroup with too large search space may lead to redundant exploration of similar regions, resulting in wasted search resources and increased computational cost [Antonio and Coello, 2013].

In addition, due to the curse of dimensionality, most existing EAs including VLEAs commonly adopt Manhattan distance to measure sample distances to capture differences in distributions among different labels [Tran et al., 2018; Qu et al., 2023; Antonio and Coello, 2013]. However, the effect of a single distance metric is less desirable for HDLSS FS problems [Happy et al., 2017]. During the variable-length process, the sample distribution dynamically changes with the search space, making single distance metrics like Manhattan distance potentially unreliable for consistent identification. [Huang et al., 2023].

Remarkably, in the study of VLEAs, we observed that a certain number of particles experience a sudden drop in performance after search space changes. This phenomenon has not been explicitly reported in prior VLEAs.

To this end, a novel multiscale adaptive and size-adaptive PSO algorithm (MASA-PSO) for gene expression analysis is proposed. It not only includes an adaptive weighting discrimination framework but also resolves the mismatch between population size and search space. Additionally, a hybrid elite strategy is proposed to mitigate the degradation of particles. The main contributions are as follows:

1. A novel multiscale-adaptive weighting framework is creatively proposed, which integrates three distance metrics to measure sample distributions for fitness evaluation, aiming at enhancing the algorithm's discrimination and stability.
2. Designing an adaptive division mechanism that initializes population based on a function measuring the matching degree between the population and the search space, addressing the imbalance of search capabilities among groups.
3. Discovering and analyzing the degradation of particles induced by search space variation, and proposing a hybrid strategy to alleviate this problem.
4. Extensive experiments on ten HDLSS gene expression datasets show that MASA-PSO outperforms compared stateof-the-art (SOTA) methods.

2 RelatedWork

2.1 Variable-Length Particle SwarmOptimization

As an important branch of EAs [Bonyadi and Michalewicz, 2017], PSO iteratively updates a swarm of particles in a Ddimensional search space, where each particle represents a candidate solution characterized by a position and a velocity. The quality of each particle's current candidate solution is evaluated by a fitness function. Guided by its local best (pbest) and the global best (gbest) among the swarm, particles move toward the promising regions and find the optimal solution based on the following equations,

where xti,d and vt i,d are the dth position and velocity of the ith particle at time t, respectively. pbestti,d and gbesttd are the local best position of the ith particle and global best position for the whole swarm in dimension d at time t. c1 and c2 are two positive constants. ω is the inertia weight. r1 and r2 are two random values uniformly distributed ranging from 0 to 1.

VLPSO [Tran et al., 2018] is the first length-adaptive PSO in high-dimensional FS, adopting the feature grouping from cooperative coevolution and a comprehensive learning strategy [Yu and Zhang, 2014] to accelerate convergence. Specifically, VLPSO relies on pbest alone for particle guidance and introduces a self-adaptive probability. It achieves a smaller feature subset and better classification accuracy in less time. However, VLEAs[Tran et al., 2018; Qu et al., 2023] ignored the population size in division which led to inefficient utilization of search resources. Yazdani et al. [Yazdani et al., 2023] suggested that there is a lack of a functional relationship between population and the search.

2.2 Distance metrics in high-dimensional FS

In high-dimensional FS, some EAs aim for higher accuracy and smaller feature subsets that introduced distance metrics. Most VLEAs [Tran et al., 2018; Qu et al., 2023] utilized the Manhattan distance as in Eq.(3).

Yuan et al. [Dubey and Saxena, 2016] applied cosine distance for FS as Eq.(4), which is also noteworthy.

Moreover, Huang et al. [Huang et al., 2023] showed that Euclidean distance as Eq.(5) may not be entirely ineffective in high-dimensional spaces.

2.3 Maximal Information Coefficient

Maximal Information Coefficient (MIC) is a correlation measure that features generality and equitability [Reshef et al., 2011]. It finds partitions schemes via grid search to maximize the mutual information, assessing the association between two variables. The formula is as follows,

where a, b are the partition numbers in directions of features F and labels C, with the constraint ab ≤ B(n). n is the dataset size. Ia,b(F;C) is the maximum mutual information under constraint a, b. MIC(F;C) ∈ [0, 1], where a higher value means a stronger correlation between F and C, and vice-versa. Typically, B(n) = n0.6 .

3 The ProposedMethod

This section elaborates the proposed MASA-PSO. As shown in Figure.1, the MASA-PSO is divided into three main parts. The initial part covers a filtering step to eliminate useless features and an adaptive division mechanism for initializing population that make it suitable for corresponding search spaces. In the second part, we propose a fitness with multiscale adaptive discrimination framework based on adaptive weighting of multiple metrics. The last part proposes a hybrid elite strategy to mitigate the particles degradation phenomenon in most existing VLEAs when search space changes.

3.1 Particle Initialization

Feature Ranking and Filtering In this study, the filtering threshold is MIC(⌈θr · m⌉), where m is the number of raw features. It denotes the value ranked in descending order at position ⌈θr·m⌉. Features with weak correlations to labels are removed. The remaining features form a set F in descending order based on MIC values. Note that θr = 0.7.

Adaptive Division and Population Initialization Existing VLEAs divides the population into several equal-size subpopulations for cooperation learning, which may lead to inefficient resource utilization when search space sizes vary significantly across different subpopulation [Yazdani et al., 2023]. Inspired by the S-shaped growth curve of species in constrained environments [Kucharavy and De Guio, 2011; Rangasamy et al., 2021], we design a novel adaptive division mechanism that measures the relationship between population and search space based on Eqs.(7)(8). By approximating the S-curve growth pattern, this mechanism dynamically adjusts initial population sizes according to each block's search space dimensions, thereby enhancing search efficiency.

where i, ˜ι is the block number and the median number of blocks, F is the number of features, M is the total number of blocks. Popmin is the minimum population size, Popup is the maximum increase number of populations. This function allows the search space to expand as the number of groups increases, with the corresponding population size growing accordingly until it reaches Popmin +Popup. (The whole operations see Algorithm 1)

3.2 Multiscale Adaptive Discrimination

VLEAs typically evaluate candidate solutions via a fitness function that combines classification accuracy with singledistance-based discrimination scores [Tran et al., 2018; Qu et al., 2023]. However, these single-distance metrics are difficult to maintain desired identification during search-space variation. To this end, we propose a fitness with the classification accuracy and a multiscale adaptive discrimination framework as Eq.(9). This framework dynamically weights Manhattan, Euclidean, and cosine distances to calculate an aggregation score. A logistic function is then constructed to maximize within-class aggregation while minimizing betweenclass aggregation.(Algorithm 2)

where γ is a weight set to 0.8, BACC is the balanced accuracy [Tran et al., 2016] for the imbalanced high-dimensional datasets as Eq.(10). Scoreagg is the aggregation score based on the adaptive weighting of three distances.

where C is the number of classes, TP c and Nc are the number of true positive samples and the total number of samples for the cth class, respectively.

Multiscale Adaptive Weighting Mechanism For a given feature subset, the distances between samples are calculated upon Eqs.(3-5), as distance sets {Dstl (si, sj ) | 1 ≤ i, j ≤ n, i ̸= j}, where l only denotes Manhattan, Euclidean, and cosine distance in this paper.

For each class c, the sample set is divided into the cclass set Sc = {sci |i = 1, 2, . . . , nc} and the non-c class set S¬c = {s¬c i |i = 1, 2, . . . , n¬c}, where nc and n¬c are the number of samples of Sc and S¬c. The distances of within-class sample pairs in Sc and of between-class sample pairs between Sc and S¬c are formulated as distance sets Dstl(Sc) = {Dstl(sci , scj ) | 1 ≤ i, j ≤ nc} and Dstl(Sc, S¬c) = {Dstl(sci , s¬c j )| 1 ≤ j ≤ n¬c}.

Algorithm 1 The particles initialization.

Input: Raw training set D, the number of blocks M, minimum population size Popmin, maximum population growth size Popup, the total number of features m, se-

lect threshold θr. Output: {Blocki|Blocki = {Xi,1,Xi,2, · · · ,Xi,pSizei }, i = 1, · · ·,M}. ▷ Swarm 1: CalculateMICj for each feature fj on D. ▷ Eq.(6) 2: Sort {fj} in descending order of {MICj}. 3: τ ←MIC(⌈θr · m⌉). 4: D← {fj ∈ D |MICj ≥ τ}. ▷ Filter step 5: for i = 1 toM do ▷ Adaptive division mechanism 6: Calculate feature space ParLeni of block i. ▷ Eq.(7) 7: Calculate population size pSizei of block i. ▷ Eq.(8) 8: for p = 1 to pSizei do ▷ Initializing swarm 9: for d = 1 to ParLeni do 10: posdi,p ←rand(0, 1). ▷ Position of particle Xi,p 11: if posdi,p > θr then 12: fd ← 1 13: else 14: fd ← 0 15: end if 16: end for 17: pbestp ← Xp, f itness(pbestp) ← f itness(Xp). 18: end for 19: end for 20: gbest ← argmax Xi,p {fitness(Xi,p)}. ▷ Initialize gbest 21: return {Blocki|Blocki = {Xi,1, · · · ,Xi,pSizei }, i = 1,· · ·,M}.

Then the within-class and between-class joint distance vectors for each sample sci is constructed as follows,

where Dstl(sci ,Sc) and Dstl(sci ,S¬c) are selected from Dstl(Sc) and Dstl(Sc, S¬c), representing distances between sci and samples of Sc and of S¬c, respectively. Within-class vector Dstw(sci ) is defined as the three maximum distance metrics in Dstl(sci , Sc), reflecting the clustering degree of class c centered on sci , while between-class vector Dstb(sci ) is the three minimum distance metrics in Dstl(sci , S¬c), indicating the distance from ¬c samples.

For all c class samples, the within-class matrix Xw(c) and the between-class matrix Xb(c) are constructed as follows,

where Xw(c) and Xb(c) are 3×nc matrices, with columns denoting the maximum within-class and the minimum betweenclass joint distance vectors, respectively.

Algorithm 2 The fitness calculation procedure.

Input: Training set S = {si|1 ≤ i ≤ n}, Classes of training

• set C, particle Xp, and outlier threshold θz. Output: fitness(Xp). 1: for each sample si ∈ S do ▷ Select features 2: Remove features not selected by particle Xp in si 3: end for 4: for l ∈{Eul,Man,Cos} do ▷ Calculate three distances 5: Calculate{Dstl(si,sj)| ∀(si,sj)∈S} via Eqs.(3-5). 6: end for 7: for class c ∈ C do ▷ Calculate the aggregation score 8: Sc←{sci | i = 1, . . . , nc}, S¬c ←S \ Sc. 9: Dstl(Sc)←{Dstl(sci , scj )| 1 ≤ i, j ≤ nc} 10: Dstl(Sc, S¬c) ← {Dstl(sci , s¬c j )| 1 ≤ j ≤ n¬c}. 11: for each sample sci ∈ Sc do 12: Obtain Dstw(sci ),Dstb(sci ) based on Eqs.(11)(12). 13: end for 14: Xw(c) ← {Dstw(sci ) | i = 1, 2, . . . , nc } ▷ Eq.(13) 15: Xb(c) ← {Dstb(sci ) | i = 1, 2, . . . , nc } ▷ Eq.(14) 16: Calculate δw(c) and δb(c) via Eqs.(15)(16). 17: end for 18: δw ← 1C ∑Cc=1 δw(c), δb ← 1C ∑Cc=1 δb(c). ▷ Average score 19: f ←- δw-δb len , Scoreagg ← 1 1+ef . ▷ Normalize scores 20: fitness(Xp)←γ·BACC+(1-γ)·Scoreagg. ▷ Eq.(9) 21: return fitness(Xp).

To integrate three distance metrics for collaboratively discriminating the dynamically changing sample distributions, while eliminating interference from correlations and dimensional inconsistency among these distance metrics, this paper develops a novel adaptive weighting framework based on precision matrix [Horn and Johnson, 2012] (Proof of decorrelation and dimension unity is in Supplementary A). Specifically, for each class c, the aggregation score of within-class δw(c) and the aggregation score of between score δb(c) are carried out via weighting operation as follows,

where k = {w, b}, µk(c) is the average joint distance vector for all samples in Xk(c). The precision matrix Pk(c) is the inverse of covariance matrix Σk(c) calculated by Eq.(16).

where the constrain nc > 3 ensures that the precision matrix can be obtained, and the main diagonal elements of the precision matrix measure the guidance ability of three distance metrics in the given feature space [Horn and Johnson, 2012].

The average within-class score δw and the average between-class score δb are calculated, based on δk = 1C ∑Cc=1 δk(c), where C is the number of classes. Inspired by the work [Tran et al., 2017], the fitness is integrated by maximizing δw, minimizing δb, and normalizing the score to [0, 1]. Since the precision matrix implies the information of features

Algorithm 3 The procedure of Elite hybrid strategy.

Output: {Blocki|Blocki ={Xi,1,Xi,2,· · · ,Xi,pSizei }, i =

1, · · ·,M}.

▷ New Swarm

• 1: E← argmax Blocki∈Swarm∑fitness(Xi,j),j=1,...,pSizei. ▷ Elite
• 2: for Blocki in Swarm do
• 3: for each Xi,j ∈Blocki do ▷ Compute forgotten value 4: Calculate the difference of fitness(Xi,j) before and after the search space changes, as fgj.
• 5: if fgj < 0 then ▷ Collect the degraded particles 6: SF ← SF ∪ {Xi,j}.
• 7: end if
• 8: end for
• 9: Sort SF by fgj in descending order.
• 10: EOL←The bottom ⌈θe· |SF |⌉ particles in SF .
• 11: EFL←The next ⌈θe· |SF |⌉ particles in SF .
• 12: R←The rest particles in SF .

13: EOL←EOLS(EOL, θs).

▷ EOL strategy

14: EFL←EFLS(EFL,E, θs).

▷ EFL strategy

• 15: Blocki ←EOL∪EFL∪R. ▷ Update block
• 16: end for
• 17: return Swarm.

[Horn and Johnson, 2012], a larger feature subset contains more information but hinder compact exploration, a regularization term len (the size of feature subset) is introduced to constrain the subset size.

3.3 Hybrid Elite Strategy for Degraded Particles

In experiments, we observed that many particles in VLEAs experience performance degradation as the search space shifts toward more promising regions. VLEAs alter particle lengths by truncating or appending features, causing deviations from the search direction. The search efficiency of these particles is impaired. Although it is inherently inevitable, a hybrid strategy based on elite forward learning (EFL) [Deng et al., 2020] and elite opposition learning (EOL) [Zhou et al., 2012] is proposed to alleviate it. Specifically, degraded particles of each block are divided among EOL, EFL and retention group R based on their degradation degree, where the most severely degraded ⌈θe · |SF |⌉ particles are assigned to EOL, the next ⌈θe · |SF |⌉ particles undergo EFL, and the remaining particles form R. The reinitialized particles from both elite strategies, combined with R, form a new population for each block. Note that θe is the elite select threshold, SF is the degraded particles set (Algorithm 3). The pseudocode of EFL and EOL strategy are in Supplementary B.

Elite Forward Learning Strategy For EFL group, the block with the highest average fitness is selected as the elite population E = {E1,E2, · · · ,En}. The selection probability of each feature is calculated as the ratio of times each feature is selected to the total number of occurrences of all features in E, the probability of each feature is as follows,

where Fnumi represents the times of feature Fi selected by E, d is the given subset size. A roulette wheel selection [Wang et al., 2020] is constructed to re-initialize particles based on these probabilities. The fitness of particles is evaluated before and after re-initialization, with only the higher fitness particles being retained.

Elite Opposition Learning Strategy For EOL group, since severe degradation after space changes indicates that the development value of their opposition space may exceed that of their original space [Zhou et al., 2012]. Hence, an elite opposition learning strategy is introduced to tackle variable-length forgetting. Let Pi = (pi,1, pi,2,· · ·,pi,d), i = 1, 2, · · · , n. represents the position of the ith degraded particles of Blocki. The opposition solution ˇ Pi is defined as follows,

where k ∈ U(0, 1) is a coefficient representing the magnitude of opposition, daj and dbj are the upper and lower bounds of Blocki in feature dimension j, as follows,

These particles are re-initialized based on the opposition solutions, and their fitness before and after initialization are compared. Particles with higher fitness are retained.

4 Experiments and Results Analysis

4.1 Experimental Setup

Benchmark Datasets 10 widely used gene expression datasets are selected to verify the performance of MASAPSO. These datasets are described in Table 1, which includes the ratio of the largest and smallest classes. Most of them are imbalanced, high-dimensional, and low-sample size that make classification particularly challenging and representative of many real-world bioinformatics and medical diagnosis problems. Experimental results on these datasets effectively demonstrate the performance of the proposed algorithm.

Baseline Methods To evaluate the performance of MASAPSO, this paper compares it with six SOTA methods in gene expression analysis, including VLPSO, ESAPSO, ManyObjective Jaccard-Based Evolutionary algorithm (JSEMO) [Saadatmand and Akbarzadeh-T, 2024], a Simple, Fast, and Efficient FS Algorithm (SFE) [Ahadzadeh et al., 2023], and an Improved Conditional Wasserstein GAN With Gradient Penalty (CWGANGP) [Han et al., 2025]. These methods involve VLEAs, other FS methods, and deep learning-based models, all of which have demonstrated competitive performance in gene expression analysis and are therefore representative baselines for comparison.(The comparative experiments with other SOTA methods and hyperparameter optimization are shown in Supplementary D and E)

Experiment Settings ALL baselines use the authors' recommended settings, while MASA-PSO parameters are detailed in Table 2. To ensure fairness, all FS methods employed K-nearest neighbor (KNN) with K = 1 classifier. Each dataset undergoes 30 independent runs using ten-fold cross validation. Leave-one-out validation on the training set evaluates candidate solutions. Performance is measured by the mean and standard deviation of BACC and the size of selected features. The Wilcoxon significance test [Derrac et al., 2011] with a 5% significance level is applied. All experiments are conducted on PC with Intel Core I7-14700KF @3.4GHz.

4.2 Experimental Results and Discussion

This section comprehensively presents the performance of MASA-PSO and five SOTA methods. Table 3 displays the results on ten gene expression datasets. " " denotes the size of final feature subset, "Best" represents the best result over 30 runs. "S" indicates the Wilcoxon significance test results, where "+" signifies a significant improvement of our algorithm over the baseline, "-" indicates that our method's BACC is significantly lower than the baseline, and "=" means no significant difference. (The computational and time cost analysis is detailed in Supplementary C)

(1) Comapred with CWGAN-GP, MASA-PSO achieves significantly better classification performance on most datasets, especially on highly imbalanced datasets (e.g., DLBCL, Bran Tumor1,and Lung Cancer).It suggests that the deep models are vulnerable to noise and prone to overfitting in HDLSS scenarios, while MASA-PSO effectively identifies informative feature subsets and offers robustness and interpretability, which is beneficial for subsequent gene analysis.

(2) Compared with SFE and JSEMO, MASA-PSO not only achieves superior classification accuracy on most datasets but also captures fewer features than these methods. For instance, on Leukemia1, MASA-PSO collects approximately 80 fewer features than SFE, while on 9 Tumors, it reduces the features by nearly 1,500 compared with JSEMO. These results indicate that the variable-length mechanism enables MASA-PSO to identify more compact feature subsets while maintaining or improving performance. (3) Compared with VLPSO and ESAPSO, MASA-PSO consistently achieves higher classification accuracy on most datasets (e.g., more than 2.18% increase on DLBCL, over 2.84% increase on Lung Cancer). Even on datasets where comparable accuracy is obtained, MASA-PSO selects significantly fewer features than these VLEAs, demonstrating its effectiveness in addressing the limitations of existing VLEAs. (A detailed comparison of optimization efficiency with VLEAs is in Supplementary E).

Overall, MASA-PSO demonstrates superior search capability over baseline methods, rapidly identifying better candidate solutions and finding more promising and smaller search space, thereby improving classification accuracy and capturing smaller feature subset. Meanwhile, it ensures the interpretability of the selected features for subsequent biological significance research.

4.3 Ablation Study

Experiments have verified that MASA-PSO outperforms the compared methods in classification accuracy, feature subset size and convergence speed. To further assess its effectiveness, ablation studies are conducted by adding or removing components. Figure.2 present the ablation results on six representative gene datasets (Results on more datasets are in the Supplementary of Figure.F4), where "D","M", and "H" denote the adaptive division, the mulitiscale framework and the hybrid elite mechanism, respectively. "+" means the inclusion of corresponding components. VLPSO is adopted as the basic algorithm, denoted as "V".

As Figure.2 shown, the introduction of D leads to varying degrees of BACC improvement. Combining the mechanism with other modules results in better BACC than when tested in isolation. For effect of M, it demonstrates the significant improvements for those applying the framework on most datasets, particularly on high-dimensional datasets (Prostate, and Lung Cancer). It indicates that M effectively adapts to dynamic search spaces, improving BACC and capturing smaller feature set. As for H, Figure.2 reveals significant BACC improvements across all datasets with the strategy, especially on DLBCL and Brain Tumor1, with increases of 7% and 4%.

4.4 Analysis of Selected Features

MASA-PSO is further examined through visualization and theoretical analysis. Figure.3 shows the distribution of selected features on DLBCL via t-SNE [Hinton and Roweis, 2002]. It shows that MASA-PSO successfully turns highdimensional features into a compact subset, enhancing between-class separability. Figure.4 reveals that more than 75% of features selected by MASA-PSO are in the significant region, with few representation from non-significant regions. It demonstrates the algorithm's capability to effectively identify biologically relevant differentially expressed features, rather than merely optimizing classification performance. (More analysis are in Supplementary G)

5 Conclusions

In this study, we propose a novel size-adaptive PSO algorithm for HDLSS gene expression analysis (MASA-PSO). a multiscale adaptive discrimination framework, an adaptive division mechanism, and a hybrid elite strategy are designed to enhance classification performance, improve search efficiency, and mitigate particle degradations. Experiments on ten gene expression datasets show MASA-PSO outperforms SOTA methods. Moreover, the selected feature subsets exhibit statistical and biological significance, facilitating the subsequent research and diagnosis. Future work will explore combining multiscale discrimination and variablelength mechanisms with deep learning models to advance bioinformatics and medical research.

Acknowledgments

This work was supported by the National Natural Science Foundation of China under Grant nos. 62576154 and

62102002.