Reliable Disentanglement Multi-view Learning Against View Adversarial Attacks

1 Introduction

In practical scenarios, an object can often be described by multiple views of different feature types and modalities, which leads to a growing interest in multi-view learning

* Corresponding authors

[Wen et al., 2021; Liang et al., 2024; Yuan et al., 2025; Yuan et al., 2024]. Thanks to the power of deep learning, deep multi-view learning has exhibited remarkable advantages by integrating and mining both valuable complementary and consistency information of multi-views [Qin et al., 2024; Zhang et al., 2020; Sun et al., 2024]. Thus, in recent years, multi-view learning has attracted widespread attention [Zhang et al., 2023; Xu et al., 2025; Li et al., 2025]. For example, predictive dynamic fusion (PDF) [Cao et al., 2024] proposes an intuitive and rigorous multimodal fusion paradigm from the perspective of generalization error.

Although these above methods have achieved pleasing performance, their results could be uncertain and unreliable due to the attribute differences and heterogeneity of multiview data. This greatly limits the application of multi-view learning in various fields, especially medical diagnosis or autonomous driving. To this end, a new trusted learning paradigm for multi-view classification is proposed to enhance trusted decisions. For instance, Trusted Multi-view Classification (TMC) [Han et al., 2022b] introduces the evidence theory to construct the Dirichlet distribution, thereby providing uncertainty estimation for multi-view decisions to enhance reliability. To make reliable decisions under noise labels, TMNR [Xu et al., 2024b] proposes trusted multi-view noise refining to overcome the negative effects of noisy labels.

Almost all existing trusted multi-view learning methods implicitly assume that multi-view data is secure [Wen et al., 2020]. In practice, however, in safety-critical applications such as autonomous driving and security monitoring, multiview data could be susceptible to adversarial attacks, which can deceive or disrupt multi-view models. This vulnerability inevitably leads to the adversarial unreliability problem (AUP) in trusted multi-view learning. As shown in Fig.1, adversarial attacks (such as projected gradient descent attack [Madry et al., 2018]) are imposed on the multi-view data. From the figure, we can observe that, after being subjected to attacks, even if only one view is attacked, the state-of-theart evidence-based method (i.e., ECML) still shows a significant decline in classification accuracy. Worse still, instead of increasing with the substantial performance decline, the estimated uncertainties are significantly lower than those in the clean setting. This indicates that the evidence-based uncertainty estimation mechanism fails under adversarial perturbations, thereby leading to the AUP.

To overcome the above AUP, we propose a novel multiview learning framework, namely Reliable Disentanglement Multi-view Learning (RDML). In the first stage, we present a perturbation-insensitive pretraining scheme to train an evidence extractor, thereby providing reliable category-level evidence and uncertainty estimation in subsequent stages. In the second stage, as shown in Fig.2, our method mainly consists of three key modules: the evidential disentanglement learning module, the feature recalibration module, and the evidential attention module. To be specific, RDML first proposes evidential disentanglement learning to decompose each view into clean and adversarial features under the guidance of pretrained corresponding evidence. Then, to prevent the negative effect of the adversarial features, we propose feature recalibration to recalibrate these feature units for additional discriminative information, thereby obtaining more robust features. Finally, to reduce the interference of stubborn adversarial features that are difficult to calibrate, we design a view-level evidential attention mechanism to enhance the robustness against adversarial features. The main contributions of this paper are as follows.

• This paper studies a less-touched adversarial unreliability problem (AUP) in trusted multi-view learning and proposes a new Reliable Disentanglement Multi-view Learning (RDML) framework against view adversarial attacks. To the best of our knowledge, for the first time, we address the AUP caused by view adversarial attacks.
• We propose evidential disentanglement learning to guide the stripping of adversarial information frommulti-view representations using a pretrained evidence model. To mitigate the interference of perturbations, we propose feature recalibration to rectify the weak adversarial features, and further present evidence attention to deal with the stubborn adversarial features.
• We conduct extensive experiments on six multi-view datasets to verify the effectiveness and robustness of our RDML under both adversarial and clean conditions.

2 RelatedWork

2.1 TrustedMulti-view Learning

Deep multi-view learning utilizes view-specific deep representations to enhance the integration and understanding of multi-source information [Geng et al., 2021; Peng et al., 2019; Xu et al., 2024c; Wen et al., 2023]. Han et al. point out that existing studies have placed too much emphasis on improving the performance of deep multi-view learning methods in various scenarios while neglecting to enhance the reliability of multi-view decision [Han et al., 2020]. They then propose the Trusted Multi-view Classification method. TMC transforms traditional classification networks into evidential neural networks (by replacing the last softmax function of the classification networks with Relu function to ensure that each output value is non-negative) and uses the extracted evidence representations to model the Dirichlet distribution [Sensoy et al., 2018; Jsang, 2018]. The Dirichlet distribution can output classification category probability and uncertainty, thereby achieving reliable multi-view decision. Finally, TMC introduces Dempster rule to fuse multi-view opinions.

Xu et al. introduce evidence learning into conflictive multiview learning, extracting view-specific opinion through the parameterized Dirichlet distribution [Xu et al., 2024a]. And a conflictive opinion aggregation method is designed for multiview fusion. Liu et al. propose an opinion fusion method based on evidence accumulation, in which the evidence representations of different views are accumulated to obtain the overall opinion [Liu et al., 2022]. Yue et al. discover the vulnerability of trusted multi-view approaches to adversarial examples and attributed this vulnerability to the difficulty of accurately assessing the quality of adversarial examples [Yue et al., 2025]. In this paper, we address the AUP from a more fundamental perspective. That is, we first enhance the adversarial insensitivity of the evidence neural network (perturbation-insensitive pretraining), and subsequently minimize the interference and harm that adversarial perturbations inflict on multi-view fusion and decision to the greatest extent possible (evidence-based disentanglement, feature recalibration and view-level evidential attention).

2.2 Deep Adversarial Defense

Improving the robustness of deep neural networks against adversarial perturbations has long been a goal in deep learning community [Long et al., 2022; Wang et al., 2022; Kurakin et al., 2018]. Since there are different defense strategies for different types of attacks, here we mainly focus on deep defense methods against white-box attacks. In recent years, adversarial training has been widely proven to be effective in enhancing the adversarial robustness of neural networks [Goodfellow et al., 2015; Ilyas et al., 2019]. By using both clean samples and adversarial samples as training data, the insensitivity of neural networks to adversarial perturbations can be enhanced.

Moreover, since disentanglement learning is good at separating information, it is naturally suitable for dealing with perturbed samples [Kim et al., 2023; Liu et al., 2024; Zhang et al., 2024]. Typically, disentanglement learning decomposes perturbed samples into clean and adversarial features to mitigate the interference caused by adversarial perturbations. However, existing disentanglement learning methods lack the support of effective clues and trusted guidance, and thus the adversarial robustness is limited. In this paper, we propose an evidence-based disentanglement method to resist adversarial perturbations. The disentanglement is guided by a pretrained evidence extractor, so it can improve the reliability and adversarial robustness of the evidential networks.

3 Method

3.1 ProblemDefinition

Suppose that there is a multi-view dataset DVN with N instances and V views, and xvn ∈ Rdv (n = 1, 2, ...,N) is an instance from DVN . dv is the dimension of the v-th view. The corresponding class labels are {yn}Nn=1 . The number of classes is K. The random view adversarial attacks are conducted to each multi-view instance by the following formula,

where dv is the dimension of the v-th view. ̂ xvn means the adversarial version of xvn . Our goal is to learn a robust evidential model against view adversarial attacks.

3.2 Overview

When facing adversarial attacks, existing trusted multi-view methods can easily lead to the AUP, thereby weakening the performance of multi-view learning models. To address the above problem, in this paper, we propose Reliable Disentanglement Multi-view Learning (RDML) against view adversarial attacks, which consist of two stages. In the first stage, we propose a perturbation-insensitive pretraining scheme to enhance the stability and adversarial insensitivity of evidential neural networks. To be specific, this approach introduces random view attacks into the pretraining of multi-view evidential networks, thereby making the evidence extractor provide the support of reliable category-level evidence and uncertainty estimation. The objective function could be formulated as follows:

where the LECL(αn) is the evidential classification loss; αn is the Dirichlet parameter; LACL denotes the adversarial consistency loss.

In the second stage, RDML first proposes evidential disentanglement learning. To be specific, we use the pretrained evidence extractor to analyze the features to be decoupled and map the extracted category-level evidence into a robustnessaware soft mask. The higher the score of the mask for a feature, the more likely the feature is to be a robust (or clean) feature, and vice versa, it is more likely to be an adversarial feature. Afterward, we utilize the mask to separate clean and adversarial features. In addition, we believe that some weak adversarial features could easily be converted into clean features. To this end, we propose feature recalibration to correct these adversarial features into clean features. For the remaining part of stubborn adversarial features, we design a viewlevel evidential attention mechanism to reduce the interference of these adversarial features that are difficult to correct, thereby enhancing the robustness against adversarial features. The objective function could be expressed as

where LEDL and LFRL represent evidential disentanglement loss and feature recalibration loss, respectively; γ1 and γ2 are two balancing parameters.

3.3 Perturbation-insensitive Pretraining

Although evidential neural networks can provide corresponding uncertainties to enhance the ability of trusted decisions, they could become unstable and even ineffective after being subjected to adversarial attacks. To overcome this issue, we propose a perturbation-insensitive pretraining strategy that incorporates adversarial samples to enhance the adversarial insensitivity of the evidence extractor Ept(·) = {Ev pt (·)}Vv=1 , thereby providing robust evidence support. To be specific, for adversarial multi-view data, to enhance the robustness of the evidence extractor for adversarial attacks, we then mix the adversarial multi-view samples with clean samples and utilize the evidence extractor for the mixed samples xmix to extract the corresponding classification evidence e, i.e.,

where K denotes the number of classes. Afterward, we obtain the parameter αvk = evk + 1 required for modeling the Dirichlet distribution based on the evidence. The viewspecific opinion ov = (bv , uv ) are also obtained according to following formula,

where b ≥ 0 and u ≥ 0 denote the belief mass and uncertainty (∑Kk=1 bvk + uv = 1), and Sv = ∑Kk=1 αvk represents the Dirichlet strength. Given the Sv and αvk , the probability of the v-th view for the k-th class is pvk = αvk Sv . After that, Dempster rule is used to combine multi-view opinions, and the joint opinion is o = o1 ⊗ o2 ⊗ ... ⊗ ov , where ⊗ is the Dempster rule based fusion operation.

To optimize the evidence extractor, following [Han et al., 2020], we use the evidential classification loss LECL(αn), i.e.,

where ψ(·) denotes the digamma function; ˜αn = yn + (1 - yn)⊙αn is the adapted Dirichlet parameter avoiding penalizing the evidence of the correct category to zero. LECE(αn) represents the evidential cross-entropy loss, which requires the model to extract more evidence for the correct category compared to other categories; and the Kullback-Leibler (KL) divergence restricts the model to extract as little evidence as possible from the incorrect categories.

Besides, since the instances in AMVL are subjected to random view attacks, we hope to enhance model robustness against adversarial attacks by constraining the differences among the predicted probability distributions of different views. Therefore, we construct the following adversarial consistency loss LACL, i.e.,

Therefore, the well-trained evidence extractor will guarantee robust and stable evidence in subsequent steps with frozen parameters.

3.4 Evidential Disentanglement Learning

Although existing disentanglement learning methods can be used for decomposing adversarial and clean features by various meticulously designed losses, the lack of support for effective clues and trusted guidance hinders their performance. To improve the credibility of separating adversarial features, we introduce the evidential learning theory to naturally provide evidence and uncertainty for the decision process. This characteristic can significantly promote reliable feature decomposition. To this end, we design an evidence disentanglement learning module.

In the training stage, given a clean multi-view instance x = {xv ∈ Rdv }Vv=1 , the random view attack is conducted on it, and its adversarial version is ̂ x. Different from existing disentanglement learning methods, we leverage the pre-trained evidence extractor Ept(·) to extract adversarially-insensitive evidence from the adversarial samples. Due to strong adversarial insensitivity, pretrained evidence extractor Ept(·) can extract effective evidence under adversarial perturbation, i.e.,

where emv ∈ RK is the evidential map, which implies the category-aware evidence of the v-th view of adversarial sample ̂ xv . Since emv is category-level evidence, we construct a multi-layer perceptron (MLP) based evidence mapping layer fEM,1(·) = {f vEM,1 (·)}Vv=1 to map the category-level evidence emv into a feature-level robustness map rmv = f vEM,1 (emv ) (rmv ∈ Rdv ), which indicates the amount of evidence for features in each dimension. Features containing more evidence are regarded as clean features, while those containing less are considered adversarial features. Subsequently, in order to facilitate the decomposition of these two types of features, we introduce Gumbel softmax [Jang et al., 2022] to convert the robustness map into a soft mask score mv , i.e.,

where each value is a non-negative value less than 1. σ(·) represents a Sigmoid function for normalization; q1 and q2 are two instances sampled from Gumbel distribution (given u ∼ Uniform(0, 1), q = - log(- log(u))); µ is a temperature coefficient. mv ∈ Rdv is a feature-level mask score and each dimension of the feature has a score ranging from 0 to 1, a higher score indicates a higher probability of being a clean feature.

Therefore, we can decompose clean and adversarial features via the following simple feature-level multiplication,

where hvc and hva are the clean feature of xv and the adversarial one, respectively.

To optimize our evidential disentanglement learning, an evidence disentanglement loss is designed. Evidential disentanglement learning utilizes the evidence output by a pretrained evidence extractor to generate a soft robustness mask. Therefore, we expect that the distribution of the decoupled clean features is as close as possible to the real distribution, while the distribution of adversarial features is in contrast. Given clean and adversarial feature of the n-th instance hc,n, ha,n, the evidence disentanglement loss can be written as

where yn is the ground truth label of hn; ̂ yn is the label of a wrong class for hn; pvc,n = f vEC (hvc,n ), pva,n = f vEC (hva,n ) are the classification probabilities of hvc,n and hva,n ; fEC(·) = {f vEC (·)}Vv=1 is a group of evidential classifiers where an activation function like Relu is added after each classifier. Note that fEC(·) only participates in the training phase and is not utilized in the pretraining and testing phase.

3.5 Feature Recalibration

For the adversarial features, we believe that a part of them can be transformed from the clean features with relatively poor robustness, and this part of features also easily has the potential to be recovered back into clean features. Therefore, we construct an MLP based feature recalibration layer fFC(·) = {f v FC (·)}Vv=1 to rectify weak adversarial features to clean and informative features,

where hvcr represents the corrected feature of ̂ xv . Then we can get the final feature hvf = hvc +hvcr . For the feature recalibration module, we expect that the corrected representations can provide as much informative and valuable features as possible for classification. Therefore, the predicted probability distribution of the corrected representations is required to be as close as possible to the ground truth distribution. Thus, we can have the following loss, i.e.,

3.6 Evidential Attention

Though decoupling adversarial features and repairing weak adversarial features can alleviate the interference of adversarial perturbations on evidential neural networks to some extent, the impairment brought about by stubborn adversarial features remains difficult to mitigate effectively. Thus, we propose a view-level evidential attention mechanism that generates the evidential attention score by conducting a robust analysis of view features using the pre-trained Ept(·). This mechanism guides the model to focus on the informative clean features and ignore the interference of strong adversarial features that are difficult to utilize. Specifically, we utilize the pretrained evidence extractor Ept(·) to conduct evidence analysis on the feature hf . The extracted evidence is transformed into evidential attention scores via a softmax function. Benefiting from the knowledge of the pre-trained evidence extractor, these attention scores imply the robustness of each dimension of hf , distinguishing clean, weak adversarial, and hard adversarial features in the form of scores. Since the evidence extractor outputs category-level scores, we map them into feature-level attentions through the evidence mapping layer fEM,2(·) = {f vEM,2 (·)}Vv=1 , i.e.,

hvf,i denotes the i-th element of hvf ; attv ∈ Rdv is the viewlevel evidential attention for hvf . The augmented feature is obtained via hvaug = hvf ⊙ attv .

3.7 TrustedMulti-view Fusion

Followed by [Han et al., 2020], Dempster combination rule is introduced for multi-view fusion. Given two augmented features of two views h1aug , h2aug , the corresponding evidences e1 = E1 c (h1aug ), e2 = E2 c (h2aug ) can be extracted via a evidence extractor Ec(·) = {Ev c (·)}Vv=1 . It is worth mentioning that the parameters of Ec(·) are copied from the pretrained Ept(·), so as to improve the overall training efficiency. Then two opinions o1 = (b1 , u1 ) and o2 = (b2 , u2 ) are constructed via Eq. (5). After that, we have joint opinion o = o1 ⊗ o2 = (b, u) where

M = ∑i̸=j b1i b2j represents the difference between two opinions. And 1 1-M is used for normalization. According to the above combination pattern, we have the joint multi-view opinion o = o1 ⊗ o2 ⊗ ... ⊗ ov = (b, u). Then the joint evidence ek = bk × S, Dirichlet parameter αk = ek + 1 and Dirichlet strength S = Ku are obtained based on Eq. (5).

4 Experiments

4.1 Datasets and Competitors

To verify the effectiveness and robustness of our method, we conduct experiments on six multi-view datasets, including PIE [Gross et al., 2010], Scene [Fei-Fei and Perona, 2005], Leaves [Cope et al., 2013], NUS-WIDE [Chua et al., 2009], MSRC [Xu et al., 2016], and Fashion [Xiao et al., 2017]. In addition, we compare our RDML method with eight state-ofthe-art multi-view learning methods, including four evidencebased methods (i.e., TMC [Han et al., 2020], ETMC [Han et al., 2022b], ECML [Xu et al., 2024a], and TMNR [Xu et al., 2024b]), and four other uncertainty/confidence based methods (i.e., DUANets [Geng et al., 2021], MMD [Han et al., 2022a], QMF [Zhang et al., 2023], and PDF [Cao et al., 2024]. The details of all datasets and methods are shown in Appendix.

4.2 Implementation Details

Our experiments are conducted based on the PyTorch 2.4.1 framework with an Nvidia RTX 3090 GPU. For all datasets, 80% of the samples are used for training (for our method, these data are also used for pretraining), and 20% of the samples are used for testing. All experiments will be run 5 times, and we will report the average performance and standard deviation based on the accuracy of each test (after the last training epoch). The pretraining epoch of Ept(·) is 1000 with a batch size of 500, and the training epoch is 500 for the cleaning setting and 400 for the adversarial setting. The learning rate is selected from [0.003, 0.005]. Ev pt (·) and Ev c (·) are with the size of [dv,K]. f vEM is with the size of [K, dv]. And f vEM is with the size of [dv, dv]. Adam is used as the optimizer. The temperature µ of Gumbel softmax is set as 0.1. We use Projected Gradient Descent for random view attack. The number of attack iterations is 10 with a maximum perturbation range of 8/255.

4.3 Experimental Results

To comprehensively evaluate the robustness of all methods, we conduct experiments on clean data and under adversarial attacks respectively. According to Table 1 and Table 2, we have following observations.

• For clean multi-view data, RDML achieves the best performance in most cases, with an average improvement of 1.41% compared to the best baselines. We attribute this to two reasons. First, the pretrained evidence extractor

can provide good parameter initialization for the evidential classifier, avoiding falling into local optima prematurely during the training process. Second, evidential disentanglement learning, especially the combination of evidence-based disentanglement and View-specific evidential attention mechanism, is proficient in extracting multi-view features that are beneficial for classification and ignoring the interference of redundant information.

• RDML demonstrates significant advantages in AMVL. RDML has an average improvement of 6.36% compared to the second best methods (including AT based methods). Unlike existing evidence learning methods, RDML obtains a robust evidence extractor through perturbation-insensitive pretraining. Moreover, evidence-based disentanglement is adept at separating clean and adversarial features. Weak adversarial features are then repaired by the feature recalibration module, while the View-specific evidential attention can shield the interference of hard adversarial features on multiview classification. Therefore, RDML is effective and robust under adversarial perturbations.
• Both evidence based methods and other types of methods are highly vulnerable to adversarial attacks. Although adversarial training can, to some extent, relieve

the sensitivity of multi-view models to adversarial attacks in many cases, due to the lack of an appropriate feature disentanglement mechanism, a large amount of adversarial and uninformative features are aggregated during multi-view fusion process, which impairs the classification performance.

4.4 Uncertainty Analysis

To study the effectiveness of the uncertainty estimation mechanism of our method under adversarial perturbation, we visualize the estimated uncertainty density in more adversarial scenarios. As we can observe from Fig.3, when only a few views are attacked, the estimated uncertainty can be well matched with that in the clean setting. However, when more views are attacked, the estimated uncertainty gradually increases. This phenomenon, on the one hand, indicates that the uncertainty estimation mechanism of our method is effective and robust under adversarial conditions. On the other hand, it also reveals that the uncertainty estimation mechanism is affected by the quality of the views. The worse the view quality, the more difficult it is to make accurate decisions. This further demonstrates the effectiveness of our evidential disentanglement learning and feature recalibration module, which are designed to decouple and repair adversarial and informative features, significantly improving the view quality.

4.5 Parameter Sensitivity Analysis

Here we study the influence of two key hyperparameters, γ1 and γ2, on model robustness. There are two points we can get from Fig.4. (1) In most cases, the accuracy rises slowly as two values increase. It reaches the optimal performance when the values are 1 or 2, and then declines slowly. Overall, our method is insensitive to parameter changes, demonstrating its robustness. (2) When the values of γ1 and γ2 are too small (0.01) or too large (10), the accuracy decreases significantly. On the one hand, this further validates the effectiveness of the evidential disentanglement loss and feature recalibration loss (γ1 and γ2 are their respective balancing coefficients). On the other hand, it also shows that overemphasizing either decoupling adversarial features or repairing adversarial information will undermine the robustness. Striking an appropriate balance is the key to achieving better performance.

4.6 Ablation Study

We conduct ablation experiments to verify the effectiveness of key components of RDML. As shown in Table 3, after removing the pretrained Ept(·) and evidence-based disentanglement respectively (the evidence attention mechanism and LEDL then get invalid automatically), the model robustness significantly decreases. This indicates that: (1) perturbationinsensitive pretraining is highly effective in helping the evidential neural network resist adversarial attacks. (2) evidence-based decoupling can effectively strip the adversarial perturbations from view representations, and effectively reduce the interference of adversarial and uninformative features. After removing the feature recalibration and evidential attention respectively, the model performance shows a decline to varying degrees in most cases. In addition, we also explore the effectiveness of three proposed losses. The results prove that the three losses, especially LEDL and LFRL, are able to improve the model robustness in most cases.

5 Conclusion

In this work, we reveal and study the AUP in trusted multiview learning. To this end, we propose a novel Reliable Disentanglement Multi-view Learning framework. Specifically, RDML designs an evidential disentanglement learning to separate clean and adversarial features, and this process is guided by a pretrained evidence extractor. To mitigate the interference of adversarial features on multi-view decision, an adversarial feature recalibration module and an evidential attention mechanism are proposed. Experiments conducted on six datasets show the effectiveness and robustness of RDML against view adversarial attacks.

Acknowledgments

This work is supported in part by the National Natural Science Foundation of China (Grant no. 62372315), Sichuan Science and Technology Planning Project (Grants nos. 2024NSFTD0049, 2024ZDZX0004, 2024YFHZ0089), the Chengdu Science and Technology Project (Grant no. 2023-XT00-00004-GX), and the Sichuan Science and Technology Miaozi Program (Grant no. MZGC20240057).